Questions tagged [parallel-processing]
Parallel processing is, in sharp contrast to just a Concurrent processing, guaranteed to start / perform / finish all thread-level and/or instruction-level tasks executed in a parallel fashion and provides a guaranteed finish of the simultaneously executed code-paths.
parallel-processing
21,820
questions
0
votes
0
answers
16
views
A generic question in CUDA (possibly straightforward for an expert)
I have an elementary understanding of how GPUs work i.e. SMs, warps, GPU memory hierarchy, threads, blocks, etc, and am familiar with some relatively more advanced concepts (but never got to use them) ...
- 174
0
votes
1
answer
25
views
Read/Write pipeline in pandas
I'm building a simple pipeline to pull a dataset from a database and write it to a csv so that I can access the data more quickly in the future.
Currently, I have this:
# data loading as a pipeline
# ...
- 305
0
votes
1
answer
20
views
Parallel Computing and usage of CPUs
I have a conceptual question. I have a MATLAB code for simulating a beam propagation (optics) which has lots of for loops inside it but every iteration in a loop depends on the previous one (n+1 th ...
- 11
0
votes
0
answers
45
views
How can I add parallelism to nested loops here?
I am currently developing a c++ minecraft clone in sfml and am trying to implement parallelism for rendering and modifying the chunks using a thread pool. The main issue I am currently facing is how ...
- 7
0
votes
0
answers
12
views
Optimize NAMD2 on multiple nodes using OpenMPI
I'm currently running NAMD2 on a server with EPYC CPUs using the specifications below:
...
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=128
#SBATCH --cpus-per-task=1
#SBATCH --mem=...
- 13
0
votes
0
answers
45
views
Parallel computing is disabled in Quanteda CRAN version. How can I enable Parallel computing in M3 Mac OS X
I am having problem with enablingh quanteda's parallel computing in R on M3 Macbook. In GitHub, the readme says
Windows or macOS users do not have to install TBB or any other packages to enable ...
0
votes
1
answer
60
views
Gnu-Parallelization within a shell script
I am using GNU parallel to run 3 scripts simultaneously. This is my current shell script.
#!/bin/bash
#SBATCH --array=0-0
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=3
#SBATCH --cpus-per-task=1
#...
- 146
0
votes
2
answers
66
views
Bash loop on two sets of files simultaneously
I have two sets of files in the same directory :
1.bam
2.bam
3.bam
and
1.txt
2.txt
3.txt
I need to run a command where 1.bam and 1.txt are interpreted together, and so on..
I tried looping through ...
- 443
2
votes
1
answer
62
views
Multiprocessing shared memory to pass large arrays between processes
Context: I need to analyse some weather data for every hour of the year. For each hour, I need to read in some inputs for each hour before performing some calculation. One of these inputs is a very ...
- 21
0
votes
0
answers
19
views
Why did I use `TParallel.&For` in Delphi, but the performance was not improved at all, but was slower than running a thread?
I'm currently learning parallel programming in Delphi. When I tried to use TParallel.&For to optimize a for loop, I found that the result was not what I wanted. The result was even worse than ...
- 3
1
vote
1
answer
32
views
parpool MATLAB running in the command-line on a slurm cluster
I've been running a matlab script in a cluster and I got a warning saying that my MATLAB programm was comsuming in excess 500% CPU, so somehow the MATLAB access the head-node.
I've been submitting my ...
0
votes
0
answers
38
views
Cannot run mclapply with oaxaca function in my dataset (parallel jobs did not deliver results) [closed]
Parallel is not working for my data. I have tried one subdata at the time (non-parallel) and all of them work.
There are two problems:
mclapply only delivers results for one of the runs (which one it ...
- 7,113
1
vote
2
answers
76
views
How to produce multiple outputs in TPL Dataflow block before completion
I have this case where I am trying to build a pipeline using TPL Dataflow that could process a big number of items. At some point in the pipeline I need to batch items and send to the next block after ...
- 446
1
vote
2
answers
95
views
How to efficiently perform parallel file search using pathlib `glob` in Python for large directory structures?
I'm working on a Python project where I need to search for specific files across a very large directory structure. Currently, I'm using the glob (or rglob) method from the pathlib module, but it is ...
- 583
-1
votes
0
answers
16
views
Taking input from an Action Classifier Model in UE5
I have trained an action classifier model that classifies players action using web cam in real time.
It uses media pipe to detect the pose of the person and feeds the model with x, y, depth and ...
