Questions tagged [hpc]
High Performance Computing (HPC) refers to the use of supercomputers and computer clusters to solve a wide range of computationally intensive problems.
1,578
questions
0
votes
1
answer
53
views
Gnu-Parallelization within a shell script
I am using GNU parallel to run 3 scripts simultaneously. This is my current shell script.
#!/bin/bash
#SBATCH --array=0-0
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=3
#SBATCH --cpus-per-task=1
#...
- 146
0
votes
0
answers
31
views
How to use multithreading to traverse a quadtree built from a point cloud model
The simplest description of this problem is how to traverse a quadtree using multiple threads.
The following figure is the quadtree model of my problem：
My algorithm does not need to traverse all ...
- 13
1
vote
1
answer
49
views
Uploading and downloading files to and from a HPC
I am using Compute Canada's HPC cluster and want to download files from the cluster to a local directory. I referred to Uploading and downloading files to and from a HPC for instructions and entered ...
- 11
0
votes
0
answers
62
views
Merge several rasters into one large raster file in R with 'terra'
I have several rasters (.tif) files and want to merge them into a final large raster. Each smaller raster file is also in relatively large size.
I used the terra R package's vrt function to first ...
- 7
1
vote
1
answer
33
views
Run one program with different arguments in parallel with SLURM
I have a program which doesn't support multi threading. And I want to run it multiple times with different arguments in parallel. Since i have 1000 CPUs available I want it to run on different nodes ...
- 11
0
votes
1
answer
23
views
How can I let higher priority Slurm jobs pass through while not sharing individual CPUs among tasks?
My IT department kindly set me up with a Linux VM with Slurm and I'm learning commands. I'm changing numbers a bit, but let's say there are 2 nodes each with 5 cores/CPUs (I think they're the same ...
- 1
0
votes
1
answer
54
views
mpfr_t in custom MPI structs
I am working on converting my program to use arbitrary precision and thus I started utilizing MPFR. I had a custom MPI struct containing my data in my original code. Now that I have converted my data ...
0
votes
0
answers
21
views
MLFLOW Artifacts stored in remote storage not showing on UI
Set up:
tracking server openstack instance
mlflow tracking database on second openstack instance
traring on on-prem HPC node
Tracking server start
python3 -m mlflow server --dev --app-name oidc-auth ...
- 315
0
votes
0
answers
34
views
METIS fails to partition a 297 nodes graph into 64 parts : bisect a graph with 0 vertices
So like said in the title, I'm trying to partition a graph using METIS. More precisely, I have a mesh that has 297 elements, so those are the 297 nodes of the graph and I want to partition it into 64 ...
- 355
0
votes
0
answers
10
views
Not able to connect to the remote server using 'simple-ssh' package of nodejs
I have a problem connecting to the remote server using the 'simple-ssh' package of nodejs
There are two issues that I am getting:
Either the error shows up as => "Error: Timed out while ...
0
votes
1
answer
35
views
Passing a string argument to bash causes syntax error
I am trying to run a bash script for rescaling fMRI data using AFNI's 3dcalc.
One of the arguments for this function is "expr" which will "Apply the expression - within quotes - to the ...
0
votes
0
answers
25
views
MLflow Tracking Server on HPC
I am considering to use MLflow in the HPC cluster from my university. However, I am concerned about what to use as a tracking server. When using MLflow on my terminal, I use localhost.
I heard there ...
0
votes
0
answers
16
views
Is there any way to get the MPE (MPI Parallel Envrionment) SDK?
I want to install the MPE SDK to help me debug my MPI project, but when I turn to this link:https://www.mcs.anl.gov/research/projects/perfvis/download/index.htm#MPE
I find that all the version of MPE ...
0
votes
1
answer
38
views
Running module commands in srun
I run into the same issue as where "module" commands are not accessible from within srun.
SLURM "srun module load" command exits with error
The answer there explains that the ...
- 265
1
vote
0
answers
34
views
Snakemake fails with executor SLURM with slurm_persist_conn_open_without_init
PROBLEM
When I use snakemake with a cluster profile and leave it running overnight, trying both with a screen process or with a nohup snakemake ... & detachment, I will run into the following ...
