Building and running SOFI3D

SOFI3D is used for processing three dimensional seismic data.

Building SOFI3D

1. Clone the git repository where SOFI3D is located

   git clone https://git.scc.kit.edu/GPIAG-Software/SOFI3D.git

2. Load one of the MPI GCC Modules

   module load mpich/ge/gcc OR module load openmpi/gcc

3. Unload the default C compiler

   module unload gcc

4. Change to the SOFI3D src directory

   cd SOFI3D/src

5. Run the make all command and watch out for errors.

If you don't get any errors, SOFI3D has been successfully built.

• NOTE: There may be some warnings about variables being set but not used. This should be fine.

Running SOFI3D on a Cluster

When running in parallel SOFI will decompose the domain you are working on and split it into smaller chunks for each processor to work on. The number of chunks will depend on the number of nodes and tasks you set in the submission file (see below) and in the SOFI3D/par/in_and_out/sofi3D.json parameter file. It is recommended to split the domain into relatively square/cubic sections.

The parameters file creates by default will not necessarily work with the settings you want to use to submit the job to the queue. You will need to edit it before attempting to submit to the cluster. The default parameters file can be found at SOFI3D/par/in_and_out/sofi3D.json. Under Domain Decomposition, the variables NPROCX, NPROCY, and NPROCZ must have their variables changed such that NPROCX * NPROCY * NPROCZ = Number of processors requested.

• For the example submission script below NPROCX * NPROCY * NPROCZ = 8 = (4 tasks per node) * (2 nodes)

Use the following script as a template submission script. Adjust paths and resource requirements where necessary.

SOFI3D/par/in_and_out/sofi3D.json:

#-----------------------------------------------------------------
#      JSON PARAMETER FILE FOR SOFI3D
#-----------------------------------------------------------------
# description: example of json input file
# description/name of the model: homogeneous full space (hh.c)
#

{
"Domain Decomposition" : "comment",
                        "NPROCX" : "2",
                        "NPROCY" : "2",
                        "NPROCZ" : "2",
}

Sbatch submission script sofi3DSubmission.sh:

#!/bin/bash
#SBATCH --ntasks-per-node=4
#SBATCH -N2
#SBATCH --job-name=SOFI3DTest
#SBATCH --workdir=./
#SBATCH -o /home/first.last/SOFI3D/Log.txt
#SBATCH -e /home/first.last/SOFI3D/Errors.txt

module purge
module load slurm
module load mpich/ge/gcc
cd /home/first.last/SOFI3D/par
srun -n $SLURM_NTASKS hostname > $SLURM_JOB_ID.hosts
mpirun -n $SLURM_NTASKS -machinefile $SLURM_JOB_ID.hosts ../bin/sofi3D ./in_and_out/sofi3D.json

After you've created your submission script and modified the sofi2D.json parameters file you can submit your job using sbatch.

sbatch sofi3DSubmission.sh

See the Slurm tutorial for more instructions on how to submit jobs.