7.14 PWFA

[TOC]

7.14.1 A Brief Introduction of the Slurm Cluster

7.14.1.1 register

• See application for your cluster user name to get registered in the cluster.

7.14.1.2 Log in

• ssh command is used to log in the cluster, <user_name>stands for your real user name.

# lxslc7 : shared login nodes
ssh <user_name>@lxslc7.ihep.ac.cn

# pvm057 : dedicated login nodes for group pwfa
ssh <user_name>@pvm057.ihep.ac.cn

7.14.1.3 Cluster usage

• There are two ways to use cluster
  • Interactive mode : Log into the node pvm057 and debug your program.
  • Job mode : submit slurm jobs with command sbatch and run your fine-tuned program.
    • Jobs can be submitted to partition pwfadebug and spub.
• More detailed usage can be found in the Slurm User Manual

7.14.2 Available computing resources

• The PWFA experiment is renting computing services of the spub computing platform.
• Available computing resources are in the Slurm cluster and can be classified into three catagories.

7.14.3 Accessable storage

• /ihepfs is used as data or home directory for the PWFA experiment, details can be found in the following table.
• <user_name> stands for your cluster user name

7.14.4 Installed software

7.14.5 Slurm job samples

• Users of the PWFA expriments can submit to partition pwfadebug and spub
• To submit jobs, some job parameters including partition, account and qos must be provided.

7.14.5.1 QuickPIC

• The following sample job script qpic_sample_job.sh show how to submit a QuickPIC job to partition spub.

$ cat qpic_sample_job.sh
#! /bin/bash

#================= Part 1 : job parameters ============
#SBATCH --partition=spub
#SBATCH --account=pwfa
#SBATCH --qos=spubpwfa
#SBATCH --ntasks=64
#SBATCH --mem-per-cpu=4GB
#SBATCH --job-name=quickPIC

#================ Part 2 : run software ===============
source /cvmfs/pwfa.ihep.ac.cn/software/env/QuickPIC_env.sh
mpirun -np 64 /cvmfs/pwfa.ihep.ac.cn/software/QuickPIC-OpenSource-dev/source/qpic.e

Explanation:

• source /cvmfs/pwfa.ihep.ac.cn/software/env/QuickPIC_env.sh is use to set environment variables for QuickPIC
• qpic.e is the excecutable program of QuickPIC, the defauld input file is qpinput.json which should be put in the current directory to submit jobs.
• Details about qpinput.json can be found with the link of Input File for QuickPIC .

• To submit QuickPIC job, type the following command

$ sbatch qpic_sample_job.sh

7.14.5.2 fbpic

• The following job script fbpic_sample_job.sh shows how to submit a fbpic job to partition spub.

$ cat fbpic_sample_job.sh
#! /bin/bash

#================= Part 1 : job parameters ============
#SBATCH --partition=spub
#SBATCH --account=pwfa
#SBATCH --qos=spubpwfa
#SBATCH --job-name=fbpic
#SBATCH --ntasks=32
#SBATCH --mem-per-cpu=4G

#================ Part 2 : run software ===============
export PATH=/cvmfs/slurm.ihep.ac.cn/centos7.9/anaconda3/bin:$PATH
source activate fbpic
mpirun -np 32 python lwfa_script.py

Explanation：

• Software fbpic is installed in a anaconda env中，source activate fbpic is used to set environment variables.
• lwfa_script.py is used to run simulations，sample lwfa_script.pycan be found with the. link of lwfa_script.py.

• 作业提交命令为

$ sbatch fbpic_sample_job.sh