11.2 Cluster Using Rules
1. HTCondor Job System
INFO
At present, parallel (such as multithreading and MPI) and GPU jobs are not supported in IHEP HTCondor cluster. Please refer to the Slurm Job System in case of any requirements for parallel jobs.
In HTCondor, jobs from different experiments are partitioned by user groups, which is similar to the job queue. User's primary OS group is used by default with HepJob. However, we can appoint a different one via the parameter of hep_sub.
| Limitation | Description | |
|---|---|---|
| Jobs Number | 10,000 Jobs/User | including the queueing, running and hold jobs, but without finished jobs. |
| Default Wall Time | 100 hours | The recommended job walltime is between 0.5 to 4 hours. Please config your jobs in reasonable grainsize. |
| Long Wall Time | 30 days | Jobs with long walltime limitation have a lower priority. Moreover, there are only a few nodes that accept long jobs. So, be cautious to submit long jobs which might be waiting for a long time in the queue. |
| Default Memory | according to the physical memory of computing nodes | The memory of IHEP cluster is enough for most user jobs. Generally it's unnecessary to concern about it. |
| Big Memory | can be specified in HepJob | There are only a few big memory nodes, and some special service jobs have higher priority to be scheduled on these nodes. So users' big memory jobs are not guaranteed. |
| Disk Space | Please refer to the contents for each experiment. |