Integrating your Application
BenchPRO requires two input files (collectively refered to as a ‘profile’) to build an application: a configuration file containing variables, as well as a template script which will be populated with these variables and executed.
Configuration file
The configuration files used for compiling application contains the following fields:
Label |
Required? |
Description |
[general] |
||
code |
Y |
Application identifier. |
version |
Y |
Application version label, accepts x.x, x-x, or strings like ‘stable’. |
system |
N |
TACC system identifier, if left blank will use $TACC_SYSTEM. |
build_prefix |
N |
Custom build (outside of default tree). |
template |
N |
Overwrite default build template file. |
module_use |
N |
Path to be added to MODULEPATH, for using nonstandard modules. |
sched_cfg |
N |
Name of nonstandard scheduler config file to use. |
[modules] |
NOTE: user may add as many custom fields to this section as needed. |
|
compiler |
Y |
Module name of compiler, eg: ‘intel/18.0.2’ or just ‘intel’ for default. |
mpi |
Y |
Module name of MPI, eg: ‘impi/18.0.2’ or just ‘impi’ for default. |
[config] |
NOTE: user may add as many fields to this section as needed. |
|
arch |
N |
Generates architecture specific optimization flags. If left blank will use system default, set to ‘system’ to combine with ‘opt_flags’ below |
opt_flags |
N |
|
build_label |
N |
Custom build label, replaces arch default eg: skylake-xeon. Required if ‘opt_flags’ is set and ‘arch’ is not |
bin_dir |
N |
Path to executable within application directory, eg: bin, run etc. |
exe |
Y |
Name of application executable, used to check compilation was successful. |
collect_hw_stats |
N |
Runs the hardware stats collection tool after build. |
script_additions |
N |
Filename in $BP_HOME/templates, to be added to build script. |
general
Fields for general application info are provide, such as name and version. You can also specify a custom system label, which limits this application to that specific system (useful for avoiding inadvertantly building the wrong application on a given system). By default BenchPRO will attempt to match this config file with its corresponsing template file using the application name. You can overwrite this default template file by adding the template
field to this section.
modules
The section compiler
and mpi
are required, while more modules can be specified if needed. Every module must be available on the local machine, if you are cross compiling to another platform (e.g. to frontera-rtx) and require system modules not present on the current node, you can set check_modules=False in user.ini to bypass this check.
module_use
can be provided to add a nonstandard path to MODULEPATH
config
where variables used in the build template script can be added.
additionally, you can define as many additional parameters as needed for your application in this section. Eg: configure flags, build options, etc. All parameters [param]
defined here will be used to fill <<<[param]>>>
variables of the same name in the template file, thus consistent naming is important.
This file must be located in $BP_HOME/build/config
, preferably with the naming scheme [label].cfg
.
files
This section allows the user to have BenchPRO automatically collect input files for the build process. BenchPRO currently supports local
and download
operations. If BenchPRO detects that the local or downloaded file is an archive, it will automatically extract the archive to the correct working directory. BenchPRO will search for local files in the $BP_REPO directory. The format of the file staging section is:
local = [list],[of],[files]
download = [list],[of],[URLs]
This file staging process occurs in 1 of 2 ways, depending on the state of sync_staging
in $BP_HOME/user.ini
. BenchPRO will either synchronously copy/download/extract during the script creation process, alternatively the file staging will occur during job execution itself. In either case BenchPRO will confirm that the file or URL specified exists before continuing.
overload
This section allows you to modify default parameters for this specific application. Refer to Changing settings for additional information.
Build template file
The build template file is used to gerenate a contextualized build script which will executed to compile the application. Variables are defined with <<<[param]>>>
syntax and populated with the variables defined in the config file above. If a <<<[param]>>>
in the build template is not successfully populated and exit_on_missing=True
in $BP_HOME/user.ini
, BenchPRO will abort the build process. You are able to make use of the local_repo
variable defined in $BP_HOME/user.ini
to store and use files locally, if you’d rather manage your input files manually. This file must be located in $BP_HOME/build/templates
, preferablly with the naming scheme [code_label].template
.
The contextualized build script generated by BenchPRO will have the format:
[scheduler options]
[job level details]
[export environment variables]
[load modules]
[file staging]
[user section]
[executable check]
Module template file (optional)
It is possible to define your own .lua module template and store it in $BP_HOME/build/templates
with the naming scheme [code_label].module
, alternatively BenchPRO will generate a generic module file for you.
The application added above would be built with the following command:
benchpro --build [code_label]
Example Script
Below is an application compilation job script generated by BenchPRO. Everything outside the ‘USER SECTION’ is produced by BenchPRO, which depends on parameters provided in the configuration file, as well as the current system and architecture.
#!/bin/bash
#SBATCH -J lammps_build
#SBATCH -o /scratch1/06280/mcawood/benchpro/apps/frontera/cascadelake/intel22/intel22impi/lammps/23Jun2022/default/stdout.log
#SBATCH -e /scratch1/06280/mcawood/benchpro/apps/frontera/cascadelake/intel22/intel22impi/lammps/23Jun2022/default/stderr.log
#SBATCH -p small
#SBATCH -t 01:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -A A-ccsc
echo "START `date +"%Y"-%m-%dT%T` `date +"%s"`"
echo "JobID: ${SLURM_JOB_ID}"
echo "User: ${USER}"
echo "Hostlist: ${SLURM_NODELIST}"
export working_path=/scratch1/06280/mcawood/benchpro/apps/frontera/cascadelake/intel22/intel22impi/lammps/23Jun2022/default/build
export install_path=/scratch1/06280/mcawood/benchpro/apps/frontera/cascadelake/intel22/intel22impi/lammps/23Jun2022/default/install
export local_repo=/scratch1/06280/mcawood/benchpro/repo
export version=23Jun2022
export opt_flags='-O2 -xCORE-AVX512 -sox'
export exe=lmp_intel_cpu_intelmpi
export build_label=default
export threads=8
# Create application directories
mkdir -p ${install_path}
mkdir -p ${working_path} && cd ${working_path}
# [config]
export arch=cascadelake
export opt_flags='-O2 -xCORE-AVX512 -sox'
export build_label=default
export bin_dir=
export exe=lmp_intel_cpu_intelmpi
export collect_stats=True
export script_additions=
export local_repo=/scratch1/06280/mcawood/benchpro/repo
export cores=56
export nodes=1
export stdout=stdout.log
export stderr=stderr.log
# [modules]
export compiler=intel/22.1.2
export mpi=intel22/impi/22.1.2
# Load modules
ml use /scratch1/hpc_tools/benchpro-dev/modulefiles
ml use /scratch1/projects/compilers/modulefiles
ml benchpro
ml intel/22.1.2
ml intel22/impi/22.1.2
ml
# Stage Files
stage https://web.corral.tacc.utexas.edu/ASC23006/apps/lammps-23Jun2022.tgz
# Compiler variables
export CC=icc
export CXX=icpc
export FC=ifort
export F77=ifort
export F90=ifort
export MPICC=mpicc
export MPICXX=mpicxx
export MPIFORT=mpifort
#------USER SECTION------
...
#------------------------
ldd ${install_path}/lmp_intel_cpu_intelmpi
echo "END `date +"%Y"-%m-%dT%T` `date +"%s"`"