Model Card for mlpf-cms-v2.2.0

This model reconstructs particles in a detector, based on the tracks and calorimeter clusters recorded by the detector. The primary difference with respect to v2.2.0 is the inclusion of the sqrt(pt) weight term in the pT and energy regression loss. Additionally, the model has been scaled down to ~5M parameters (previously ~100M) for more efficient inference.

Model Details

The performance is measured with respect to generator-level jets and MET computed from Pythia particles, i.e. the truth-level jets and MET.

Jet performance

MET performance

Model Description

Developed by: CMS MLPF Team
Model type: transformer
License: Apache License

Model Sources

Repository: https://github.com/jpata/particleflow/releases/tag/v2.2.0

Uses

Direct Use

This model may be used to study the physics and computational performance on ML-based reconstruction in simulation within the CMS collaboration.

Out-of-Scope Use

This model is not intended for physics measurements on real data or for use outside the CMS collaboration.

Bias, Risks, and Limitations

The model has only been trained on simulation data and has not been validated against real data. The model has not been peer reviewed or published in a peer-reviewed journal.

How to Get Started with the Model

Use the code below to get started with the model.

#get the code
git clone https://github.com/jpata/particleflow
cd particleflow
git checkout v2.2.0

#get the models
git clone https://huggingface.co/jpata/particleflow models

Training Details

Trained on 1x A100 for 5 epochs over ~2 days.

Training Data

The following datasets were used:

18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/1/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/2/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/3/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/4/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/5/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/6/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/7/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/8/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/9/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd/10/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/1/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/2/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/3/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/4/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/5/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/6/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/7/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/8/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/9/2.5.0
8.4G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_qcd_nopu/10/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/1/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/2/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/3/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/4/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/5/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/6/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/7/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/8/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/9/2.5.0
19G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar/10/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/1/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/2/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/3/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/4/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/5/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/6/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/7/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/8/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/9/2.5.0
8.6G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ttbar_nopu/10/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/1/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/2/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/3/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/4/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/5/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/6/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/7/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/8/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/9/2.5.0
18G     /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt/10/2.5.0
5.8G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/1/2.5.0
5.8G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/2/2.5.0
5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/3/2.5.0
5.8G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/4/2.5.0
5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/5/2.5.0
5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/6/2.5.0
5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/7/2.5.0
5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/8/2.5.0
5.7G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/9/2.5.0
5.8G    /scratch/persistent/joosep/tensorflow_datasets/cms_pf_ztt_nopu/10/2.5.0

Training Procedure

#!/bin/bash
#SBATCH --partition gpu
#SBATCH --gres gpu:a100:1
#SBATCH --mem-per-gpu 300G
#SBATCH -o logs/slurm-%x-%j-%N.out

IMG=/home/software/singularity/pytorch.simg:2024-12-03
cd ~/particleflow

ulimit -n 100000
singularity exec -B /scratch/persistent --nv \
    --env PYTHONPATH=`pwd` \
    --env KERAS_BACKEND=torch \
    $IMG python3 mlpf/pipeline.py --gpus 1 \
    --data-dir /scratch/persistent/joosep/tensorflow_datasets --config parameters/pytorch/pyg-cms.yaml \
    --train --conv-type attention \
    --gpu-batch-multiplier 5 --checkpoint-freq 1 --num-workers 8 --prefetch-factor 50 --comet --ntest 1000 --test-datasets cms_pf_qcd_nopu

Evaluation

#!/bin/bash
#SBATCH --partition gpu
#SBATCH --gres gpu:mig:1
#SBATCH --mem-per-gpu 100G
#SBATCH -o logs/slurm-%x-%j-%N.out

IMG=/home/software/singularity/pytorch.simg:2024-08-18
cd ~/particleflow

WEIGHTS=experiments/pyg-cms_20241212_101648_120237/checkpoints/checkpoint-05-3.498507.pth
DATASET=$1
env
singularity exec -B /local -B /scratch/persistent --nv \
     --env PYTHONPATH=`pwd` \
     --env KERAS_BACKEND=torch \
     $IMG python mlpf/pipeline.py --gpus 1 \
     --data-dir /scratch/persistent/joosep/tensorflow_datasets --config parameters/pytorch/pyg-cms.yaml \
     --test --make-plots --gpu-batch-multiplier 2 --load $WEIGHTS --ntest 10000 --dtype bfloat16 --num-workers 1 --prefetch-factor 10 --test-datasets $DATASET

jpata
/

particleflow