add v2.1.0 CMS model

cms/v2.1.0/pyg-cms_20241101_090645_682892/.gitattributes

@@ -0,0 +1,3 @@
+plots_checkpoint-08-2.986092 filter=lfs diff=lfs merge=lfs -text
+runs/ filter=lfs diff=lfs merge=lfs -text
+checkpoints/ filter=lfs diff=lfs merge=lfs -text

cms/v2.1.0/pyg-cms_20241101_090645_682892/README.md

@@ -0,0 +1,140 @@
+# Model Card for mlpf-cms-v2.1.0
+
+This model reconstructs particles in a detector, based on the tracks and calorimeter clusters recorded by the detector.
+
+## Model Details
+
+The performance is measured with respect to generator-level jets and MET computed from Pythia particles, i.e. the truth-level jets and MET.
+
+<details>
+  <summary>Jet performance</summary>
+  
+  <img src="plots_checkpoint-08-2.986092/cms_pf_qcd/jet_response_iqr_over_med_pt.png" alt="ttbar jet resolution" width="300"/>
+  <img src="plots_checkpoint-08-2.986092/cms_pf_ttbar/jet_response_iqr_over_med_pt.png" alt="qq jet resolution" width="300"/>
+  <img src="plots_checkpoint-08-2.986092/cms_pf_ztt/jet_response_iqr_over_med_pt.png" alt="ttbar jet resolution" width="300"/>
+
+</details>
+
+<details>
+  <summary>MET performance</summary>
+  
+  <img src="plots_checkpoint-08-2.986092/cms_pf_qcd/met_response_iqr_over_med.png" alt="ttbar MET resolution" width="300"/>
+  <img src="plots_checkpoint-08-2.986092/cms_pf_ttbar/met_response_iqr_over_med.png" alt="qq MET resolution" width="300"/>
+  <img src="plots_checkpoint-08-2.986092/cms_pf_ztt/met_response_iqr_over_med.png" alt="ttbar MET resolution" width="300"/>
+
+</details>
+
+### Model Description
+
+- **Developed by:** CMS MLPF Team 
+- **Model type:** transformer
+- **License:** Apache License
+
+### Model Sources
+
+- **Repository:** https://github.com/jpata/particleflow/releases/tag/v2.1.0
+
+## Uses
+### Direct Use
+
+This model may be used to study the physics and computational performance on ML-based reconstruction in simulation within the CMS collaboration.
+
+### Out-of-Scope Use
+
+This model is not intended for physics measurements on real data or for use outside the CMS collaboration.
+
+## Bias, Risks, and Limitations
+
+The model has only been trained on simulation data and has not been validated against real data.
+The model has not been peer reviewed or published in a peer-reviewed journal.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+```
+#get the code
+git clone https://github.com/jpata/particleflow
+cd particleflow
+git checkout v2.1.0
+
+#get the models
+git clone https://huggingface.co/jpata/particleflow models
+```
+
+## Training Details
+Trained on 8x MI250X for 8 epochs over ~8 days.
+The training was continued multiple times from a checkpoint due to the 24h time limit.
+
+### Training Data
+The following datasets were used:
+```
+179G    cms_pf_qcd
+84G     cms_pf_qcd_nopu
+179G    cms_pf_ttbar
+86G     cms_pf_ttbar_nopu
+173G    cms_pf_ztt
+57G     cms_pf_ztt_nopu
+```
+
+## Training Procedure 
+
+```bash
+#!/bin/bash
+#SBATCH --job-name=mlpf-train
+#SBATCH --account=project_465000301
+#SBATCH --time=3-00:00:00
+#SBATCH --nodes=1
+#SBATCH --ntasks-per-node=1
+#SBATCH --cpus-per-task=32
+#SBATCH --mem=400G
+#SBATCH --gpus-per-task=8
+#SBATCH --partition=small-g
+#SBATCH --no-requeue
+#SBATCH -o logs/slurm-%x-%j-%N.out
+
+cd /scratch/project_465000301/particleflow
+
+module load LUMI/24.03 partition/G
+
+export IMG=/scratch/project_465000301/pytorch-rocm6.2.simg
+export PYTHONPATH=`pwd`
+export TFDS_DATA_DIR=/scratch/project_465000301/tensorflow_datasets
+#export MIOPEN_DISABLE_CACHE=true
+export MIOPEN_USER_DB_PATH=/tmp/${USER}-${SLURM_JOB_ID}-miopen-cache
+export MIOPEN_CUSTOM_CACHE_DIR=${MIOPEN_USER_DB_PATH}
+export TF_CPP_MAX_VLOG_LEVEL=-1 #to suppress ROCm fusion is enabled messages
+export ROCM_PATH=/opt/rocm
+#export NCCL_DEBUG=INFO
+#export MIOPEN_ENABLE_LOGGING=1
+#export MIOPEN_ENABLE_LOGGING_CMD=1
+#export MIOPEN_LOG_LEVEL=4
+export KERAS_BACKEND=torch
+
+env
+
+#TF training
+singularity exec \
+    --rocm \
+    -B /scratch/project_465000301 \
+    -B /tmp \
+    --env LD_LIBRARY_PATH=/opt/rocm/lib/ \
+    --env CUDA_VISIBLE_DEVICES=$ROCR_VISIBLE_DEVICES \
+     $IMG python3 mlpf/pipeline.py --gpus 8 \
+     --data-dir $TFDS_DATA_DIR --config parameters/pytorch/pyg-cms.yaml \
+     --train --gpu-batch-multiplier 5 --num-workers 8 --prefetch-factor 50 --checkpoint-freq 1 --conv-type attention --dtype bfloat16 --lr 0.0001
+```
+
+
+## Citation
+
+## Glossary
+
+- PF: particle flow reconstruction
+- MLPF: machine learning for particle flow
+- CMS: Compact Muon Solenoid
+
+## Model Card Contact
+
+Joosep Pata, [email protected]
+

cms/v2.1.0/pyg-cms_20241101_090645_682892/checkpoints/checkpoint-08-2.986092.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67fa037c0247ef3cc9ba7a2e01c717d3e0b8de18f7e77188c21248b655657f86
+size 1386307830

cms/v2.1.0/pyg-cms_20241101_090645_682892/history/epoch_1.json

@@ -0,0 +1 @@
+{"train": {"Regression_pt": 0.3077394842880334, "Regression_eta": 0.00132643331811482, "Regression_sin_phi": 0.0006513258653524121, "Regression_cos_phi": 0.0006563989022422504, "Regression_energy": 0.3663458205453361, "Classification_binary": 2.9862765464019265, "Classification": 0.04570725841941588, "MET": 459.89318280503693, "Sliced_Wasserstein_Loss": 168.10397505230333, "Total": 3.7087050487506414}, "valid": {"Regression_pt": 0.3087922769314699, "Regression_eta": 0.0013286831028171695, "Regression_sin_phi": 0.0006538974789752973, "Regression_cos_phi": 0.0006583031814998147, "Regression_energy": 0.3644685508006841, "Classification_binary": 2.9960305892412937, "Classification": 0.04493322691005769, "MET": 33.47015480632552, "Sliced_Wasserstein_Loss": 167.72577958422175, "Total": 3.716865935390014}, "epoch_train_time": 77175.13525176048, "epoch_valid_time": 2187.155287027359, "epoch_total_time": 79369.70944404602}

cms/v2.1.0/pyg-cms_20241101_090645_682892/history/epoch_2.json

@@ -0,0 +1 @@
+{"train": {"Regression_pt": 0.28896851747226937, "Regression_eta": 0.00127929147211294, "Regression_sin_phi": 0.0006381754520750401, "Regression_cos_phi": 0.0006429972426700182, "Regression_energy": 0.34543455392916356, "Classification_binary": 2.804126534559665, "Classification": 0.044099002651864665, "MET": 729.5687048513796, "Sliced_Wasserstein_Loss": 161.20007500098686, "Total": 3.485195631685943}, "valid": {"Regression_pt": 0.2998550078846837, "Regression_eta": 0.0013050050200006176, "Regression_sin_phi": 0.0006479521804272747, "Regression_cos_phi": 0.000653431529090518, "Regression_energy": 0.35510648660991917, "Classification_binary": 2.7479750383129, "Classification": 0.043945683938184356, "MET": 32.64908992981521, "Sliced_Wasserstein_Loss": 158.85165689410093, "Total": 3.4494887448916134}, "epoch_train_time": 78409.91169214249, "epoch_valid_time": 2127.3791201114655, "epoch_total_time": 80543.92620301247}

cms/v2.1.0/pyg-cms_20241101_090645_682892/history/epoch_3.json

@@ -0,0 +1 @@
+{"train": {"Regression_pt": 0.2843597885539612, "Regression_eta": 0.0012700329294461831, "Regression_sin_phi": 0.0006345689346132176, "Regression_cos_phi": 0.0006393563171899903, "Regression_energy": 0.3410772032156673, "Classification_binary": 2.653088979789208, "Classification": 0.04347674425866212, "MET": 798.5169541704496, "Sliced_Wasserstein_Loss": 155.93235108356689, "Total": 3.324547033513599}, "valid": {"Regression_pt": 0.289105489225191, "Regression_eta": 0.0012866090016748011, "Regression_sin_phi": 0.0006430510857805091, "Regression_cos_phi": 0.0006481815757019433, "Regression_energy": 0.3468229921472437, "Classification_binary": 2.6298736160159026, "Classification": 0.04359702819954358, "MET": 31.368653384861407, "Sliced_Wasserstein_Loss": 153.5130263859275, "Total": 3.311975973925018}, "epoch_train_time": 78404.13188505173, "epoch_valid_time": 2096.961377620697, "epoch_total_time": 80507.32924699783}

cms/v2.1.0/pyg-cms_20241101_090645_682892/history/epoch_4.json

@@ -0,0 +1 @@
+{"train": {"Regression_pt": 0.27708855544151895, "Regression_eta": 0.0012621611729661303, "Regression_sin_phi": 0.0006314986401397063, "Regression_cos_phi": 0.000636240245746963, "Regression_energy": 0.33459549272441086, "Classification_binary": 2.568074497690759, "Classification": 0.043273851092510265, "MET": 732.3399518414716, "Sliced_Wasserstein_Loss": 151.24538151817788, "Total": 3.2255636176133895}, "valid": {"Regression_pt": 0.27584819346348616, "Regression_eta": 0.0012768102014157987, "Regression_sin_phi": 0.0006391416649998039, "Regression_cos_phi": 0.0006440805050186341, "Regression_energy": 0.3346875806861341, "Classification_binary": 2.5616322336420576, "Classification": 0.04344770733820435, "MET": 31.370474635749822, "Sliced_Wasserstein_Loss": 148.34397210376687, "Total": 3.2181763476146057}, "epoch_train_time": 78592.42363166809, "epoch_valid_time": 2161.9929819107056, "epoch_total_time": 80760.85575890541}

cms/v2.1.0/pyg-cms_20241101_090645_682892/history/epoch_5.json

@@ -0,0 +1 @@
+{"train": {"Regression_pt": 0.2677935176881439, "Regression_eta": 0.0012539196620497397, "Regression_sin_phi": 0.0006286499649256859, "Regression_cos_phi": 0.0006334169812684882, "Regression_energy": 0.3263203658518533, "Classification_binary": 2.511015769944341, "Classification": 0.04317117575919996, "MET": 732.609244858485, "Sliced_Wasserstein_Loss": 146.67466348241425, "Total": 3.1508198298661823}, "valid": {"Regression_pt": 0.26606543350490847, "Regression_eta": 0.0012669664964492895, "Regression_sin_phi": 0.0006357697874582992, "Regression_cos_phi": 0.0006402864385007033, "Regression_energy": 0.3259657349413646, "Classification_binary": 2.5117636913090795, "Classification": 0.043381846196785326, "MET": 31.427449804548687, "Sliced_Wasserstein_Loss": 144.7462131307747, "Total": 3.149722231476546}, "epoch_train_time": 78312.97973370552, "epoch_valid_time": 2098.589688539505, "epoch_total_time": 80418.28772830963}

cms/v2.1.0/pyg-cms_20241101_090645_682892/history/epoch_6.json

@@ -0,0 +1 @@
+{"train": {"Regression_pt": 0.25986982301987527, "Regression_eta": 0.0012467644506665358, "Regression_sin_phi": 0.0006262281524232959, "Regression_cos_phi": 0.0006310321366806251, "Regression_energy": 0.31916886601715155, "Classification_binary": 2.462185871689101, "Classification": 0.04310707354599238, "MET": 566.5121777918131, "Sliced_Wasserstein_Loss": 142.5576323372676, "Total": 3.0868386220542376}, "valid": {"Regression_pt": 0.25891829045342485, "Regression_eta": 0.0012578428765942886, "Regression_sin_phi": 0.0006332250885194129, "Regression_cos_phi": 0.0006378759876920893, "Regression_energy": 0.319457964432858, "Classification_binary": 2.4648086992492892, "Classification": 0.04331902976398699, "MET": 31.408612628820183, "Sliced_Wasserstein_Loss": 140.83813077469793, "Total": 3.089034459399431}, "epoch_train_time": 78398.8162457943, "epoch_valid_time": 2083.1339452266693, "epoch_total_time": 80492.1939125061}

cms/v2.1.0/pyg-cms_20241101_090645_682892/history/epoch_7.json

@@ -0,0 +1 @@
+{"train": {"Regression_pt": 0.25354415898975646, "Regression_eta": 0.0012409184756697045, "Regression_sin_phi": 0.0006241587397660043, "Regression_cos_phi": 0.0006290005201396115, "Regression_energy": 0.31335638590844944, "Classification_binary": 2.4164084395847314, "Classification": 0.04305974015951279, "MET": 371.12268582481346, "Sliced_Wasserstein_Loss": 138.81873445703232, "Total": 3.0288599603284254}, "valid": {"Regression_pt": 0.2529466055603678, "Regression_eta": 0.0012534307772670982, "Regression_sin_phi": 0.0006307690975593181, "Regression_cos_phi": 0.0006355680551779719, "Regression_energy": 0.31393011834660406, "Classification_binary": 2.4206331276097193, "Classification": 0.043286926117702504, "MET": 31.22202603056148, "Sliced_Wasserstein_Loss": 137.05965707178393, "Total": 3.0333176472547976}, "epoch_train_time": 78172.47966051102, "epoch_valid_time": 2206.4134516716003, "epoch_total_time": 80385.80634713173}

cms/v2.1.0/pyg-cms_20241101_090645_682892/history/epoch_8.json

@@ -0,0 +1 @@
+{"train": {"Regression_pt": 0.2486735087077626, "Regression_eta": 0.001236475700312463, "Regression_sin_phi": 0.0006224918708052877, "Regression_cos_phi": 0.0006273420084874742, "Regression_energy": 0.30882288541625547, "Classification_binary": 2.3751925909386964, "Classification": 0.04301502328823817, "MET": 329.7067856156002, "Sliced_Wasserstein_Loss": 135.67856156002054, "Total": 2.9781877269766706}, "valid": {"Regression_pt": 0.2481330269689499, "Regression_eta": 0.001247925883756094, "Regression_sin_phi": 0.0006288004074015343, "Regression_cos_phi": 0.0006337736884837635, "Regression_energy": 0.3095986394062056, "Classification_binary": 2.3826093097570187, "Classification": 0.043240930141077535, "MET": 31.110252309879176, "Sliced_Wasserstein_Loss": 133.8612184612651, "Total": 2.986092486784826}, "epoch_train_time": 78312.4050514698, "epoch_valid_time": 2108.51451086998, "epoch_total_time": 80428.31892037392}

cms/v2.1.0/pyg-cms_20241101_090645_682892/hyperparameters.json

@@ -0,0 +1 @@
+{"Num of mlpf parameters": 115486753, "config": "parameters/pytorch/pyg-cms.yaml", "prefix": null, "data_dir": "/scratch/project_465000301/tensorflow_datasets", "gpus": 8, "gpu_batch_multiplier": 5, "num_workers": 8, "prefetch_factor": 50, "resume_training": null, "load": "experiments/pyg-cms_20241101_090645_682892/checkpoints/checkpoint-06-3.089034.pth", "train": true, "test": null, "num_epochs": null, "patience": null, "lr": 0.0001, "conv_type": "attention", "num_convs": null, "make_plots": null, "export_onnx": null, "ntrain": null, "ntest": null, "nvalid": null, "val_freq": null, "checkpoint_freq": 1, "hpo": null, "ray_train": false, "local": null, "ray_cpus": null, "ray_gpus": null, "raytune_num_samples": null, "comet": false, "comet_offline": false, "comet_step_freq": null, "experiments_dir": null, "pipeline": null, "dtype": "bfloat16", "attention_type": null, "test_datasets": {"cms_pf_ttbar": {"version": "2.5.0", "splits": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}, "cms_pf_qcd": {"version": "2.5.0", "splits": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}, "cms_pf_ztt": {"version": "2.5.0", "splits": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}, "cms_pf_ttbar_nopu": {"version": "2.5.0", "splits": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}, "cms_pf_qcd_nopu": {"version": "2.5.0", "splits": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}, "cms_pf_ztt_nopu": {"version": "2.5.0", "splits": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}}}

cms/v2.1.0/pyg-cms_20241101_090645_682892/mlpf_losses.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fd55a482a23d5808de41476ad911bf77976f9bbebb06d1c5855d50003abbc77
+size 392

cms/v2.1.0/pyg-cms_20241101_090645_682892/model_kwargs.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02d9451fe7debc0805f6486e50bd602916ee7e401cc55326e48cdc7da11038d4
+size 565