Ligeng-Zhu's picture
Upload files with `vila-upload`.
342f304 verified
2025-07-01 08:44:47.309 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.309 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.309 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.309 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.309 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.309 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.309 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.309 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.310 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.310 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.310 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.310 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.310 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.310 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.310 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.310 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.320 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.320 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.320 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.320 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.320 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.320 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.320 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.320 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.433 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.433 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.433 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.433 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.433 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.433 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.433 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.434 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.444 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.444 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.444 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.444 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.444 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.444 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.444 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.444 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.499 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.499 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.499 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.499 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.499 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.499 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.499 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.499 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.500 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.500 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.500 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.500 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.500 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.500 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.500 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.500 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.509 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.509 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.509 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.509 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.509 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.509 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.509 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.510 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.626 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.626 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.626 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.626 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.626 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.626 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.626 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.626 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.661 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.661 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.661 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.661 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.661 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.661 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.661 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.661 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:47.662 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.662 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.662 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.662 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.662 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.662 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.662 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.662 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:47.672 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.672 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.672 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.672 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.672 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.672 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.672 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:47.672 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.363 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.363 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.363 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.363 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.363 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.363 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.363 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.363 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.364 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.364 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.364 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.364 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.364 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.364 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.364 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.364 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.374 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.374 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.374 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.374 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.374 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.374 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.374 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.374 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.932 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.933 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.933 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.933 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.933 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.933 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.933 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:48.942 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.942 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.942 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.942 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.942 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.942 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.942 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:48.942 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:49.108 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:49.108 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:49.108 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:49.108 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:49.108 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:49.108 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:49.108 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:49.108 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'.
2025-07-01 08:44:49.109 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:49.109 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:49.109 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:49.109 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:49.109 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:49.109 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:49.109 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:49.109 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'.
2025-07-01 08:44:49.119 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:49.119 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:49.119 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:49.119 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:49.119 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:49.119 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:49.119 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
2025-07-01 08:44:49.119 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
Did not find AutoResume SDK!
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 4655.66it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 5840.49it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 4761.87it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 4907.26it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7367.04it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8162.39it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6491.77it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8838.97it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8027.74it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7848.90it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 11712.82it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8595.72it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6813.68it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7599.69it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7458.12it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7904.55it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 5252.57it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7255.98it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9405.27it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8371.86it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7353.51it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8126.25it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9310.82it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8344.11it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7661.83it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8049.02it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8091.90it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8699.30it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 5884.97it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8724.29it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6548.73it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9129.39it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8879.07it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 10035.36it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7104.98it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6637.56it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7606.91it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 12334.46it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7151.12it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8355.19it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7711.47it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8905.10it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7614.80it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9716.53it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6460.82it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 5397.74it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7394.88it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8229.50it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 10014.82it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7427.93it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8252.64it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8643.81it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6620.10it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8241.83it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8342.53it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9438.53it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 4962.00it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8768.58it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8251.86it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7133.75it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 10649.30it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8796.60it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 11047.33it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s] Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 10216.96it/s]
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.57s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.49s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.49s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.57s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.57s/it] Loading checkpoint shards: 2 Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.54s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.57s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.58s/it] Loading checkpoint shards: 2 Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.46s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.48s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.51s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.52s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.52s/it] Loading checkpoint shards: 2 Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.55s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.61s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.65s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.65s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.66s/it] Loading checkpoint shards: 2 Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.45s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.49s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.61s/it] Loading checkpoint shards: 2 Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.51s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.58s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.61s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.61s/it] Loading checkpoint shards: 2 Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.54s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.63s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.65s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.65s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.57s/it] Loading checkpoint shards: 2 Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.64s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.64s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.63s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.64s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.63s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.61s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.82s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.63s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.65s/it] Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:13, 4.57s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.82s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.82s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.64s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.65s/it] Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:13, 4.60s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.55s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.81s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.81s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.82s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.82s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.83s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.63s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.65s/it] Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:13, 4.62s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.67s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.80s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.94s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.63s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.84s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.51s/it]
5%|██▌ | 1/4 [00:04<00:13, 4.66s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.65s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.64s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.64s/it] Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:13, 4.66s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.57s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.82s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.83s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.64s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:13, 4.59s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.61s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.79s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.81s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.63s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.64s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.86s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.53s/it]
▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.86s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.53s/it]
5%|██▌ | 1/4 [00:04<00:13, 4.64s/it] Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it] Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████ Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.86s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.54s/it]
▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.86s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.51s/it]
▌ | 3/4 [00:14<00:04, 4.65s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.65s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.86s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.54s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.86s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.52s/it]
▌ | 3/4 [00:13<00:04, 4.63s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.64s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.64s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.65s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.65s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.87s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.52s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
▌ | 3/4 [00:14<00:04, 4.65s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.65s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.65s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.88s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.88s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.53s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.88s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.87s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.53s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.77s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it]
▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:15<00:00, 3.19s/it] Loading checkpoint shards: 100%|██████████| 4/4 [00:15<00:00, 3.78s/it]
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`
2025-07-01 08:48:06.462 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.472 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.524 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.549 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.551 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.714 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.731 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.751 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.755 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.755 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.779 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.822 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.852 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.856 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.863 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.872 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.876 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.894 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.937 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:06.993 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:06.996 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.011 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.017 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.017 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.020 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.044 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.044 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.044 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.045 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.048 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.057 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.059 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.061 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.068 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.075 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.080 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.102 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.106 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.106 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.116 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.127 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.141 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.143 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.145 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.149 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.158 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.162 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.165 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.185 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.189 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.201 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.202 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.210 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.210 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.216 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.216 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.227 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.237 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.240 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.244 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.250 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.271 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
2025-07-01 08:48:07.316 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
2025-07-01 08:48:07.329 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'.
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead.
super().__init__(*args, **kwargs)
[rank0]: Traceback (most recent call last):
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/train_mem.py", line 49, in <module>
[rank0]: train()
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/train.py", line 906, in train
[rank0]: trainer.train(resume_from_checkpoint=resume_from_checkpoint)
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/trainer.py", line 2122, in train
[rank0]: return inner_training_loop(
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py", line 665, in _inner_training_loop
[rank0]: return super()._inner_training_loop(batch_size, *args, **kwargs)
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/trainer.py", line 2381, in _inner_training_loop
[rank0]: self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/trainer_callback.py", line 468, in on_train_begin
[rank0]: return self.call_event("on_train_begin", args, state, control)
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/trainer_callback.py", line 518, in call_event
[rank0]: result = getattr(callback, event)(
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 911, in on_train_begin
[rank0]: self.setup(args, state, model, **kwargs)
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/integrations/integration_utils.py", line 838, in setup
[rank0]: self._wandb.init(
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1623, in init
[rank0]: wandb._sentry.reraise(e)
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/wandb/analytics/sentry.py", line 156, in reraise
[rank0]: raise exc.with_traceback(sys.exc_info()[2])
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 1551, in init
[rank0]: wi.maybe_login(init_settings)
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/wandb/sdk/wandb_init.py", line 191, in maybe_login
[rank0]: wandb_login._login(
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 315, in _login
[rank0]: key, key_status = wlogin.prompt_api_key(referrer=referrer)
[rank0]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/wandb/sdk/wandb_login.py", line 243, in prompt_api_key
[rank0]: raise UsageError("api_key not configured (no-tty). call " + directive)
[rank0]: wandb.errors.errors.UsageError: api_key not configured (no-tty). call wandb.login(key=[your_api_key])
W0701 08:48:31.378000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2073409 closing signal SIGTERM
W0701 08:48:31.380000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2073410 closing signal SIGTERM
W0701 08:48:31.381000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2073411 closing signal SIGTERM
W0701 08:48:31.382000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2073412 closing signal SIGTERM
W0701 08:48:31.383000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2073413 closing signal SIGTERM
W0701 08:48:31.387000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2073414 closing signal SIGTERM
W0701 08:48:31.387000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2073415 closing signal SIGTERM
E0701 08:48:38.726000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: 1) local_rank: 0 (pid: 2073408) of binary: /lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/bin/python3
Traceback (most recent call last):
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/bin/torchrun", line 10, in <module>
sys.exit(main())
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 347, in wrapper
return f(*args, **kwargs)
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/run.py", line 879, in main
run(args)
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/run.py", line 870, in run
elastic_launch(
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
return launch_agent(self._config, self._entrypoint, list(args))
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 263, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
llava/train/train_mem.py FAILED
------------------------------------------------------------
Failures:
<NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
time : 2025-07-01_08:48:31
host : pool0-01504.cm.cluster
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 2073408)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
W0701 08:48:39.238000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers
W0701 08:48:39.238000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1498437 closing signal SIGTERM
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1498438 closing signal SIGTERM
W0701 08:48:39.238000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1066066 closing signal SIGTERM
W0701 08:48:39.238000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1845860 closing signal SIGTERM
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2994607 closing signal SIGTERM
W0701 08:48:39.240000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1881736 closing signal SIGTERM
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 126184 closing signal SIGTERM
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers
W0701 08:48:39.239000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 627617 closing signal SIGTERM
W0701 08:48:39.241000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1881737 closing signal SIGTERM
W0701 08:48:39.240000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1845861 closing signal SIGTERM
W0701 08:48:39.240000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1066067 closing signal SIGTERM
W0701 08:48:39.240000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2994608 closing signal SIGTERM
W0701 08:48:39.240000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 126185 closing signal SIGTERM
W0701 08:48:39.240000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 627618 closing signal SIGTERM
W0701 08:48:39.242000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1881738 closing signal SIGTERM
W0701 08:48:39.241000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1845862 closing signal SIGTERM
W0701 08:48:39.241000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1498439 closing signal SIGTERM
W0701 08:48:39.241000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2994609 closing signal SIGTERM
W0701 08:48:39.241000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1066068 closing signal SIGTERM
W0701 08:48:39.242000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 126186 closing signal SIGTERM
W0701 08:48:39.242000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 627619 closing signal SIGTERM
W0701 08:48:39.242000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1066069 closing signal SIGTERM
W0701 08:48:39.243000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1498440 closing signal SIGTERM
W0701 08:48:39.244000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1881739 closing signal SIGTERM
W0701 08:48:39.243000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2994610 closing signal SIGTERM
W0701 08:48:39.243000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 126187 closing signal SIGTERM
W0701 08:48:39.243000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1845863 closing signal SIGTERM
W0701 08:48:39.243000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 627620 closing signal SIGTERM
W0701 08:48:39.244000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1498441 closing signal SIGTERM
W0701 08:48:39.244000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1066070 closing signal SIGTERM
W0701 08:48:39.244000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 126188 closing signal SIGTERM
W0701 08:48:39.246000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1881740 closing signal SIGTERM
W0701 08:48:39.245000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1845864 closing signal SIGTERM
W0701 08:48:39.245000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2994611 closing signal SIGTERM
W0701 08:48:39.245000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1066071 closing signal SIGTERM
W0701 08:48:39.246000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1498442 closing signal SIGTERM
W0701 08:48:39.245000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 627621 closing signal SIGTERM
W0701 08:48:39.246000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 126189 closing signal SIGTERM
W0701 08:48:39.247000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1881741 closing signal SIGTERM
W0701 08:48:39.246000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1845865 closing signal SIGTERM
W0701 08:48:39.246000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2994612 closing signal SIGTERM
W0701 08:48:39.247000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1066072 closing signal SIGTERM
W0701 08:48:39.247000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1498443 closing signal SIGTERM
W0701 08:48:39.247000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 627622 closing signal SIGTERM
W0701 08:48:39.247000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 126190 closing signal SIGTERM
W0701 08:48:39.248000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1845866 closing signal SIGTERM
W0701 08:48:39.249000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1881742 closing signal SIGTERM
W0701 08:48:39.248000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 126191 closing signal SIGTERM
W0701 08:48:39.248000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1498444 closing signal SIGTERM
W0701 08:48:39.248000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2994613 closing signal SIGTERM
W0701 08:48:39.249000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 627623 closing signal SIGTERM
W0701 08:48:39.249000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1066073 closing signal SIGTERM
W0701 08:48:39.250000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1881743 closing signal SIGTERM
W0701 08:48:39.249000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1845867 closing signal SIGTERM
W0701 08:48:39.249000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2994614 closing signal SIGTERM
W0701 08:48:39.251000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 627624 closing signal SIGTERM