|
|
2025-07-01 09:10:29.995 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:29.995 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.004 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.578 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.578 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.587 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.587 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.587 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.596 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.614 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.614 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.621 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.621 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.623 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.625 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.625 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.630 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.634 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.635 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.635 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.635 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.636 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.644 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.645 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.658 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.658 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.667 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.698 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.698 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.707 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.723 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.723 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.727 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.727 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.731 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.736 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.757 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.757 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.757 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.757 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.757 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.757 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.759 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:30.759 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:30.766 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.766 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.766 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:30.768 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9962.72it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.241 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.242 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.252 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.251 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.251 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.252 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.252 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.252 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.252 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.252 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.252 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.251 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.252 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.251 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.251 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.251 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.251 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.251 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.298 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.298 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.298 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.298 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.298 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.298 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.298 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.298 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.299 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.299 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.299 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.299 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.299 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.299 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.299 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.299 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.309 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.309 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.309 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.309 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.309 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.309 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.309 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.309 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.337 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.337 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.337 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.337 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.338 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.348 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.348 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.348 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.348 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.348 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.348 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.348 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.348 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.849 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.849 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.849 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.849 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.849 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.849 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.849 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.849 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:35.850 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.850 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.850 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.850 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.850 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.850 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.850 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.850 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:35.859 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.859 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.859 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.859 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.859 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.859 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.859 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:35.859 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6709.35it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6131.17it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 10933.51it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7788.52it/s] |
|
|
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 10061.73it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7569.64it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7479.02it/s] |
|
|
2025-07-01 09:10:37.437 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:37.437 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:37.437 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:37.437 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:37.437 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:37.437 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:37.437 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:37.437 | INFO | llava.data.builder:register_datasets:39 - Registering datasets from environment: 'nvidia,draco-oci-iad'. |
|
|
2025-07-01 09:10:37.438 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:37.438 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:37.438 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:37.438 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:37.438 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:37.438 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:37.438 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:37.438 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/nvidia.yaml'. |
|
|
2025-07-01 09:10:37.447 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:37.447 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:37.447 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:37.447 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:37.447 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:37.447 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:37.447 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
2025-07-01 09:10:37.448 | INFO | llava.data.builder:register_datasets:44 - Registering datasets from: '/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/data/registry/datasets/draco-oci-iad.yaml'. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7429.18it/s] |
|
|
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7792.66it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8480.68it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9139.81it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6384.95it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6651.09it/s] |
|
|
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6589.88it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7109.56it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers |
|
|
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
Did not find AutoResume SDK! |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/training_args.py:1559: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead |
|
|
warnings.warn( |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6114.99it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6666.19it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6767.09it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 5388.83it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7783.02it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6963.98it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7444.25it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8388.61it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9515.00it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7917.34it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6034.97it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6913.69it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6356.38it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8130.00it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7299.28it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8578.14it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 5765.56it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7983.36it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7462.54it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 10852.68it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7087.25it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8101.58it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8695.86it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7340.03it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9034.81it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7230.96it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7338.81it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6829.00it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7137.22it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8151.82it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 9267.72it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7987.70it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 11666.28it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8648.90it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7111.86it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6680.85it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6345.39it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8934.91it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7196.11it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 11528.85it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6086.26it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 6922.38it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7310.18it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8038.00it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7345.54it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 8634.49it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 7395.50it/s] |
|
|
Fetching 21 files: 0%| | 0/21 [00:00<?, ?it/s]
Fetching 21 files: 100%|██████████| 21/21 [00:00<00:00, 11680.20it/s] |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`. |
|
|
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.54s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.55s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it]
Loading checkpoint shards: 2
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.72s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.63s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.63s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.72s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.77s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.77s/it]
Loading checkpoint shards: 2
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.58s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.53s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.58s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.61s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.63s/it]
Loading checkpoint shards: 2
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.58s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.58s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.59s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.68s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.67s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.67s/it]
Loading checkpoint shards: 2
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.57s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.57s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.62s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.62s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.63s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.66s/it]
Loading checkpoint shards: 2
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.56s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.58s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.65s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it]
Loading checkpoint shards: 2
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.70s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.70s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.80s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.80s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.80s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.80s/it]
Loading checkpoint shards: 2
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.53s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.63s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.68s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.62s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.67s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.67s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.67s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.67s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:13, 4.65s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.62s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:13, 4.64s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.64s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.83s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:14, 4.78s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.79s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.91s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.91s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.93s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.93s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.93s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.93s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.70s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:14, 4.68s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.76s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.81s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.99s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:13<00:04, 4.65s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.85s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.52s/it] |
|
|
5%|██▌ | 1/4 [00:04<00:13, 4.67s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.66s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.87s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it]
Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:13, 4.65s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:13, 4.60s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.83s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.84s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.85s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.88s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.86s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████5%|██▌ | 1/4 [00:04<00:14, 4.79s/it]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:04<00:14, 4.80s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.89s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.94s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.94s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.95s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.94s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.94s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:09<00:09, 4.94s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.70s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.70s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.88s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.88s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it] |
|
|
▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.88s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it] |
|
|
▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it] |
|
|
▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.70s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.72s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.66s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.68s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.56s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
▌ | 3/4 [00:14<00:04, 4.66s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.67s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.89s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.55s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.90s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.62s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.91s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.57s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it] |
|
|
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.70s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.70s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.69s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.92s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.58s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.96s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.64s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.96s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.64s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.96s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.64s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.96s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.96s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.64s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.64s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.96s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.64s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.96s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.64s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.96s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.64s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.93s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.95s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.95s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.59s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.95s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.95s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.95s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.95s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.94s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.60s/it] |
|
|
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 2.95s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:14<00:00, 3.61s/it] |
|
|
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:14<00:04, 4.82s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:15<00:00, 3.25s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:15<00:00, 3.84s/it] |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
The new lm_head weights will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False` |
|
|
2025-07-01 09:13:55.597 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:55.644 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:55.649 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:55.692 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:55.810 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:55.872 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:55.887 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:55.959 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:55.965 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:55.966 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:55.971 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:55.975 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:55.984 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.055 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.060 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.076 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.078 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.083 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.083 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.110 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.129 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.137 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.142 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.144 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.154 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.156 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.162 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.167 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.183 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.184 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.187 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.192 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.204 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.213 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.215 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.233 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.256 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.256 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.259 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.260 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.275 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.276 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.277 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.278 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.278 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.287 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.287 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.294 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.302 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.312 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.317 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.317 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.320 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.328 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.334 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.338 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.341 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.360 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.379 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.380 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
2025-07-01 09:13:56.380 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.406 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.416 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
2025-07-01 09:13:56.438 | WARNING | llava.data.builder:build_dataset:92 - Training VILA with mixture 'vila_quantumn_qa_train'. |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py:592: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `LLaVATrainer.__init__`. Use `processing_class` instead. |
|
|
super().__init__(*args, **kwargs) |
|
|
wandb: Currently logged in as: ligeng-zhu to https://api.wandb.ai. Use `wandb login --relogin` to force relogin |
|
|
wandb: creating run |
|
|
wandb: Tracking run with wandb version 0.20.1 |
|
|
wandb: Run data is saved locally in runs/train/NVILA-Lite-8B-quantumn-qa-train/wandb/run-20250701_091410-NVILA-Lite-8B-quantumn-qa-train |
|
|
wandb: Run `wandb offline` to turn off syncing. |
|
|
wandb: Syncing run NVILA-Lite-8B-quantumn-qa-train |
|
|
wandb: ⭐️ View project at https://wandb.ai/ligeng-zhu/vila |
|
|
wandb: 🚀 View run at https://wandb.ai/ligeng-zhu/vila/runs/NVILA-Lite-8B-quantumn-qa-train |
|
|
0%| | 0/7 [00:00<?, ?it/s]/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/model/llava_arch.py:534: UserWarning: Truncating sequences to `model_max_length` (4096). |
|
|
warnings.warn(f"Truncating sequences to `model_max_length` ({self.tokenizer.model_max_length}).") |
|
|
[rank63]: Traceback (most recent call last): |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/train_mem.py", line 49, in <module> |
|
|
[rank63]: train() |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/train.py", line 906, in train |
|
|
[rank63]: trainer.train(resume_from_checkpoint=resume_from_checkpoint) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/trainer.py", line 2122, in train |
|
|
[rank63]: return inner_training_loop( |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/llava_trainer.py", line 665, in _inner_training_loop |
|
|
[rank63]: return super()._inner_training_loop(batch_size, *args, **kwargs) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/transformers/trainer.py", line 2474, in _inner_training_loop |
|
|
[rank63]: tr_loss_step = self.training_step(model, inputs, num_items_in_batch) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/llava/train/transformer_normalize_monkey_patch.py", line 247, in training_step |
|
|
[rank63]: self.accelerator.backward(loss, **kwargs) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/accelerate/accelerator.py", line 2188, in backward |
|
|
[rank63]: self.deepspeed_engine_wrapped.backward(loss, **kwargs) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 166, in backward |
|
|
[rank63]: self.engine.backward(loss, **kwargs) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn |
|
|
[rank63]: ret_val = func(*args, **kwargs) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1861, in backward |
|
|
[rank63]: self.optimizer.backward(loss, retain_graph=retain_graph) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn |
|
|
[rank63]: ret_val = func(*args, **kwargs) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/deepspeed/runtime/zero/stage3.py", line 1993, in backward |
|
|
[rank63]: self.loss_scaler.backward(loss.float(), retain_graph=retain_graph) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/deepspeed/runtime/fp16/loss_scaler.py", line 63, in backward |
|
|
[rank63]: scaled_loss.backward(retain_graph=retain_graph) |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/_tensor.py", line 525, in backward |
|
|
[rank63]: torch.autograd.backward( |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/autograd/__init__.py", line 267, in backward |
|
|
[rank63]: _engine_run_backward( |
|
|
[rank63]: File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/autograd/graph.py", line 744, in _engine_run_backward |
|
|
[rank63]: return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass |
|
|
[rank63]: torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 15.07 GiB. GPU has a total capacity of 79.10 GiB of which 14.49 GiB is free. Including non-PyTorch memory, this process has 64.58 GiB memory in use. Of the allocated memory 56.11 GiB is allocated by PyTorch, and 2.37 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) |
|
|
W0701 09:14:43.717000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 909056 closing signal SIGTERM |
|
|
W0701 09:14:43.719000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 909057 closing signal SIGTERM |
|
|
W0701 09:14:43.720000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 909058 closing signal SIGTERM |
|
|
W0701 09:14:43.721000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 909059 closing signal SIGTERM |
|
|
W0701 09:14:43.723000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 909060 closing signal SIGTERM |
|
|
W0701 09:14:43.724000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 909061 closing signal SIGTERM |
|
|
W0701 09:14:43.725000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 909062 closing signal SIGTERM |
|
|
E0701 09:14:53.609000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: 1) local_rank: 7 (pid: 909063) of binary: /lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/bin/python3 |
|
|
Traceback (most recent call last): |
|
|
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/bin/torchrun", line 10, in <module> |
|
|
sys.exit(main()) |
|
|
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 347, in wrapper |
|
|
return f(*args, **kwargs) |
|
|
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/run.py", line 879, in main |
|
|
run(args) |
|
|
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/run.py", line 870, in run |
|
|
elastic_launch( |
|
|
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in __call__ |
|
|
return launch_agent(self._config, self._entrypoint, list(args)) |
|
|
File "/lustre/fs1/portfolios/nvr/projects/nvr_elm_llm/users/ligengz/workspace/vila-q/.venv/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 263, in launch_agent |
|
|
raise ChildFailedError( |
|
|
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: |
|
|
============================================================ |
|
|
llava/train/train_mem.py FAILED |
|
|
------------------------------------------------------------ |
|
|
Failures: |
|
|
<NO_OTHER_FAILURES> |
|
|
------------------------------------------------------------ |
|
|
Root Cause (first observed failure): |
|
|
[0]: |
|
|
time : 2025-07-01_09:14:43 |
|
|
host : pool0-02015.cm.cluster |
|
|
rank : 63 (local_rank: 7) |
|
|
exitcode : 1 (pid: 909063) |
|
|
error_file: <N/A> |
|
|
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html |
|
|
============================================================ |
|
|
Jul 01 09:14:54.370040 1375942 slurmstepd 0x1555438fa640: error: *** STEP 1038286.0 ON pool0-02124 CANCELLED AT 2025-07-01T09:14:54 *** |
|
|
W0701 09:14:54.371000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers |
|
|
W0701 09:14:54.371000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers |
|
|
W0701 09:14:54.371000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers |
|
|
W0701 09:14:54.371000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 4024631 closing signal SIGTERM |
|
|
W0701 09:14:54.371000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3532828 closing signal SIGTERM |
|
|
W0701 09:14:54.372000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1376010 closing signal SIGTERM |
|
|
W0701 09:14:54.371000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers |
|
|
W0701 09:14:54.371000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers |
|
|
W0701 09:14:54.372000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1853391 closing signal SIGTERM |
|
|
W0701 09:14:54.372000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3926495 closing signal SIGTERM |
|
|
W0701 09:14:54.372000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3926496 closing signal SIGTERM |
|
|
W0701 09:14:54.371000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers |
|
|
W0701 09:14:54.372000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3060110 closing signal SIGTERM |
|
|
W0701 09:14:54.372000 23456244200576 torch/distributed/elastic/agent/server/api.py:741] Received Signals.SIGTERM death signal, shutting down workers |
|
|
W0701 09:14:54.372000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2511052 closing signal SIGTERM |
|
|
W0701 09:14:54.372000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 4024633 closing signal SIGTERM |
|
|
W0701 09:14:54.373000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1376011 closing signal SIGTERM |
|
|
W0701 09:14:54.373000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1853392 closing signal SIGTERM |
|
|
W0701 09:14:54.373000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3060111 closing signal SIGTERM |
|
|
W0701 09:14:54.373000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3532829 closing signal SIGTERM |
|
|
W0701 09:14:54.374000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2511053 closing signal SIGTERM |
|
|
W0701 09:14:54.373000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 4024635 closing signal SIGTERM |
|
|
W0701 09:14:54.374000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1853393 closing signal SIGTERM |
|
|
W0701 09:14:54.374000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3532830 closing signal SIGTERM |
|
|
W0701 09:14:54.375000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3926497 closing signal SIGTERM |
|
|
W0701 09:14:54.375000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1376012 closing signal SIGTERM |
|
|
W0701 09:14:54.375000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3060112 closing signal SIGTERM |
|
|
W0701 09:14:54.375000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2511054 closing signal SIGTERM |
|
|
W0701 09:14:54.376000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1853394 closing signal SIGTERM |
|
|
W0701 09:14:54.376000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3532831 closing signal SIGTERM |
|
|
W0701 09:14:54.376000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1376013 closing signal SIGTERM |
|
|
W0701 09:14:54.376000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2511055 closing signal SIGTERM |
|
|
W0701 09:14:54.375000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 4024636 closing signal SIGTERM |
|
|
W0701 09:14:54.376000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3060113 closing signal SIGTERM |
|
|
W0701 09:14:54.376000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3926498 closing signal SIGTERM |
|
|
W0701 09:14:54.377000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1853395 closing signal SIGTERM |
|
|
W0701 09:14:54.377000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3532832 closing signal SIGTERM |
|
|
W0701 09:14:54.377000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 4024637 closing signal SIGTERM |
|
|
W0701 09:14:54.378000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3926499 closing signal SIGTERM |
|
|
W0701 09:14:54.378000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1376014 closing signal SIGTERM |
|
|
W0701 09:14:54.378000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1853396 closing signal SIGTERM |
|
|
W0701 09:14:54.379000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3926500 closing signal SIGTERM |
|
|
W0701 09:14:54.378000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 4024638 closing signal SIGTERM |
|
|
W0701 09:14:54.379000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1376015 closing signal SIGTERM |
|
|
W0701 09:14:54.378000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3060114 closing signal SIGTERM |
|
|
W0701 09:14:54.379000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3532833 closing signal SIGTERM |
|
|
W0701 09:14:54.379000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2511056 closing signal SIGTERM |
|
|
W0701 09:14:54.379000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3060115 closing signal SIGTERM |
|
|
W0701 09:14:54.380000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1853397 closing signal SIGTERM |
|
|
W0701 09:14:54.380000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2511057 closing signal SIGTERM |
|
|
W0701 09:14:54.380000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1376016 closing signal SIGTERM |
|
|
W0701 09:14:54.380000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 4024639 closing signal SIGTERM |
|
|
W0701 09:14:54.380000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3532834 closing signal SIGTERM |
|
|
W0701 09:14:54.381000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3926501 closing signal SIGTERM |
|
|
W0701 09:14:54.381000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1853398 closing signal SIGTERM |
|
|
W0701 09:14:54.381000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2511058 closing signal SIGTERM |
|
|
W0701 09:14:54.381000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 4024640 closing signal SIGTERM |
|
|
W0701 09:14:54.381000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3060116 closing signal SIGTERM |
|
|
W0701 09:14:54.382000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3926502 closing signal SIGTERM |
|
|
W0701 09:14:54.382000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3532835 closing signal SIGTERM |
|
|
W0701 09:14:54.383000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1376017 closing signal SIGTERM |
|
|
W0701 09:14:54.384000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 2511059 closing signal SIGTERM |
|
|
W0701 09:14:54.383000 23456244200576 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 3060117 closing signal SIGTERM |
|
|
|
