tiiuae/Falcon3-1B-Base · Falcon3-1B-Base issue with

When attempting to download Falcon3-1B-Base with litgpt I get the below error, which is generated because the download package only includes a single model.safetensors file but the model.safetensors.index.json index json expects both model-00001-of-00002.safetensors and model-00002-of-00002.safetensors. This was confirmed by manual inspection of the files in the tiiuae/Falcon3-1B-Base repository.

This suggests that the wrong model.safetensors.index.json is being used; by comparison with other Falcon models' index files it appears that this is probably an index file from a Falcon3-3B model.

Could you please update the model.safetensors.index.json file to ensure that downstream systems like litgpt which rely in the index.json file don't fail?

$ litgpt download tiiuae/Falcon3-1B-Base
Setting HF_HUB_ENABLE_HF_TRANSFER=1
Converting checkpoint files to LitGPT format.
{'checkpoint_dir': PosixPath('checkpoints/tiiuae/Falcon3-1B-Base'),
 'debug_mode': False,
 'dtype': None,
 'model_name': None}
Traceback (most recent call last):
  File "/opt/anaconda3/envs/lightning/bin/litgpt", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/anaconda3/envs/lightning/lib/python3.12/site-packages/litgpt/__main__.py", line 71, in main
    CLI(parser_data)
  File "/opt/anaconda3/envs/lightning/lib/python3.12/site-packages/jsonargparse/_cli.py", line 119, in CLI
    return _run_component(component, init.get(subcommand))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lightning/lib/python3.12/site-packages/jsonargparse/_cli.py", line 204, in _run_component
    return component(**cfg)
           ^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lightning/lib/python3.12/site-packages/litgpt/scripts/download.py", line 99, in download_from_hub
    convert_hf_checkpoint(checkpoint_dir=directory, dtype=dtype, model_name=model_name)
  File "/opt/anaconda3/envs/lightning/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lightning/lib/python3.12/site-packages/litgpt/scripts/convert_hf_checkpoint.py", line 575, in convert_hf_checkpoint
    total_size = max(1, sum(os.path.getsize(bin_file) for bin_file in bin_files))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/lightning/lib/python3.12/site-packages/litgpt/scripts/convert_hf_checkpoint.py", line 575, in <genexpr>
    total_size = max(1, sum(os.path.getsize(bin_file) for bin_file in bin_files))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen genericpath>", line 62, in getsize
FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/tiiuae/Falcon3-1B-Base/model-00001-of-00002.safetensors'

Also reported here:
https://github.com/Lightning-AI/litgpt/issues/1954