Spaces:

---

lightonai

/

MCPyLate

like 8

Running

Hey @NohTow , Spaces already have 50GB of ephemeral storage. If you are trying to add some large files directly to the Space repo, then we can suggest uploading the datasets to a dataset repo and downloading them at the startup time of the Space. Let me know if this works for you?

Hey,

The files are not parquet-like files, they are a combination of .pt, .json and .sqlite files.
Does it work anyways? Could you share some pointer on how to use this?

Edit: else I can make it so the indexes are built at startup, but it would be quite slow and would require GPU. I think it's best to use the indexes I pre-computed.

The leetcode split of BRIGHT is 4.2Go, could be an interesting demo as well.

Tagging @hysts for his views and help on this.

Hi @NohTow
To download a dataset repo, you can use snapshot_download() of the huggingface-hub library. Docs: https://huggingface.co/docs/huggingface_hub/main/en/guides/download

Thanks @hysts ❤️

Thanks indeed, managed to make it work, the space is now working correctly!