Dedicated Endpoint deployment fails for canary-1b
#26
by
anemdat
- opened
I've tried creating a dedicated endpoint for canary-1b
here on HuggingFace (https://ui.endpoints.huggingface.co/<my_hf_account>/endpoints/dedicated
), and got this error:
repository does not appear to have a file named config.json.
Is there any workaround for this? To compare, I deployed whisper-large-v3
in the same way without any issues.
The log for canary-1b
endpoint deployment:
- 2024-06-15T12:10:48.278+00:00 /usr/local/lib/python3.10/dist-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
- 2024-06-15T12:10:48.278+00:00 torch.utils._pytree._register_pytree_node(
- 2024-06-15T12:10:48.342+00:00 2024-06-15 12:10:48,342 | INFO | Initializing model from directory:/repository
- 2024-06-15T12:10:48.342+00:00 2024-06-15 12:10:48,342 | INFO | No custom pipeline found at /repository/handler.py
- 2024-06-15T12:10:48.342+00:00 2024-06-15 12:10:48,342 | INFO | Using device CPU
- 2024-06-15T12:10:48.343+00:00 Traceback (most recent call last):
- 2024-06-15T12:10:48.343+00:00 File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 732, in lifespan
- 2024-06-15T12:10:48.343+00:00 async with self.lifespan_context(app) as maybe_state:
- 2024-06-15T12:10:48.343+00:00 File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 608, in __aenter__
- 2024-06-15T12:10:48.343+00:00 await self._router.startup()
- 2024-06-15T12:10:48.343+00:00 File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 709, in startup
- 2024-06-15T12:10:48.343+00:00 await handler()
- 2024-06-15T12:10:48.343+00:00 File "/app/webservice_starlette.py", line 60, in some_startup_task
- 2024-06-15T12:10:48.343+00:00 inference_handler = get_inference_handler_either_custom_or_default_handler(HF_MODEL_DIR, task=HF_TASK)
- 2024-06-15T12:10:48.343+00:00 File "/app/huggingface_inference_toolkit/handler.py", line 54, in get_inference_handler_either_custom_or_default_handler
- 2024-06-15T12:10:48.343+00:00 return HuggingFaceHandler(model_dir=model_dir, task=task)
- 2024-06-15T12:10:48.343+00:00 File "/app/huggingface_inference_toolkit/handler.py", line 18, in __init__
- 2024-06-15T12:10:48.343+00:00 self.pipeline = get_pipeline(
- 2024-06-15T12:10:48.343+00:00 File "/app/huggingface_inference_toolkit/utils.py", line 276, in get_pipeline
- 2024-06-15T12:10:48.343+00:00 hf_pipeline = pipeline(
- 2024-06-15T12:10:48.343+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/__init__.py", line 815, in pipeline
- 2024-06-15T12:10:48.343+00:00 config = AutoConfig.from_pretrained(
- 2024-06-15T12:10:48.343+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1111, in from_pretrained
- 2024-06-15T12:10:48.343+00:00 config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
- 2024-06-15T12:10:48.343+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 633, in get_config_dict
- 2024-06-15T12:10:48.343+00:00 config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
- 2024-06-15T12:10:48.343+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 688, in _get_config_dict
- 2024-06-15T12:10:48.343+00:00 resolved_config_file = cached_file(
- 2024-06-15T12:10:48.343+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 369, in cached_file
- 2024-06-15T12:10:48.343+00:00 raise EnvironmentError(
- 2024-06-15T12:10:48.343+00:00 OSError: /repository does not appear to have a file named config.json. Checkout 'https://huggingface.co//repository/None' for available files.
- 2024-06-15T12:10:48.343+00:00
- 2024-06-15T12:10:48.343+00:00 Application startup failed. Exiting.
- 2024-06-15T12:10:51.377+00:00 /usr/local/lib/python3.10/dist-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
- 2024-06-15T12:10:51.377+00:00 torch.utils._pytree._register_pytree_node(
- 2024-06-15T12:10:51.445+00:00 2024-06-15 12:10:51,445 | INFO | Initializing model from directory:/repository
- 2024-06-15T12:10:51.445+00:00 2024-06-15 12:10:51,445 | INFO | No custom pipeline found at /repository/handler.py
- 2024-06-15T12:10:51.445+00:00 2024-06-15 12:10:51,445 | INFO | Using device CPU
- 2024-06-15T12:10:51.446+00:00 Traceback (most recent call last):
- 2024-06-15T12:10:51.446+00:00 File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 732, in lifespan
- 2024-06-15T12:10:51.446+00:00 async with self.lifespan_context(app) as maybe_state:
- 2024-06-15T12:10:51.446+00:00 File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 608, in __aenter__
- 2024-06-15T12:10:51.446+00:00 await self._router.startup()
- 2024-06-15T12:10:51.446+00:00 File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 709, in startup
- 2024-06-15T12:10:51.446+00:00 await handler()
- 2024-06-15T12:10:51.446+00:00 File "/app/webservice_starlette.py", line 60, in some_startup_task
- 2024-06-15T12:10:51.446+00:00 inference_handler = get_inference_handler_either_custom_or_default_handler(HF_MODEL_DIR, task=HF_TASK)
- 2024-06-15T12:10:51.446+00:00 File "/app/huggingface_inference_toolkit/handler.py", line 54, in get_inference_handler_either_custom_or_default_handler
- 2024-06-15T12:10:51.446+00:00 return HuggingFaceHandler(model_dir=model_dir, task=task)
- 2024-06-15T12:10:51.446+00:00 File "/app/huggingface_inference_toolkit/handler.py", line 18, in __init__
- 2024-06-15T12:10:51.446+00:00 self.pipeline = get_pipeline(
- 2024-06-15T12:10:51.446+00:00 File "/app/huggingface_inference_toolkit/utils.py", line 276, in get_pipeline
- 2024-06-15T12:10:51.446+00:00 hf_pipeline = pipeline(
- 2024-06-15T12:10:51.446+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/__init__.py", line 815, in pipeline
- 2024-06-15T12:10:51.446+00:00 config = AutoConfig.from_pretrained(
- 2024-06-15T12:10:51.446+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1111, in from_pretrained
- 2024-06-15T12:10:51.446+00:00 config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
- 2024-06-15T12:10:51.446+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 633, in get_config_dict
- 2024-06-15T12:10:51.446+00:00 config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
- 2024-06-15T12:10:51.446+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 688, in _get_config_dict
- 2024-06-15T12:10:51.446+00:00 resolved_config_file = cached_file(
- 2024-06-15T12:10:51.446+00:00 File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 369, in cached_file
- 2024-06-15T12:10:51.446+00:00 raise EnvironmentError(
- 2024-06-15T12:10:51.446+00:00 OSError: /repository does not appear to have a file named config.json. Checkout 'https://huggingface.co//repository/None' for available files.
- 2024-06-15T12:10:51.446+00:00
- 2024-06-15T12:10:51.446+00:00 Application startup failed. Exiting.
Canary requires NeMo ASR to run. It might be possible to convert the checkpoint to HF format but that is not on our list of things to do at the moment.
Any chance of getting a HF format? Thanks.