google/gemma-2-2b-jpn-it · hidden_act is missing　config.json

2 days ago

thank you for great work!

diff /gemma-2-2b-jpn-it/config.json /gemma-2-2b-it/config.json 
10,11c10,13
<   "dtype": "bfloat16",
<   "eos_token_id": 1,
---
>   "eos_token_id": [
>     1,
>     107
>   ],
13a16
>   "hidden_act": "gelu_pytorch_tanh",
24c27
<   "query_pre_attn_scalar": 224,
---
>   "query_pre_attn_scalar": 256,
29c32
<   "transformers_version": "4.44.2",
---
>   "transformers_version": "4.42.4",

"hidden_act": "gelu_pytorch_tanh", cause problem.
I try to serve gemma-2-2b-jpn-it with vLLM, it raise Error.

   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/gemma2.py", line 421, in __init__
     self.model = Gemma2Model(config, cache_config, quant_config)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/gemma2.py", line 265, in __init__
     self.start_layer, self.end_layer, self.layers = make_layers(
                                                     ^^^^^^^^^^^^
   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/utils.py", line 408, in make_layers
     maybe_offload_to_cpu(layer_fn(prefix=f"{prefix}.{idx}"))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/gemma2.py", line 267, in <lambda>
     lambda prefix: Gemma2DecoderLayer(int(prefix.split(".")[
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/gemma2.py", line 200, in __init__
     hidden_act=config.hidden_act,
                ^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.12/dist-packages/transformers/configuration_utils.py", line 202, in __getattribute__
     return super().__getattribute__(key)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 AttributeError: 'Gemma2Config' object has no attribute 'hidden_act'. Did you mean: 'hidden_size'?

I copy and paste hidden_act line from /gemma-2-2b-it/config.json it works.

s-natsu changed discussion title from config,json diff from gemma-2-2b-it to hidden_act is missing　config.json 2 days ago

GopiUppari

Google org 1 day ago

Hi @s-natsu ,

hidden_act is a legacy or deprecated parameter in some configurations, but it exists for backward compatibility with older versions of models or configurations. This parameter is overwritten by hidden_activation.
hidden_act and hidden_size are different. Because hidden_act defines defines the non-linear activation function used in the model and hidden_size defines dimensionality of the hidden layers.

For further information, could you please refer to this reference

Thank you.

hidden_act is missing config.json

hidden_act is missing　config.json