transmogrifier
commited on
Commit
•
277bc96
1
Parent(s):
6395e6f
Upload model card
Browse files
README.md
CHANGED
@@ -14,7 +14,7 @@ thumbnail: https://h2o.ai/etc.clientlibs/h2o/clientlibs/clientlib-site/resources
|
|
14 |
## Summary
|
15 |
|
16 |
This model was trained using [H2O LLM Studio](https://github.com/h2oai/h2o-llmstudio).
|
17 |
-
- Base model: [tiiuae/falcon-7b](https://huggingface.co/tiiuae/falcon-7b)
|
18 |
|
19 |
|
20 |
## Usage
|
@@ -149,7 +149,7 @@ You can load the models using quantization by specifying ```load_in_8bit=True```
|
|
149 |
```
|
150 |
RWForCausalLM(
|
151 |
(transformer): RWModel(
|
152 |
-
(word_embeddings): Embedding(
|
153 |
(h): ModuleList(
|
154 |
(0-31): 32 x DecoderLayer(
|
155 |
(input_layernorm): LayerNorm((4544,), eps=1e-05, elementwise_affine=True)
|
@@ -168,7 +168,7 @@ RWForCausalLM(
|
|
168 |
)
|
169 |
(ln_f): LayerNorm((4544,), eps=1e-05, elementwise_affine=True)
|
170 |
)
|
171 |
-
(lm_head): Linear(in_features=4544, out_features=
|
172 |
)
|
173 |
```
|
174 |
|
|
|
14 |
## Summary
|
15 |
|
16 |
This model was trained using [H2O LLM Studio](https://github.com/h2oai/h2o-llmstudio).
|
17 |
+
- Base model: [tiiuae/falcon-7b-instruct](https://huggingface.co/tiiuae/falcon-7b-instruct)
|
18 |
|
19 |
|
20 |
## Usage
|
|
|
149 |
```
|
150 |
RWForCausalLM(
|
151 |
(transformer): RWModel(
|
152 |
+
(word_embeddings): Embedding(65027, 4544)
|
153 |
(h): ModuleList(
|
154 |
(0-31): 32 x DecoderLayer(
|
155 |
(input_layernorm): LayerNorm((4544,), eps=1e-05, elementwise_affine=True)
|
|
|
168 |
)
|
169 |
(ln_f): LayerNorm((4544,), eps=1e-05, elementwise_affine=True)
|
170 |
)
|
171 |
+
(lm_head): Linear(in_features=4544, out_features=65027, bias=False)
|
172 |
)
|
173 |
```
|
174 |
|