Update app.py
Browse files
app.py
CHANGED
@@ -77,7 +77,7 @@ if __name__ == "__main__":
|
|
77 |
The grammar is defined in `json_minimal.ebnf` and is written in the **Extended Backus-Naur Form (EBNF)**.
|
78 |
|
79 |
Internally, it relies on the library [`transformers-cfg`](https://github.com/epfl-dlab/transformers-CFG).
|
80 |
-
For demo purpose, gpt2
|
81 |
|
82 |
The inference is a bit slow because of the inference is run on **CPU(~20s for 30 tokens)**.
|
83 |
The constraint itself **doesn't** introduce significant overhead to the inference.
|
|
|
77 |
The grammar is defined in `json_minimal.ebnf` and is written in the **Extended Backus-Naur Form (EBNF)**.
|
78 |
|
79 |
Internally, it relies on the library [`transformers-cfg`](https://github.com/epfl-dlab/transformers-CFG).
|
80 |
+
For demo purpose, gpt2 is used, but you can use much larger models for better performance.
|
81 |
|
82 |
The inference is a bit slow because of the inference is run on **CPU(~20s for 30 tokens)**.
|
83 |
The constraint itself **doesn't** introduce significant overhead to the inference.
|