Allow local models to be loaded

Increase the temperature again

Add ability to configure model name

Increase default context

Adjust temperature

Command line argument for context size

Add code to select the number of GPU layers

Command line flag to chose a local model

Reduce the context size again

Increase context to 1024 and use larger example prompt.

Add description and title

Fixed stop tokens and increased line limit now that the model is better

Changed the input boxes a bit

Back to one line

Log the generated text

Move back to 8bit quantization

Unused import

Use fp16 model

Break on a single #

Allow multiline processing

Streaming code working

Initial attempt at streaming

Use the quantized model

Switched to using llama_cpp

Try to load the model in 8 bits

Another fix

Also parse correctly in the absense of a newline

Use a fake area prefix

WiP

A few tweaks

Update app.py

Create app.py