metadata

license: apache-2.0

Model Card for Model ID

slim-sentiment is part of the SLIM ("Structured Language Instruction Model") model series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.

slim-sentiment has been fine-tuned for sentiment analysis function calls, generating output consisting of a python dictionary corresponding to specified keys.

Each slim model has a corresponding 'tool' in a separate repository, e.g.,

'slim-sentiment-tool', which a 4-bit quantized gguf version of the model that is intended to be used for inference.

Inference speed and loading time is much faster with the 'tool' versions of the model, and multiple tools can be deployed concurrently and run on a local CPU-based laptop or server.

Model Description

Developed by: llmware
Model type: SLIM - small, specialized LLM
Language(s) (NLP): English
License: Apache 2.0
Finetuned from model: Tiny Llama 1B

Uses

The intended use of SLIM models is to re-imagine traditional 'hard-coded' classifiers through the use of function calls, and to provide a natural language flexible tool that can be used as decision gates and processing steps in a complex LLM-based automation workflow.

Example:

text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."   

model generation output-  {"sentiment": ["negative"]}

function = "classify"  
keys = "sentiment"

All of the SLIM models use a novel prompt instruction structured as follows:

"<human> " + {text} + "\n" +

"<{function}> " + {keys} + "</{function}>" + 

"/n<bot>:"

For example, in this case, the prompt would be as follows:

"<human>" + "The stock market declined yesterday ..." + "\n" + "<classify> sentiment </classify>" + "\n<bot>:"

The model generation output will be a string in the form of a well-formed python dictionary, which can be converted as follows:

try:
    # convert llm response output from string to json
    output_only = ast.literal_eval(output_only)
    print("converted to python dictionary automatically")

    #   look for the key passed in the prompt as a dictionary entry
    if keys in output_only:
        if "negative" in output_only[keys]:
            print("sentiment appears negative - need to handle ...")
    else:
        print("response does not appear to include the designated key - will need to try again.")

except:
    print("could not convert to python dictionary automatically - ", output_only)

How to Get Started with the Model

The fastest way to get started with SLIM is through direct import in transformers:

import ast
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")

text = "The markets declined for a second straight days on news of disappointing earnings."

keys = "sentiment"

prompt = "<human>: " + text + "\n" + "<classify> " + keys + "</classify>" + "\n<bot>: "

#   huggingface standard generation script
inputs = tokenizer(prompt, return_tensors="pt")
start_of_output = len(inputs.input_ids[0])

outputs = model.generate(inputs.input_ids.to('cpu'), eos_token_id=tokenizer.eos_token_id,
                         pad_token_id=tokenizer.eos_token_id, do_sample=True, temperature=0.3, max_new_tokens=100)

output_only = tokenizer.decode(outputs[0][start_of_output:], skip_special_tokens=True)

print("input text sample - ", text)
print("llm_response - ", output_only)

#   where it gets interesting

Using as Function Call in LLMWare

We envision the slim models deployed in a pipeline/workflow/templating framework that handles the prompt packaging more elegantly.

Check out llmware for one such implementation:

from llmware.models import ModelCatalog
slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
response = slim_model.function_call(text,params=["sentiment"], function="classify")

print("llmware - llm_response: ", response)

Model Card Contact

Darren Oberst & llmware team