Hugging Face's logo

Spaces:
DesertWolf
/
test3
Configuration error

App Files Community
test3 / docs /my-website /release_notes /v1.56.3 /index.md
DesertWolf's picture
DesertWolf
Upload folder using huggingface_hub
447ebeb verified
preview code
|
raw
history blame contribute delete
3.08 kB
metadata
title: v1.56.3
slug: v1.56.3
date: 2024-12-28T10:00:00.000Z
authors:
  - name: Krrish Dholakia
    title: CEO, LiteLLM
    url: https://www.linkedin.com/in/krish-d/
    image_url: >-
      https://media.licdn.com/dms/image/v2/D4D03AQGrlsJ3aqpHmQ/profile-displayphoto-shrink_400_400/B4DZSAzgP7HYAg-/0/1737327772964?e=1749686400&v=beta&t=Hkl3U8Ps0VtvNxX0BNNq24b4dtX5wQaPFp6oiKCIHD8
  - name: Ishaan Jaffer
    title: CTO, LiteLLM
    url: https://www.linkedin.com/in/reffajnaahsi/
    image_url: >-
      https://media.licdn.com/dms/image/v2/D4D03AQGiM7ZrUwqu_Q/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1675971026692?e=1741824000&v=beta&t=eQnRdXPJo4eiINWTZARoYTfqh064pgZ-E21pQTSy8jc
tags:
  - guardrails
  - logging
  - virtual key management
  - new models
hide_table_of_contents: false

import Image from '@theme/IdealImage';

guardrails, logging, virtual key management, new models

:::info

Get a 7 day free trial for LiteLLM Enterprise here.

no call needed

:::

New Features

✨ Log Guardrail Traces

Track guardrail failure rate and if a guardrail is going rogue and failing requests. Start here

Traced Guardrail Success

<Image img={require('../../img/gd_success.png')} />

Traced Guardrail Failure

<Image img={require('../../img/gd_fail.png')} />

/guardrails/list

/guardrails/list allows clients to view available guardrails + supported guardrail params

curl -X GET 'http://0.0.0.0:4000/guardrails/list'

Expected response

{
    "guardrails": [
        {
        "guardrail_name": "aporia-post-guard",
        "guardrail_info": {
            "params": [
            {
                "name": "toxicity_score",
                "type": "float",
                "description": "Score between 0-1 indicating content toxicity level"
            },
            {
                "name": "pii_detection",
                "type": "boolean"
            }
            ]
        }
        }
    ]
}

✨ Guardrails with Mock LLM

Send mock_response to test guardrails without making an LLM call. More info on mock_response here

curl -i http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is [email protected]"}
    ],
    "mock_response": "This is a mock response",
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

Assign Keys to Users

You can now assign keys to users via Proxy UI

<Image img={require('../../img/ui_key.png')} />

New Models

  • openrouter/openai/o1
  • vertex_ai/mistral-large@2411

Fixes