Hugging Face's logo

Spaces:
DesertWolf
/
test3
Configuration error

App Files Community
test3
File size: 4,283 Bytes 
447ebeb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
 
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eKXncoQbU_2j"
   },
   "source": [
    "# Using Nemo-Guardrails with LiteLLM Server\n",
    "\n",
    "[Call Bedrock, TogetherAI, Huggingface, etc. on the server](https://docs.litellm.ai/docs/providers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZciYaLwvuFbu"
   },
   "source": [
    "## Using with Bedrock\n",
    "\n",
    "`docker run -e PORT=8000 -e AWS_ACCESS_KEY_ID=<your-aws-access-key> -e AWS_SECRET_ACCESS_KEY=<your-aws-secret-key> -p 8000:8000 ghcr.io/berriai/litellm:latest`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vOUwGSJ2Vsy3"
   },
   "outputs": [],
   "source": [
    "pip install nemoguardrails langchain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xXEJNxe7U0IN"
   },
   "outputs": [],
   "source": [
    "from langchain.chat_models import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(model_name=\"anthropic.claude-v2\", openai_api_base=\"http://0.0.0.0:8000\", openai_api_key=\"my-fake-key\")\n",
    "\n",
    "from nemoguardrails import LLMRails, RailsConfig\n",
    "\n",
    "config = RailsConfig.from_path(\"./config.yml\")\n",
    "app = LLMRails(config, llm=llm)\n",
    "\n",
    "new_message = app.generate(messages=[{\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"Hello! What can you do for me?\"\n",
    "}])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vz5n00qyuKjp"
   },
   "source": [
    "## Using with TogetherAI\n",
    "\n",
    "1. You can either set this in the server environment:\n",
    "`docker run -e PORT=8000 -e TOGETHERAI_API_KEY=<your-together-ai-api-key> -p 8000:8000 ghcr.io/berriai/litellm:latest`\n",
    "\n",
    "2. **Or** Pass this in as the api key `(...openai_api_key=\"<your-together-ai-api-key>\")`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XK1sk-McuhpE"
   },
   "outputs": [],
   "source": [
    "from langchain.chat_models import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(model_name=\"together_ai/togethercomputer/CodeLlama-13b-Instruct\", openai_api_base=\"http://0.0.0.0:8000\", openai_api_key=\"my-together-ai-api-key\")\n",
    "\n",
    "from nemoguardrails import LLMRails, RailsConfig\n",
    "\n",
    "config = RailsConfig.from_path(\"./config.yml\")\n",
    "app = LLMRails(config, llm=llm)\n",
    "\n",
    "new_message = app.generate(messages=[{\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"Hello! What can you do for me?\"\n",
    "}])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8A1KWKnzuxAS"
   },
   "source": [
    "### CONFIG.YML\n",
    "\n",
    "save this example `config.yml` in your current directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NKN1GmSvu0Cx"
   },
   "outputs": [],
   "source": [
    "# instructions:\n",
    "#   - type: general\n",
    "#     content: |\n",
    "#       Below is a conversation between a bot and a user about the recent job reports.\n",
    "#       The bot is factual and concise. If the bot does not know the answer to a\n",
    "#       question, it truthfully says it does not know.\n",
    "\n",
    "# sample_conversation: |\n",
    "#   user \"Hello there!\"\n",
    "#     express greeting\n",
    "#   bot express greeting\n",
    "#     \"Hello! How can I assist you today?\"\n",
    "#   user \"What can you do for me?\"\n",
    "#     ask about capabilities\n",
    "#   bot respond about capabilities\n",
    "#     \"I am an AI assistant that helps answer mathematical questions. My core mathematical skills are powered by wolfram alpha.\"\n",
    "#   user \"What's 2+2?\"\n",
    "#     ask math question\n",
    "#   bot responds to math question\n",
    "#     \"2+2 is equal to 4.\"\n",
    "\n",
    "# models:\n",
    "#   - type: main\n",
    "#     engine: openai\n",
    "#     model: claude-instant-1"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}