import gradio as gr from blindbox.requests import SecureSession DEMO_SERVER = "4.208.9.167:80" def run_query( server, policy, prompt): if prompt == None or server == None or policy == None: return("⛔ Error: please select an option for stages 1-3") if len(prompt) == 0 or len(policy) == 0 or len(server) == 0: return("⛔ Error: please select an option for stages 1-3") if server != "Authentic confidential VM server": return ("⛔ Error: you can only connect to an application running on a Confidential VM") if policy == "Expected Santacoder app policy file": POLICY = "./cce_policy.txt" elif policy == "Unexpected Hello World app policy file": POLICY = "./hello_world.txt" else: POLICY = "fake.txt" try: with SecureSession(f"http://{DEMO_SERVER}", POLICY) as secure_session: res = secure_session.post(endpoint="/generate", json={"input_text": prompt}) cleaned = res.text.replace('\\n', '\n').split('\n\n')[0].split(':"')[1] return("✅ Query successful\n" + cleaned) except Exception as err: return(f"⛔ Query failed!\n{err}") with gr.Blocks(theme=gr.themes.Soft()) as demo: gr.Markdown("

🔒Confidential code generation with BlindBox and Santacoder

") gr.Markdown("

This is the demo for our article on deploying code generation LLM models with BlindBox: AI-assisted code generation with privacy guarantees: Securely deploy SantaCoder with BlindBox
You can view the article here!

") gr.Markdown("

You can use this demo to send a function definition to BigCode's open-source Santacoder model and get back an auto-completed function.

") gr.Markdown("

The model is deployed within a highly-isolated Trusted Execution Environment, meaning that we, as the service provider, have no access to the data sent to this model!

") gr.Markdown("

You can see how we deployed the model by checking out the integration section of our documentation!

") gr.Markdown(">

Step 1: Check we are connecting to an authentic confidential VM") gr.Markdown("

This first option allows you to choose whether to connect to the Whisper application deployed with BlindBox on a confidential VM or the same application deployed on a dummy server which is not within a confidential VM!
This demonstrates how BlindBox blocks requests to non-authentic confidential VMs!

") with gr.Column(): server = gr.Radio( ["Authentic confidential VM server", "Unauthentic dummy server"], label="Select the server you want to connect to" ) gr.Markdown(">

Step 2: Check we are connecting to the latest official SaaS application image

") gr.Markdown("

You can think of this second verification a bit like a checksum. Here, you can see what happens if the end user queries the application whilst providing the latest policy file for our Santacoder application, the latest policy file for a different Hello World application or sends no policy file at all.

") with gr.Column(): policy = gr.Radio( ["Expected Santacoder app policy file", "Unexpected Hello World app policy file", "No policy file"], label="Select your CCE policy file" ) gr.Markdown(">

Step 3: Select your prompt

") gr.Markdown("

Select between the following prompt examples we provide.

") with gr.Column(): prompt = gr.Radio( ["def sum(x, y):", "def print_name(name):", "def hello_world():", "def square_root(nbr):"], label="Select your user prompt" ) gr.Markdown(">

Query the Santacoder model

") with gr.Column(): trigger = gr.Button("Test query") with gr.Column(): output = gr.Textbox(placeholder="Output", label="See the output of your query here") trigger.click(fn=run_query, inputs=[server, policy, prompt], outputs=output) if __name__ == "__main__": demo.launch()