import gradio as gr from blindbox.requests import SecureSession DEMO_SERVER = "4.208.9.167:80" def run_query( server, policy, prompt): if prompt == None or server == None or policy == None: return("⛔ Error: please select an option for stages 1-3") if len(prompt) == 0 or len(policy) == 0 or len(server) == 0: return("⛔ Error: please select an option for stages 1-3") if server != "Authentic confidential VM server": return ("⛔ Error: you can only connect to an application running on a Confidential VM") if policy == "Expected Santacoder app policy file": POLICY = "./cce_policy.txt" elif policy == "Unexpected Hello World app policy file": POLICY = "./hello_world.txt" else: POLICY = "fake.txt" try: with SecureSession(f"http://{DEMO_SERVER}", POLICY) as secure_session: res = secure_session.post(endpoint="/generate", json={"input_text": prompt}) cleaned = res.text.replace('\\n', '\n').split('\n\n')[0].split(':"')[1] return("✅ Query successful\n" + cleaned) except Exception as err: return(f"⛔ Query failed!\n{err}") with gr.Blocks(theme=gr.themes.Soft()) as demo: gr.Markdown("

🔒Confidential code generation with BlindBox and Santacoder

") gr.Markdown("

This is the demo for our article on deploying code generation LLM models with BlindBox: AI-assisted code generation with privacy guarantees: Securely deploy SantaCoder with BlindBox
You can view the article here!

") gr.Markdown("

You can use this demo to send a function definition to BigCode's open-source Santacoder model and get back an auto-completed function.

") gr.Markdown("

The model is deployed within a highly-isolated Trusted Execution Environment, meaning that we, as the service provider, have no access to the data sent to this model!

") gr.Markdown(">

Step 1: Check we are connecting to an authentic confidential VM") gr.Markdown("

This first option allows you to choose whether to connect to the Whisper application deployed with BlindBox on a confidential VM or the same application deployed on a dummy server which is not within a confidential VM!
This demonstrates how BlindBox blocks requests to non-authentic confidential VMs!

") with gr.Column(): server = gr.Radio( ["Authentic confidential VM server", "Unauthentic dummy server"], label="Select the server you want to connect to" ) gr.Markdown(">

Step 2: Check we are connecting to the latest official SaaS application image

") gr.Markdown("

You can think of this second verification a bit like a checksum. Here, you can see what happens if the end user queries the application whilst providing the latest policy file for our Santacoder application, the latest policy file for a different Hello World application or sends no policy file at all.

") with gr.Column(): policy = gr.Radio( ["Expected Santacoder app policy file", "Unexpected Hello World app policy file", "No policy file"], label="Select your CCE policy file" ) gr.Markdown(">

Step 3: Select your prompt

") gr.Markdown("

Select between the following prompt examples we provide.

") with gr.Column(): prompt = gr.Radio( ["def sum(x, y):", "def print_name(name):", "def hello_world():", "def square_root(nbr):"], label="Select your user prompt" ) gr.Markdown(">

Query the Santacoder model

") with gr.Column(): trigger = gr.Button("Test query") with gr.Column(): output = gr.Textbox(placeholder="Output", label="See the output of your query here") trigger.click(fn=run_query, inputs=[server, policy, prompt], outputs=output) if __name__ == "__main__": demo.launch()