Update app.py
Browse files
app.py
CHANGED
@@ -161,11 +161,15 @@ with gr.Blocks(theme=gr.themes.Default(), css=css) as demo:
|
|
161 |
'''This idea was explored in the paper **Patchscopes** ([Ghandeharioun et al., 2024](https://arxiv.org/abs/2401.06102)) and was later investigated further in **SelfIE** ([Chen et al., 2024](https://arxiv.org/abs/2403.10949)).
|
162 |
An honorary mention of **Speaking Probes** ([Dar, 2023](https://towardsdatascience.com/speaking-probes-self-interpreting-models-7a3dc6cb33d6) -- my own work!! π₯³) which was less mature but had the same idea in mind.
|
163 |
We will follow the SelfIE implementation in this space for concreteness. Patchscopes are so general that they encompass many other interpretation techniques too!!!
|
164 |
-
|
165 |
-
|
166 |
-
|
167 |
-
|
168 |
-
|
|
|
|
|
|
|
|
|
169 |
with gr.Column(scale=1):
|
170 |
gr.Markdown('<span style="font-size:180px;">π€</span>')
|
171 |
|
|
|
161 |
'''This idea was explored in the paper **Patchscopes** ([Ghandeharioun et al., 2024](https://arxiv.org/abs/2401.06102)) and was later investigated further in **SelfIE** ([Chen et al., 2024](https://arxiv.org/abs/2403.10949)).
|
162 |
An honorary mention of **Speaking Probes** ([Dar, 2023](https://towardsdatascience.com/speaking-probes-self-interpreting-models-7a3dc6cb33d6) -- my own work!! π₯³) which was less mature but had the same idea in mind.
|
163 |
We will follow the SelfIE implementation in this space for concreteness. Patchscopes are so general that they encompass many other interpretation techniques too!!!
|
164 |
+
''', line_breaks=True)
|
165 |
+
|
166 |
+
with gr.Accordion(label='πΎ The idea is really simple: models are able to understand their own hidden states by nature! πΎ',
|
167 |
+
elem_classes=['explanation_accordion']):
|
168 |
+
gr.Markdown(
|
169 |
+
'''If I give a model a prompt of the form ``User: [X] Assistant: Sure'll I'll repeat your message`` and replace ``[X]`` *during computation* with the hidden state we want to understand,
|
170 |
+
we hope to get back a summary of the information that exists inside the hidden state, because it is encoded in a latent space the model uses itself!! How cool is that! π―π―π―
|
171 |
+
''', line_breaks=True)
|
172 |
+
|
173 |
with gr.Column(scale=1):
|
174 |
gr.Markdown('<span style="font-size:180px;">π€</span>')
|
175 |
|