dar-tau commited on
Commit
f833d09
Β·
verified Β·
1 Parent(s): 17d4734

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +9 -5
app.py CHANGED
@@ -161,11 +161,15 @@ with gr.Blocks(theme=gr.themes.Default(), css=css) as demo:
161
  '''This idea was explored in the paper **Patchscopes** ([Ghandeharioun et al., 2024](https://arxiv.org/abs/2401.06102)) and was later investigated further in **SelfIE** ([Chen et al., 2024](https://arxiv.org/abs/2403.10949)).
162
  An honorary mention of **Speaking Probes** ([Dar, 2023](https://towardsdatascience.com/speaking-probes-self-interpreting-models-7a3dc6cb33d6) -- my own work!! πŸ₯³) which was less mature but had the same idea in mind.
163
  We will follow the SelfIE implementation in this space for concreteness. Patchscopes are so general that they encompass many other interpretation techniques too!!!
164
-
165
- πŸ‘Ύ **The idea is really simple: models are able to understand their own hidden states by nature!** πŸ‘Ύ
166
- If I give a model a prompt of the form ``User: [X] Assistant: Sure'll I'll repeat your message`` and replace ``[X]`` *during computation* with the hidden state we want to understand,
167
- we hope to get back a summary of the information that exists inside the hidden state, because it is encoded in a latent space the model uses itself!! How cool is that! 😯😯😯
168
- ''', line_breaks=True)
 
 
 
 
169
  with gr.Column(scale=1):
170
  gr.Markdown('<span style="font-size:180px;">πŸ€”</span>')
171
 
 
161
  '''This idea was explored in the paper **Patchscopes** ([Ghandeharioun et al., 2024](https://arxiv.org/abs/2401.06102)) and was later investigated further in **SelfIE** ([Chen et al., 2024](https://arxiv.org/abs/2403.10949)).
162
  An honorary mention of **Speaking Probes** ([Dar, 2023](https://towardsdatascience.com/speaking-probes-self-interpreting-models-7a3dc6cb33d6) -- my own work!! πŸ₯³) which was less mature but had the same idea in mind.
163
  We will follow the SelfIE implementation in this space for concreteness. Patchscopes are so general that they encompass many other interpretation techniques too!!!
164
+ ''', line_breaks=True)
165
+
166
+ with gr.Accordion(label='πŸ‘Ύ The idea is really simple: models are able to understand their own hidden states by nature! πŸ‘Ύ',
167
+ elem_classes=['explanation_accordion']):
168
+ gr.Markdown(
169
+ '''If I give a model a prompt of the form ``User: [X] Assistant: Sure'll I'll repeat your message`` and replace ``[X]`` *during computation* with the hidden state we want to understand,
170
+ we hope to get back a summary of the information that exists inside the hidden state, because it is encoded in a latent space the model uses itself!! How cool is that! 😯😯😯
171
+ ''', line_breaks=True)
172
+
173
  with gr.Column(scale=1):
174
  gr.Markdown('<span style="font-size:180px;">πŸ€”</span>')
175