aksell commited on
Commit
cdcf3af
·
1 Parent(s): b65afcb

Update documentation

Browse files
Files changed (1) hide show
  1. hexviz/pages/2_📄Documentation.py +19 -11
hexviz/pages/2_📄Documentation.py CHANGED
@@ -25,19 +25,27 @@ tool for interpreting transformer model internals see fex ([Abnar et al.
25
  2020](https://arxiv.org/abs/2005.00928v2)). [BERTology meets
26
  biology](https://arxiv.org/abs/2006.15222) provides a thorough introduction to
27
  how we can analyze Transformer protein models through the lens of attention,
28
- they show exciting findings such as: > Attention: (1) captures the folding
29
- structure of proteins, connecting amino acids that are far apart in the
30
- underlying sequence, but spatially close in the three-dimensional structure, (2)
31
- targets binding sites, a key functional component of proteins, and (3) focuses
32
- on progressively more complex biophysical properties with increasing layer depth
 
33
 
34
  Most existing tools for analyzing and visualizing attention patterns focus on
35
- models trained on text. It can be hard to analyze protein sequences using these
36
- tools as sequences can be long and we lack intuition about how the language of
37
- proteins work. BERTology meets biology shows visualizing attention patterns in
38
- the context of protein structure can facilitate novel discoveries about what
39
- models learn. [**Hexviz**](https://huggingface.co/spaces/aksell/hexviz) is a
40
- tool to simplify analyzing attention patterns in the context of protein
 
 
 
 
 
 
 
41
  structure. We hope this can enable domain experts to explore and interpret the
42
  knowledge contained in pLMs.
43
 
 
25
  2020](https://arxiv.org/abs/2005.00928v2)). [BERTology meets
26
  biology](https://arxiv.org/abs/2006.15222) provides a thorough introduction to
27
  how we can analyze Transformer protein models through the lens of attention,
28
+ they show exciting findings such as:
29
+ > Attention: (1) captures the folding
30
+ > structure of proteins, connecting amino acids that are far apart in the
31
+ > underlying sequence, but spatially close in the three-dimensional structure, (2)
32
+ > targets binding sites, a key functional component of proteins, and (3) focuses
33
+ > on progressively more complex biophysical properties with increasing layer depth
34
 
35
  Most existing tools for analyzing and visualizing attention patterns focus on
36
+ models trained on text ([BertViz](https://github.com/jessevig/bertviz),
37
+ [exBERT], [exBERT](https://exbert.net/)). It can be hard to analyze protein
38
+ sequences using these tools as we don't have any intuitive understand about the
39
+ protein language when looking at an amino acid sequence in the same way we do
40
+ for natural language. Experts studying proteins do have an understanding of
41
+ proteins, but it is mostly in in the context of a protein's structure, not its
42
+ sequence. Can we build a tool for analyzing attention patterns that can leverage
43
+ expert's knowledge of protein structure to understand what pLMs learn?
44
+
45
+ BERTology meets biology shows how visualizing attention patterns in the context
46
+ of protein structure can facilitate novel discoveries about what models learn.
47
+ [**Hexviz**](https://huggingface.co/spaces/aksell/hexviz) builds on this, and is
48
+ a tool to simplify analyzing attention patterns in the context of protein
49
  structure. We hope this can enable domain experts to explore and interpret the
50
  knowledge contained in pLMs.
51