antalvdb commited on
Commit
a0f85c8
·
verified ·
1 Parent(s): 38e6262

Update index.html

Browse files
Files changed (1) hide show
  1. index.html +14 -5
index.html CHANGED
@@ -100,8 +100,16 @@
100
  <h2 class="title is-3">WOPR in brief</h2>
101
  <div class="content has-text-justified">
102
  <p>
103
- WOPR, Word Predictor, is a memory-based language model developed in 2006-2011,
104
- and woken up from its cryogenic sleep in a more suitable era.
 
 
 
 
 
 
 
 
105
  </p>
106
  <p>
107
  A memory-based language model, in this case running on the
@@ -120,13 +128,14 @@
120
  and compressing it into a decision tree structure. This can be done on CPUs,
121
  with sufficient RAM. In short, its <b>ecological footprint is dramatically lower</b>;</li>
122
  <li>pretty efficient in generation when running with the fastest decision-tree
123
- approximations of <i>k</i>-NN classification. This can be done on CPUs as well.</li>
124
- <li>completely transparent in their functioning. There is also no question about
125
  the fact that <b>they memorize training data patterns</b>.</li>
126
  </ul>
127
  <p>On the downside,</p>
128
  <ul>
129
- <li>Not as great as current Transformer-based LLMs, but we have not trained
 
130
  beyond data set sizes with orders of magnitudes above 100 million words.
131
  Watch this space!</li>
132
  <li>Memory requirements during training are <b>heavy with large datasets</b>
 
100
  <h2 class="title is-3">WOPR in brief</h2>
101
  <div class="content has-text-justified">
102
  <p>
103
+ WOPR, Word Predictor, is a memory-based language model developed in 2006-2011.
104
+ It just woke up from its cryogenic sleep and is figuring out what is
105
+ all the fuss about LLMs.
106
+ </p>
107
+ <p>
108
+ WOPR is an ecologically friendly alternative LLM with a staggeringly simple
109
+ core. Everyone who took "Machine Learning 101" knows that the <i>k</i>-nearest
110
+ neighbor classifier is among the simplest yet most robust ML classifiers out
111
+ there, perhaps only beaten by the Naive Bayes classifier. So what happens if
112
+ you train a <i>k</i>-NN classifier to predict words? ...
113
  </p>
114
  <p>
115
  A memory-based language model, in this case running on the
 
128
  and compressing it into a decision tree structure. This can be done on CPUs,
129
  with sufficient RAM. In short, its <b>ecological footprint is dramatically lower</b>;</li>
130
  <li>pretty efficient in generation when running with the fastest decision-tree
131
+ approximations of <i>k</i>-NN classification. <b>This can be done on CPUs as well</b>;</li>
132
+ <li>completely transparent in their functioning. There can also be no doubt about
133
  the fact that <b>they memorize training data patterns</b>.</li>
134
  </ul>
135
  <p>On the downside,</p>
136
  <ul>
137
+ <li><b>Their performance is currently not as great as current Transformer-based LLMs</b>,
138
+ but we have not trained
139
  beyond data set sizes with orders of magnitudes above 100 million words.
140
  Watch this space!</li>
141
  <li>Memory requirements during training are <b>heavy with large datasets</b>