Update index.html
Browse files- index.html +14 -5
index.html
CHANGED
@@ -100,8 +100,16 @@
|
|
100 |
<h2 class="title is-3">WOPR in brief</h2>
|
101 |
<div class="content has-text-justified">
|
102 |
<p>
|
103 |
-
WOPR, Word Predictor, is a memory-based language model developed in 2006-2011
|
104 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
105 |
</p>
|
106 |
<p>
|
107 |
A memory-based language model, in this case running on the
|
@@ -120,13 +128,14 @@
|
|
120 |
and compressing it into a decision tree structure. This can be done on CPUs,
|
121 |
with sufficient RAM. In short, its <b>ecological footprint is dramatically lower</b>;</li>
|
122 |
<li>pretty efficient in generation when running with the fastest decision-tree
|
123 |
-
approximations of <i>k</i>-NN classification. This can be done on CPUs as well
|
124 |
-
<li>completely transparent in their functioning. There
|
125 |
the fact that <b>they memorize training data patterns</b>.</li>
|
126 |
</ul>
|
127 |
<p>On the downside,</p>
|
128 |
<ul>
|
129 |
-
<li>
|
|
|
130 |
beyond data set sizes with orders of magnitudes above 100 million words.
|
131 |
Watch this space!</li>
|
132 |
<li>Memory requirements during training are <b>heavy with large datasets</b>
|
|
|
100 |
<h2 class="title is-3">WOPR in brief</h2>
|
101 |
<div class="content has-text-justified">
|
102 |
<p>
|
103 |
+
WOPR, Word Predictor, is a memory-based language model developed in 2006-2011.
|
104 |
+
It just woke up from its cryogenic sleep and is figuring out what is
|
105 |
+
all the fuss about LLMs.
|
106 |
+
</p>
|
107 |
+
<p>
|
108 |
+
WOPR is an ecologically friendly alternative LLM with a staggeringly simple
|
109 |
+
core. Everyone who took "Machine Learning 101" knows that the <i>k</i>-nearest
|
110 |
+
neighbor classifier is among the simplest yet most robust ML classifiers out
|
111 |
+
there, perhaps only beaten by the Naive Bayes classifier. So what happens if
|
112 |
+
you train a <i>k</i>-NN classifier to predict words? ...
|
113 |
</p>
|
114 |
<p>
|
115 |
A memory-based language model, in this case running on the
|
|
|
128 |
and compressing it into a decision tree structure. This can be done on CPUs,
|
129 |
with sufficient RAM. In short, its <b>ecological footprint is dramatically lower</b>;</li>
|
130 |
<li>pretty efficient in generation when running with the fastest decision-tree
|
131 |
+
approximations of <i>k</i>-NN classification. <b>This can be done on CPUs as well</b>;</li>
|
132 |
+
<li>completely transparent in their functioning. There can also be no doubt about
|
133 |
the fact that <b>they memorize training data patterns</b>.</li>
|
134 |
</ul>
|
135 |
<p>On the downside,</p>
|
136 |
<ul>
|
137 |
+
<li><b>Their performance is currently not as great as current Transformer-based LLMs</b>,
|
138 |
+
but we have not trained
|
139 |
beyond data set sizes with orders of magnitudes above 100 million words.
|
140 |
Watch this space!</li>
|
141 |
<li>Memory requirements during training are <b>heavy with large datasets</b>
|