Update README.md

README.md

@@ -1,3 +1,69 @@
----
-license: mit
----
+---
+license: mit
+datasets:
+- kenhktsui/math-classifiers-data
+language:
+- en
+metrics:
+- f1
+pipeline_tag: text-classification
+library_name: fasttext
+---
+# maths-fasttext-classifier
+
+[Dataset](https://huggingface.co/datasets/kenhktsui/math-classifiers-data)
+This is part of my [fasttext classifier collection](https://huggingface.co/collections/kenhktsui/fasttext-model-for-pretraining-data-curation-67220374c8acb97a1839553c) for curating pretraining dataset.
+This classifier classifies a text into Maths or Others.   
+The model is trained over 1.6M records, which is a 50:50 mix of maths and non maths in website and achieved a test F1 score of 0.97. It is an intended upsampling of maths data.
+The classifier can be used for LLM pretraining data curation, to enhance capability in mathematics.
+It is ultra fast ⚡ with a throughtput of ~2000 doc/s with CPU.
+
+Don't underestimate the "old" fasttext classiifer! It is indeed a good and scalable practice. 
+For example, [QWEN2.5-MATH](https://arxiv.org/pdf/2409.12122) leverages fasttext to curate pretraining data, althought its classifier is not open sourced.
+
+
+## 🛠️Usage
+```python
+from typing import List
+import re
+from huggingface_hub import hf_hub_download
+import fasttext
+
+
+model_hf = fasttext.load_model(hf_hub_download("kenhktsui/maths-fasttext-classifier", "model.bin"))
+
+
+def replace_newlines(text: str) -> str:
+  return re.sub("\n+", " ", text)
+
+
+def predict(text_list: List[str]) -> List[dict]:
+  text_list = [replace_newlines(text) for text in text_list]
+  pred = model.predict(text_list)
+  return [{"label": l[0].lstrip("__label__"), "score": s[0]}
+           for l, s in zip(*pred)]
+
+
+predict([
+  """This is a lightning fast model, which can classify at throughtput of 2000 doc/s with CPU""",
+  """Differential geometry is a mathematical discipline that studies the geometry of smooth shapes and smooth spaces, otherwise known as smooth manifolds. It uses the techniques of single variable calculus, vector calculus, linear algebra and multilinear algebra.""",
+])
+# [{'label': 'Others', 'score': 0.99998367},
+# {'label': 'Maths', 'score': 0.99995637},
+```
+## 📊Evaluation
+full version
+```
+              precision    recall  f1-score   support
+
+       Maths       0.98      0.98      0.98    200000
+      Others       0.98      0.98      0.98    200000
+
+    accuracy                           0.98    400000
+   macro avg       0.98      0.98      0.98    400000
+weighted avg       0.98      0.98      0.98    400000
+```
+
+
+## ⚠️Known Limitation
+The classifier does not handle short text well, which might not be surprising.