Update README.md

README.md

@@ -15,10 +15,23 @@ base_model:
 
 # PhiMiX-2x2B
 
-PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
+
+## Code is work in progress
+
+<img src="https://static.wixstatic.com/media/3eee0b_4d2513facc0f4a1cb21b2b8f4e1df179~mv2.jpg/v1/fill/w_600,h_400,al_c,q_80,usm_0.66_1.00_0.01,enc_auto/3eee0b_4d2513facc0f4a1cb21b2b8f4e1df179~mv2.jpg" />
+
+PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using mergekit:
 * [cognitivecomputations/dolphin-2_6-phi-2](https://huggingface.co/cognitivecomputations/dolphin-2_6-phi-2)
 * [rhysjones/phi-2-orange](https://huggingface.co/rhysjones/phi-2-orange)
 
+
+## ©️ Credits
+* [mlabonne's phixtral](https://huggingface.co/mlabonne/phixtral-4x2_8) for the PhiConfig and inference code.
+* [mergekit](https://github.com/cg123/mergekit) code which I tweaked (you can find the PhiConfig [here](https://github.com/cg123/mergekit/blob/508348ae34be17ea0a95d0a288a6e34491a2558a/mergekit/architecture.py#L289))
+by mainly adding the config in the `moe_mixtral.py` script from `mixtral` branch.
+
+
+
 ## 🧩 Configuration
 
 ```yaml