Hugging Face's logo

QuietImpostor
/
Rasphi-MoE-Instruct-Unfinetuned

Text Generation
Transformers
Safetensors
rasphi
conversational
custom_code
Model card Files Community
Rasphi-MoE-Instruct-Unfinetuned / README.md
QuietImpostor's picture
QuietImpostor
Remove erroneous pipeline tag
2fb9864 verified
preview code
|
raw
history blame
759 Bytes
metadata
license: mit
base_model:
  - microsoft/Phi-3.5-MoE-instruct
library_name: transformers

Model Card for Rasphi

Rasphi (pronounced rasp-fee, though the name may be changed.) is a WIP architecture derrived from Microsoft's Phi 3.5 MoE / GRIN model. It aims to improve reasoning by having a dedicated reasoning stream in which half of all experts are allocated to it. Due to the experts being split directly in half, there is a high chance of instability / overall incoherence in both streams.

Model Details

Uses

Rasphi can be used for research purposes and or finetuning to gauge the performance of the new architecture. However, it is highly not recommended in its current state to be used for any user-facing applications. Or at all.