Na0s/Qwen1.5-MoE-A2.7B-Chat-20_experts_Maths_FT_1k_cosine Text Generation • 6B • Updated Dec 19, 2024 • 2
Na0s/Qwen1.5-MoE-A2.7B-Chat-20_experts-L2Norm-Pruning Text Generation • 6B • Updated Dec 18, 2024 • 3
Na0s/sft-ready-Text-Generation-Augmented-Data-Alpaca-Format Viewer • Updated Dec 13, 2024 • 7.67M • 53 • 2
Pruned MoEs (Mixtral-8x7B-Instruct-v0.1) Collection Pruned experts from Mixtral-8x7B-Instruct-v0.1 with respect to the paper "A Provably Effective Method for Pruning Experts in Fine-tuned Sparse MoEs" • 15 items • Updated Nov 18, 2024