Qwen2.5 Technical Report
Paper
•
2412.15115
•
Published
•
325
zhanghang
hangzhang-nlp
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
5 days ago
Qwen2.5 Technical Report
liked
a dataset
about 2 months ago
BAAI/Infinity-Instruct
upvoted
a
paper
2 months ago
Breaking the Memory Barrier: Near Infinite Batch Size Scaling for
Contrastive Loss
Organizations
hangzhang-nlp's activity
upvoted
a
paper
5 days ago
liked
a
dataset
about 2 months ago
upvoted
2
papers
2 months ago
liked
a
Space
4 months ago
upvoted
a
paper
5 months ago
liked
a
model
5 months ago
liked
a
Space
5 months ago
reacted to
stas's
post with 👍
7 months ago
Post
If you're trying to run MoE Mixtral-8x7b under DeepSpeed w/ HF Transformers it's likely to hang on the first forward.
The solution is here https://github.com/microsoft/DeepSpeed/pull/4966?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US#issuecomment-1989671378
and you need deepspeed>=0.13.0
Thanks to Masahiro Tanaka for the fix.
The solution is here https://github.com/microsoft/DeepSpeed/pull/4966?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US#issuecomment-1989671378
and you need deepspeed>=0.13.0
Thanks to Masahiro Tanaka for the fix.
Where is the GSD dataset?
1
#6 opened 8 months ago
by
hangzhang-nlp
upvoted
an
article
8 months ago
Article
Introducing Idefics2: A Powerful 8B Vision-Language Model for the community
•
170
upvoted
a
collection
8 months ago
upvoted
a
paper
9 months ago
liked
a
dataset
9 months ago
authored
a
paper
about 1 year ago
liked
a
model
about 1 year ago
updated
a
Space
about 1 year ago
upvoted
2
papers
over 1 year ago
liked
a
model
over 1 year ago
