Balancing Speed and Stability: The Trade-offs of FP8 vs. BF16 Training in LLMs
Paper
•
2411.08719
•
Published
Open-ψ (Open-Sci) Collective
non-profit
AI & ML interests
Researching and building foundation models with improved generalization and reasoning. LAION & friends spin-off for open-sourcing foundation models with strong generalization and reasoning , including datasets necessary for their creation, to serve as common open, reproducible grounds for further research experiments.
Recent Activity
open-sci's activity
Taishi-N324
authored
3
papers
7 days ago
JJitsev
authored
a
paper
14 days ago
Taishi-N324
authored
a
paper
15 days ago
Taishi-N324
authored
a
paper
5 months ago
JJitsev
authored
8
papers
5 months ago
A Comparative Study on Generative Models for High Resolution Solar Observation Imaging
Paper
•
2304.07169
•
Published
DataComp: In search of the next generation of multimodal datasets
Paper
•
2304.14108
•
Published
•
2
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs
Paper
•
2111.02114
•
Published
LAION-5B: An open large-scale dataset for training next generation image-text models
Paper
•
2210.08402
•
Published
•
4
Reproducible scaling laws for contrastive language-image learning
Paper
•
2212.07143
•
Published
•
1
Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models
Paper
•
2406.02061
•
Published
•
1
DataComp-LM: In search of the next generation of training sets for language models
Paper
•
2406.11794
•
Published
•
50
Resolving Discrepancies in Compute-Optimal Scaling of Language Models
Paper
•
2406.19146
•
Published
Taishi-N324
authored
3
papers
8 months ago
Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities
Paper
•
2404.17790
•
Published
•
5
Building a Large Japanese Web Corpus for Large Language Models
Paper
•
2404.17733
•
Published
•
4
LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
Paper
•
2407.03963
•
Published
•
19
TomerPorian
authored
a
paper
8 months ago
Taishi-N324
authored
a
paper
11 months ago
JJitsev
authored
a
paper
about 1 year ago