Update README.md

README.md

@@ -3,8 +3,176 @@ title: README
 emoji: 📈
 colorFrom: indigo
 colorTo: gray
-sdk: streamlit
+sdk: static
 pinned: false
 ---
 
-Edit this `README.md` markdown file to author your organization card.
+
+## Who We Are
+
+AstroMLab is a dynamic group of *astrophysicists* and *computer scientists* passionate about pushing the boundaries of **Large Language Models (LLMs)in astronomy**. Our team includes:
+
+- *Leading astronomers, astrophysicists, and cosmologists*
+- *Top natural language processing experts* from Oak Ridge National Laboratory and Argonne National Laboratory
+- *Frontier arXivists* from the NASA Astrophysics Data System
+- *Enthusiastic young researchers* bridging the gap between astronomy and LLMs
+
+While LLMs are advancing rapidly, we believe that real progress in *AI-driven astronomical research* requires *deep domain knowledge*. This conviction drives us to tackle the challenges in applying LLMs to astronomy head-on.
+
+## Our Goals
+
+Our ultimate aim is to:
+
+1. Develop specialized LLMs for astronomy
+2. Create **reliable, light-weight, and open-source models** adaptable for advanced research agents
+3. **Expedite scientific discovery** through LLM-driven end-to-end research
+4. Push the boundaries of what's possible in astronomical research
+
+## Our Achievements
+
+Despite being a young group, we've made significant strides:
+
+- Curated the **first extensive astronomy-based benchmarking dataset** using high-quality review articles ([Ting et al. 2024](https://arxiv.org/abs/2407.11194))
+- Explored training of specialized astronomy LLMs
+- Released three model sets:
+  - **AstroSage-8B** (coming soon, de Haan et al. 2024)
+  - **AstroLLaMA-2-70B** ([Pan et al. 2024](https://arxiv.org/abs/2407.11194))
+  - **AstroLLaMA-3-8B** ([Pan et al. 2024](https://arxiv.org/abs/2407.11194))
+  - AstroLLaMA-2-7B ([Perkowski et al. 2024](https://arxiv.org/abs/2401.01916), [Nguyen et al. 2023](https://arxiv.org/abs/2309.06126), developed during our time at *UniverseTBD*)
+
+Our flagship model, AstroSage-8B, demonstrates remarkable performance when compared to other models in the 7B class. It achieves a substantial lead of 3.5 percentage points over its closest competitor, which translates to an estimated **10-fold reduction** in computational costs (see the [AstroBench page](benchmarking.html) for details).
+
+| Model | Score (%) |
+|-------|-----------|
+| **<span style="color: #3366cc;">AstroSage-8B (AstroMLab)</span>** | **<span style="color: #3366cc;">77.2</span>** |
+| LLaMA-3.1-8B | 73.7 |
+| **<span style="color: #3366cc;">AstroLLaMA-2-70B (AstroMLab)</span>** | **<span style="color: #3366cc;">72.3</span>** |
+| Gemma-2-9B | 71.5 |
+| Qwen-2.5-7B | 70.4 |
+| Yi-1.5-9B | 68.4 |
+| InternLM-2.5-7B | 64.0 |
+| Mistral-7B-v0.3 | 63.9 |
+| ChatGLM3-6B | 50.4 |
+| AstroLLaMA-2-7B (UniverseTBD) | 44.3 |
+
+![Cost and performance trade-off in astronomical Q&A](figures/AstroBench.png)
+
+The exceptional performance of AstroSage-8B showcases the potential for more efficient and cost-effective agentic research in astronomy. This advancement opens up new possibilities for widespread application of AI in astronomical research, making sophisticated analysis more accessible to a broader range of institutions and researchers.
+
+
+## Open Source Commitment
+
+We are fully committed to open source:
+
+- All our models are released on **Hugging Face**
+- Find our models here: [AstroMLab on Hugging Face](https://huggingface.co/AstroMLab)
+
+
+## Our Support and Vision
+
+We are grateful for our supporters:
+
+- Access to the Frontier nodes at Oak Ridge Leadership Computing Facility
+- Backing from Microsoft's Accelerating Foundation Models Research (AFMR)
+
+
+## Join Us
+
+Our team is expanding, and we'd love to hear from you! 
+
+- Contact us: [[email protected]](mailto:[email protected])
+
+<br>
+
+---
+
+## Team
+
+<table>
+  <tr>
+    <td align="center" width="25%"><img src="figures/Members_Yuan-Sen_Ting.png" alt="Yuan-Sen Ting"></td>
+    <td align="center" width="25%"><img src="figures/Members_Tirthankar_Ghosal.png" alt="Tirthankar Ghosal"></td>
+    <td align="center" width="25%"><img src="figures/Members_Tijmen_de_Haan.png" alt="Tijmen de Haan"></td>
+    <td align="center" width="25%"><img src="figures/Members_Josh_Nguyen.png" alt="Josh Nguyen"></td>
+  </tr>
+  <tr>
+    <td align="center"><strong>Yuan-Sen Ting</strong><br>The Ohio State University</td>
+    <td align="center"><strong>Tirthankar Ghosal</strong><br>Oak Ridge National Laboratory</td>
+    <td align="center"><strong>Tijmen de Haan</strong><br>KEK</td>
+    <td align="center"><strong>Josh Nguyen</strong><br>University of Pennsylvania</td>
+  </tr>
+  <tr>
+    <td align="center"><img src="figures/Members_Rui_Pan.png" alt="Rui Pan"></td>
+    <td align="center"><img src="figures/Members_Hardik_Arora.png" alt="Hardik Arora"></td>
+    <td align="center"><img src="figures/Members_Emily_Herron.png" alt="Emily Herron"></td>
+    <td align="center"><img src="figures/Members_Yuwei_Yang.png" alt="Yuwei Yang"></td>
+  </tr>
+  <tr>
+    <td align="center"><strong>Rui Pan</strong><br>University of Illinois Urbana-Champaign</td>
+    <td align="center"><strong>Hardik Arora</strong><br>Indian Institutes of Technology</td>
+    <td align="center"><strong>Emily Herron</strong><br>Oak Ridge National Laboratory</td>
+    <td align="center"><strong>Yuwei Yang</strong><br>Australian National University</td>
+  </tr>
+  <tr>
+    <td align="center"><img src="figures/Members_Zechang_Sun.png" alt="Alberto Accomazzi"></td>
+    <td align="center"><img src="figures/Members_Alberto_Accomazzi.png" alt="Alberto Accomazzi"></td>
+    <td align="center"><img src="figures/Members_Argonne.png" alt="Azton Wells"></td>
+    <td align="center"><img src="figures/Members_Nesar_Ramachandra.png" alt="Nesar Ramachandra"></td>
+    <td align="center"><img src="figures/Members_Sandeep_Madireddy.png" alt="Sandeep Madireddy"></td>
+  </tr>
+  <tr>
+    <td align="center"><strong>Zechang Sun</strong><br>Tsinghua University</td>
+    <td align="center"><strong>Alberto Accomazzi</strong><br>NASA Astrophysics Data System</td>
+    <td align="center"><strong>Azton Wells</strong><br>Argonne National Laboratory</td>
+    <td align="center"><strong>Nesar Ramachandra</strong><br>Argonne National Laboratory</td>
+  </tr>
+  <tr>
+    <td align="center"><img src="figures/Members_Sandeep_Madireddy.png" alt="Sandeep Madireddy"></td>
+  </tr>
+  <tr>
+    <td align="center"><strong>Sandeep Madireddy</strong><br>Argonne National Laboratory</td>
+  </tr>
+</table>
+
+<br>
+
+---
+
+## Publications
+
+### AstroMLab 1: Who Wins Astronomy Jeopardy!?
+
+**[Yuan-Sen Ting, et al., 2024, arXiv:2407.11194](https://arxiv.org/abs/2407.11194)**
+
+We present a comprehensive evaluation of proprietary and open-weights large language models using the first astronomy-specific benchmarking dataset. This dataset comprises 4,425 multiple-choice questions curated from the Annual Review of Astronomy and Astrophysics, covering a broad range of astrophysical topics.
+
+Key findings:
+- Claude-3.5-Sonnet outperforms competitors, achieving 85.0% accuracy.
+- Open-weights models like LLaMA-3-70b (80.6%) and Qwen-2-72b (77.7%) now compete with some of the best proprietary models.
+- We identify performance variations across astronomical subfields, with challenges in exoplanet-related fields, stellar astrophysics, and instrumentation.
+- Top-performing models demonstrate well-calibrated confidence, with correlations above 0.9 between confidence and correctness.
+- The rapid progress suggests that LLM-driven research in astronomy may become feasible in the near future.
+
+
+<br> 
+
+### AstroMLab 2: AstroLLaMA-2-70B Model and Benchmarking Specialised LLMs for Astronomy
+
+**[Rui Pan, Josh Nguyen, et al., 2024](https://arxiv.org/abs/2407.11194)**
+
+We introduce new models: AstroLLaMA-3-8B and AstroLLaMA-2-70B, building upon the previous AstroLLaMA series and quantitatively assess specialized LLMs in astronomy, leveraging recently curated high-quality astronomical MCQs. 
+
+Key points:
+- Previously released AstroLLaMA series (based on LLaMA-2-7B) underperforms compared to the native LLaMA model.
+- Performance degradation can be partially mitigated by using high-quality data for continual pretraining.
+- Continual pretraining on the 70B model can yield improvements, despite observed catastrophic forgetting in smaller models.
+
+<br> 
+
+### Legacy Output: The AstroLLaMA Series
+
+1. **[Josh Nguyen, et al., 2023, arXiv:2309.06126](https://arxiv.org/abs/2309.06126)**
+2. **[Ernest Perkowski, Rui Pan, et al., 2024, arXiv:2401.01916](https://arxiv.org/abs/2401.01916)**
+
+The first open-source conversational AI tool tailored for the astronomy community -- AstroLLaMA-2-7B and AstroLLaMA-2-7B-Chat.
+