Model Card for LLaMat-3-CIF

LLaMat-3-CIF is a specialized large language model designed to generate and extract information from Crystallographic Information Files.

The model is developed after continued pretraining of LLaMat-3 on 7M instruction-output pairs obtained using CIFs from Materials Project, Google GNoME, and AMCSD

---

Overview

• Model Type: Large Language Model (LLM)
• Base Model: LLaMat-3 (continued pretraining of LLaMat-3 on CIFs)
• Language: English
• License: LLaMA-3 License
• Tags: Material Science, Domain Adaptation, Crystal Structure Generation

---

Model Details

Key Features

• Instruction Following Abilities: Answers questions based on CIF files.
• Applications: Crystal structure generation

Development and Support

• Developed by: M3RG, IIT Delhi & DAIR, IIT Delhi
• Compute Support:
  • Edinburgh International Data Facility (EIDF): Provided access to Cerebras CS2 clusters for pretraining.
  • IIT Delhi High-Performance Computing Cluster: Supported fine-tuning and inference stages.

---

Technical Specifications

Hardware Infrastructure

• Pretraining: 2 Cerebras CS-2 Wafer-Scale Engines (WSE-2)
• Finetuning: 2 Cerebras CS-2 Wafer-Scale Engines (WSE-2)
• Inferencing: 1 NVIDIA A100 80GB GPU

Software Stack

• Frameworks: PyTorch, Hugging Face Transformers

---

Model Sources

• Repository: LLaMat on GitHub
• Compute Resources: EIDF Cerebras CS Clusters