Hugging Face's logo

m3rg-iitd
/
llamat-3-cif

Safetensors
English
llama
crystal structure generation
CIF understanding
Model card Files Community
llamat-3-cif / README.md
m3rg-iitd's picture
m3rg-iitd
Update README.md
271db01 verified
preview code
|
raw
history blame contribute delete
1.86 kB
metadata
license: llama3
language:
  - en
base_model:
  - m3rg-iitd/llamat-3
tags:
  - crystal structure generation
  - CIF understanding

Model Card for LLaMat-3-CIF

LLaMat-3-CIF is a specialized large language model designed to generate and extract information from Crystallographic Information Files.

The model is developed after continued pretraining of LLaMat-3 on 7M instruction-output pairs obtained using CIFs from Materials Project, Google GNoME, and AMCSD

Overview

  • Model Type: Large Language Model (LLM)
  • Base Model: LLaMat-3 (continued pretraining of LLaMat-3 on CIFs)
  • Language: English
  • License: LLaMA-3 License
  • Tags: Material Science, Domain Adaptation, Crystal Structure Generation

Model Details

Key Features

  • Instruction Following Abilities: Answers questions based on CIF files.
  • Applications: Crystal structure generation

Development and Support

  • Developed by: M3RG, IIT Delhi & DAIR, IIT Delhi
  • Compute Support:
    • Edinburgh International Data Facility (EIDF): Provided access to Cerebras CS2 clusters for pretraining.
    • IIT Delhi High-Performance Computing Cluster: Supported fine-tuning and inference stages.

Technical Specifications

Hardware Infrastructure

  • Pretraining: 2 Cerebras CS-2 Wafer-Scale Engines (WSE-2)
  • Finetuning: 2 Cerebras CS-2 Wafer-Scale Engines (WSE-2)
  • Inferencing: 1 NVIDIA A100 80GB GPU

Software Stack

  • Frameworks: PyTorch, Hugging Face Transformers

Model Sources