---
license: cc-by-nc-nd-3.0
---

# VyLinh-Lite: Vietnamese 3B Reasoning Language Model

## Model Details

- **Language(s)**: Vietnamese
- **Base Model**: Qwen2.5-3B
- **Model Size**: 3 billion parameters

## Intended Use

- **Primary intended uses**: Vietnamese language understanding, reasoning, and generation
- **Primary intended users**: Researchers, developers, and practitioners working with Vietnamese language AI
- **Out-of-scope use cases**: Production deployments without additional safety measures

## Training Details

### Training Data

The model underwent a sophisticated training process involving multiple stages of distillation and adaptation:

1. Initial knowledge distillation from Llama 3.1 405B
2. Architecture adaptation using mergekit-tokensurgeon
3. Secondary distillation to Qwen architecture
4. Parallel distillation from Qwen2-72B
5. Final fusion and fine-tuning using EvolKit dataset

### Training Procedure

#### Distillation Process
1. **Logit Distillation**
   - Source: Llama 3.1 405B
   - Method: Offline distillation
   - Storage: Top-K logits preservation
   
2. **Cross-Architecture Adaptation**
   - Tool: mergekit-tokensurgeon
   - Process: Vocabulary alignment with Llama 3.1 405B
   
3. **Architecture Transformation**
   - Target: 3B parameter configuration
   - Method: Progressive knowledge transfer

#### Fine-tuning
- **Final Stage**: EvolKit dataset utilization
- **Optimization**: Focus on coherence and reasoning capabilities
- **Vocabulary**: Qwen-native vocabulary restoration

## Performance and Limitations

### Benchmarks
Will be updated throughout the day

### Limitations
- Model size constraints may impact certain complex reasoning tasks
- Performance may vary on domain-specific Vietnamese content
- Limited context window compared to larger models

## Ethical Considerations

- **Data Bias**: May reflect biases present in training data
- **Environmental Impact**: Reduced compared to larger models due to efficient distillation
- **Societal Impact**: Potential influence on Vietnamese language technology landscape

## Technical Specifications

- **Parameter Count**: 3 billion
- **Context Window**: 32K