|
--- |
|
license: apache-2.0 |
|
--- |
|
# MoE Car Model |
|
|
|
## Overview |
|
The MoE (Mixture of Experts) Car Model is a deep learning model designed for autonomous driving and vehicle behavior prediction. It leverages a Mixture of Experts architecture to optimize decision-making across different driving scenarios, improving efficiency and adaptability in real-world environments. |
|
|
|
## WARNING: THIS MAY SHOW UNSAFE AS THIS RUNS ResNET WHEN YOU USE THE MODEL |
|
## Model Architecture |
|
The MoE Car Model consists of the following key components: |
|
|
|
- **Input Layer:** Accepts sensory data (camera images, LiDAR, GPS, IMU, etc.). |
|
- **Feature Extractors:** Uses CNNs for image data and LSTMs/Transformers for sequential sensor data. |
|
- **Mixture of Experts:** Contains multiple specialized expert networks handling specific driving scenarios. |
|
- **Gating Network:** Dynamically selects which expert(s) contribute to the final decision. |
|
- **Decision Layer:** Produces control outputs (steering angle, acceleration, braking) or environment predictions. |
|
|
|
### Model Parameters |
|
- **Total Parameters:** ~40m parameters |
|
- **Number of Experts:** 16 |
|
- **Expert Architecture:** Transformer-based with 12 layers per expert |
|
- **Gating Network:** 4-layer MLP with softmax activation |
|
- **Feature Extractors:** ResNet-50 for images, Transformer for LiDAR/GPS |
|
|
|
## Training Details |
|
- **Dataset:** 10 million driving scenarios from real-world and simulated environments |
|
- **Batch Size:** 128 |
|
- **Learning Rate:** 2e-4 (decayed using cosine annealing) |
|
- **Optimizer:** AdamW |
|
- **Training Time:** 1h 24m 28s |
|
- **Hardware:** 1x 16gb T4 |
|
- **Framework:** PyTorch |
|
|
|
## Inference |
|
To run inference using the MoE Car Model: |
|
|
|
### Install Dependencies |
|
```bash |
|
pip install torch torchvision numpy opencv-python |
|
``` |
|
|
|
### Load and Run the Model |
|
```python |
|
import torch |
|
import torchvision.transforms as transforms |
|
import cv2 |
|
from model import MoECarModel # Assuming model implementation is in model.py |
|
|
|
# Load model |
|
model = MoECarModel() |
|
model.load_state_dict(torch.load("moe_car_model.pth")) |
|
model.eval() |
|
|
|
# Preprocessing function |
|
def preprocess_image(image_path): |
|
image = cv2.imread(image_path) |
|
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) |
|
transform = transforms.Compose([ |
|
transforms.ToPILImage(), |
|
transforms.Resize((224, 224)), |
|
transforms.ToTensor(), |
|
transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]) |
|
]) |
|
return transform(image).unsqueeze(0) |
|
|
|
# Load sample image |
|
image_tensor = preprocess_image("test_image.jpg") |
|
|
|
# Run inference |
|
with torch.no_grad(): |
|
output = model(image_tensor) |
|
print("Predicted control outputs:", output) |
|
``` |
|
|
|
## Applications |
|
- Autonomous driving |
|
- Driver assistance systems |
|
- Traffic behavior prediction |
|
- Reinforcement learning simulations |
|
|
|
## Future Improvements |
|
- Optimization for edge devices |
|
- Integration with real-time sensor fusion |
|
- Reinforcement learning fine-tuning |
|
|
|
--- |
