# CLIP Sparse Autoencoder Checkpoint

## Model Overview
This model is a sparse autoencoder trained on CLIP's internal representations. Pretrained on Imagenet and Finetuned on Waterbirds


## Architecture Details
- **Layer**: 11  
- **Layer Type**: hook_resid_post
- **Model**: open-clip:laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K  
- **Dictionary Size**: 49,152  
- **Input Dimension**: 768  
- **Expansion Factor**: 64  
- **CLS Token Only**: False  

## Performance Metrics
The model has been evaluated on standard metrics with the following results:
- **L0**: 359  
- **Explained Variance**: 0.85  
- **MSE Loss**: 0.003  
- **Overall Loss**: 0.008  

## Additional Information
Detailed logs and visualizations of the model's fine-tuning process are available on **Weights & Biases**:  
[wandb.ai/perceptual-alignment/waterbirds-finetuning-sweep/runs/cxgrs9zt/workspace](https://wandb.ai/perceptual-alignment/waterbirds-finetuning-sweep/runs/cxgrs9zt/workspace)

---

Feel free to reach out for any additional clarifications or details!