File size: 2,927 Bytes
2570e4e 761adfb b4efb25 73f5f27 b4efb25 7cf9278 b4efb25 0e28b67 b4efb25 761adfb 9309ac4 8635999 9309ac4 e888dd5 8635999 b4efb25 8635999 b4efb25 8635999 2e76bb1 9309ac4 b4efb25 9309ac4 14bff6c 761adfb b4efb25 14bff6c b4efb25 2e76bb1 b4efb25 9309ac4 2e76bb1 14bff6c 2e76bb1 9309ac4 2e76bb1 b4efb25 2e76bb1 9309ac4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
---
base_model:
- timm/vit_small_patch16_384.augreg_in21k_ft_in1k
library_name: transformers
license: mit
pipeline_tag: image-classification
tags:
- image-classification
- timm
- transformers
- detection
- deepfake
- forensics
- deepfake_detection
- community
- opensight
---
# Trained on 2.7M samples across 4,803 generators (see Training Data)
Model presented in [Community Forensics: Using Thousands of Generators to Train Fake Image Detectors](https://huggingface.co/papers/2411.04125).
**Uploaded for community validation as part of OpenSight** - An upcoming open-source framework for adaptive deepfake detection.
**Project OpenSight HF Spaces coming soon with an eval playground and eventually a leaderboard. Preview:**
## Model Details
### Model Description
Vision Transformer (ViT) model trained on the largest dataset to-date for detecting AI-generated images in forensic applications.
- **Developed by:** Jeongsoo Park and Andrew Owens, University of Michigan
- **Model type:** Vision Transformer (ViT-Small)
- **License:** MIT (compatible with CreativeML OpenRAIL-M referenced in [2411.04125v1.pdf])
- **Finetuned from:** timm/vit_small_patch16_384.augreg_in21k_ft_in1k
- **Adapted for HF** inference compatibility by AI Without Borders.
**HF Space will be open sourced shortly showcasing various ways to run ultra-fast inference. Make sure to follow us for updates, as we will be releasing a slew of projects in the coming weeks.**
### Links
- **Repository:** [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
- **Paper:** [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
- **Project Page:** https://jespark.net/projects/2024/community_forensics
## Training Details
### Training Data
- 2.7mil images from 15+ generators, 4600+ models
- Over 1.15TB worth of images
### Training Hyperparameters
- **Framework:** PyTorch 2.0
- **Precision:** bf16 mixed
- **Optimizer:** AdamW (lr=5e-5)
- **Epochs:** 10
- **Batch Size:** 32
## Evaluation
### Unverified Testing Results
- Only unverified because we currently lack resources to evaluate a dataset over 1.4T large.
| Metric | Value |
|---------------|-------|
| Accuracy | 97.2% |
| F1 Score | 0.968 |
| AUC-ROC | 0.992 |
| FP Rate | 2.1% |
## Re-sampled and refined dataset
- **Coming soon™**
## Citation
**BibTeX:**
```bibtex
@misc{park2024communityforensics,
title={Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
author={Jeongsoo Park and Andrew Owens},
year={2024},
eprint={2411.04125},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2411.04125},
}
``` |