Create README.md

README.md

@@ -0,0 +1,93 @@
+---
+license: mit
+datasets:
+- awsaf49/sonics
+language:
+- en
+metrics:
+- f1
+pipeline_tag: audio-classification
+tags:
+- deepfake
+- audio_classification
+- fake_song_detection
+- music
+- song
+---
+
+<div align="center">
+    <img src="https://i.postimg.cc/3Jx3yZ5b/real-vs-fake-sonics-w-logo.jpg" width="250">
+</div>
+
+<div align="center">
+    <h1>SONICS: Synthetic Or Not - Identifying Counterfeit Songs</h1>
+    <h3><span style="color:red;"><b>ICLR 2025 [Poster]</b></span></h3>
+</div>
+
+
+## Overview 
+
+The recent surge in AI-generated songs presents exciting possibilities and challenges. These innovations necessitate the ability to distinguish between human-composed and synthetic songs to safeguard artistic integrity and protect human musical artistry. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI-generated but the instrumental music is sourced from real songs. However, these approaches are inadequate for detecting contemporary end-to-end artificial songs where all components (vocals, music, lyrics, and style) could be AI-generated.  Additionally, existing datasets lack music-lyrics diversity, long-duration songs, and open-access fake songs. To address these gaps, we introduce **SONICS**, a novel dataset for end-to-end **Synthetic Song Detection (SSD)**, comprising over **97k songs (4,751 hours)**, with over **49k synthetic songs** from popular platforms like **Suno and Udio**. Furthermore, we highlight the importance of modeling long-range temporal dependencies in songs for effective authenticity detection, an aspect entirely overlooked in existing methods.  To utilize long-range patterns, we introduce **SpecTTTra**, a novel architecture that significantly improves time and memory efficiency over conventional CNN and Transformer-based models. In particular, for long audio samples, our top-performing variant **outperforms ViT by 8% F1 score while being 38% faster and using 26% less memory**. Additionally, in comparison with ConvNeXt, our model achieves **1% gain in F1 score with a 20% boost in speed and 67% reduction in memory usage**.
+
+
+## Resources 
+
+- 📄 [**Paper**](https://openreview.net/forum?id=PY7KSh29Z8)  
+- 🎵 [**Dataset**](https://huggingface.co/datasets/awsaf49/sonics)  
+- 🔬 [**ArXiv**](https://arxiv.org/abs/2408.14080)  
+- 💻 [**GitHub**](https://github.com/awsaf49/sonics)  
+
+## Model Variants
+
+<style>
+    .hf-button {
+        display: inline-flex;
+        align-items: center;
+        gap: 6px;
+        padding: 6px 12px;
+        font-size: 14px;
+        font-weight: bold;
+        color: white;
+        border-radius: 6px;
+        text-decoration: none;
+    }
+    .hf-button img {
+        height: 18px;
+    }
+</style>
+
+| Model Name                     | HF Link | Variant        | Duration | f_clip | t_clip | F1  | Sensitivity | Specificity | Speed (A/S) | FLOPs (G) | Mem. (GB) | # Act. (M) | # Param. (M) |
+|--------------------------------|---------|---------------|----------|--------|--------|-----|-------------|-------------|-------------|-----------|-----------|------------|-------------|
+| `sonics-spectttra-alpha-5s`   | <a class="hf-button" href="https://huggingface.co/awsaf49/sonics-spectttra-alpha-5s"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg">HF</a>  | SpecTTTra-α   | 5s       | 1      | 3      | 0.78 | 0.69        | 0.94        | 148         | 2.9       | 0.5       | 6          | 17          |
+| `sonics-spectttra-beta-5s`    | <a class="hf-button" href="https://huggingface.co/awsaf49/sonics-spectttra-beta-5s"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg">HF</a>  | SpecTTTra-β   | 5s       | 3      | 5      | 0.78 | 0.69        | 0.94        | 152         | 1.1       | 0.2       | 5          | 17          |
+| `sonics-spectttra-gamma-5s`   | <a class="hf-button" href="https://huggingface.co/awsaf49/sonics-spectttra-gamma-5s"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg">HF</a>  | SpecTTTra-γ   | 5s       | 5      | 7      | 0.76 | 0.66        | 0.92        | 154         | 0.7       | 0.1       | 2          | 17          |
+| `sonics-spectttra-alpha-120s` | <a class="hf-button" href="https://huggingface.co/awsaf49/sonics-spectttra-alpha-120s"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg">HF</a>  | SpecTTTra-α   | 120s     | 1      | 3      | 0.97 | 0.96        | 0.99        | 47          | 23.7      | 3.9       | 50         | 19          |
+| `sonics-spectttra-beta-120s`  | <a class="hf-button" href="https://huggingface.co/awsaf49/sonics-spectttra-beta-120s"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg">HF</a>  | SpecTTTra-β   | 120s     | 3      | 5      | 0.92 | 0.86        | 0.99        | 80          | 14.0      | 2.3       | 29         | 17          |
+| `sonics-spectttra-gamma-120s` | <a class="hf-button" href="https://huggingface.co/awsaf49/sonics-spectttra-gamma-120s"><img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg">HF</a>  | SpecTTTra-γ   | 120s     | 5      | 7      | 0.97 | 0.96        | 0.99        | 97          | 10.1      | 1.6       | 138        | 22          |
+
+## Model Architecture
+
+- **Base Model:** SpectTTTra (Spectro-Temporal Tokens Transformer)  
+- **Embedding Dimension:** 384  
+- **Number of Heads:** 6  
+- **Number of Layers:** 12  
+- **MLP Ratio:** 2.67  
+
+## Audio Processing
+
+- **Sample Rate:** 16kHz  
+- **FFT Size:** 2048  
+- **Hop Length:** 512  
+- **Mel Bands:** 128  
+- **Frequency Range:** 20Hz - 8kHz  
+- **Normalization:** Mean-std normalization  
+
+## Usage 
+
+```python
+# Install from GitHub
+pip install git+https://github.com/awsaf49/sonics.git
+
+# Load model
+from sonics import HFAudioClassifier
+model = HFAudioClassifier.from_pretrained("awsaf49/sonics-spectttra-beta-120s")