Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,62 @@
|
|
1 |
-
---
|
2 |
-
license: mit
|
3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
---
|
4 |
+
|
5 |
+
# Self-Supervised Learning Framework
|
6 |
+
|
7 |
+
This project implements a Self-Supervised Learning (SSL) framework using the CIFAR-10 dataset and a ResNet-18 backbone. The goal of the project is to train a model to learn robust image representations without relying on labeled data. This framework utilizes contrastive learning with data augmentations and a custom contrastive loss function.
|
8 |
+
|
9 |
+
---
|
10 |
+
|
11 |
+
## **Features**
|
12 |
+
**Data Augmentation**:
|
13 |
+
|
14 |
+
- Random cropping, flipping, color jitter, grayscale conversion, Gaussian blur, and normalization.
|
15 |
+
|
16 |
+
**Backbone Architecture**:
|
17 |
+
|
18 |
+
- ResNet-18 with a custom projection head.
|
19 |
+
|
20 |
+
**Contrastive Learning**:
|
21 |
+
|
22 |
+
- Contrastive loss function with positive and negative pair sampling.
|
23 |
+
|
24 |
+
**Optimization**:
|
25 |
+
|
26 |
+
- Gradient clipping and weight decay for numerical stability.
|
27 |
+
|
28 |
+
**Model Checkpointing**:
|
29 |
+
|
30 |
+
- Save model weights at the end of each epoch.
|
31 |
+
|
32 |
+
|
33 |
+
## **How It Works**
|
34 |
+
1. **Data Augmentation**:
|
35 |
+
- Two augmented views of each image are created for contrastive learning.
|
36 |
+
|
37 |
+
2. **Contrastive Loss**:
|
38 |
+
- Positive pairs: Augmented views of the same image.
|
39 |
+
- Negative pairs: Augmented views of different images.
|
40 |
+
- Loss is computed using the similarity of positive pairs while minimizing similarity with negative pairs.
|
41 |
+
|
42 |
+
3. **Optimization**:
|
43 |
+
- The model uses the Adam optimizer with a learning rate of `3e-4` and weight decay of `1e-4`.
|
44 |
+
- Gradient clipping ensures numerical stability.
|
45 |
+
|
46 |
+
---
|
47 |
+
|
48 |
+
## **Results and Evaluation**
|
49 |
+
- **Training Loss**:
|
50 |
+
- Observe the training loss decreasing across epochs, indicating successful representation learning.
|
51 |
+
- **Downstream Tasks**:
|
52 |
+
- Evaluate the learned embeddings on classification or clustering tasks.
|
53 |
+
|
54 |
+
---
|
55 |
+
|
56 |
+
## **Acknowledgments**
|
57 |
+
- CIFAR-10 dataset: https://www.cs.toronto.edu/~kriz/cifar.html
|
58 |
+
- PyTorch: https://pytorch.org/
|
59 |
+
- ResNet-18 architecture.
|
60 |
+
|
61 |
+
---
|
62 |
+
|