torinriley commited on
Commit
751f70a
·
verified ·
1 Parent(s): 29a4de2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +62 -3
README.md CHANGED
@@ -1,3 +1,62 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+
5
+ # Self-Supervised Learning Framework
6
+
7
+ This project implements a Self-Supervised Learning (SSL) framework using the CIFAR-10 dataset and a ResNet-18 backbone. The goal of the project is to train a model to learn robust image representations without relying on labeled data. This framework utilizes contrastive learning with data augmentations and a custom contrastive loss function.
8
+
9
+ ---
10
+
11
+ ## **Features**
12
+ **Data Augmentation**:
13
+
14
+ - Random cropping, flipping, color jitter, grayscale conversion, Gaussian blur, and normalization.
15
+
16
+ **Backbone Architecture**:
17
+
18
+ - ResNet-18 with a custom projection head.
19
+
20
+ **Contrastive Learning**:
21
+
22
+ - Contrastive loss function with positive and negative pair sampling.
23
+
24
+ **Optimization**:
25
+
26
+ - Gradient clipping and weight decay for numerical stability.
27
+
28
+ **Model Checkpointing**:
29
+
30
+ - Save model weights at the end of each epoch.
31
+
32
+
33
+ ## **How It Works**
34
+ 1. **Data Augmentation**:
35
+ - Two augmented views of each image are created for contrastive learning.
36
+
37
+ 2. **Contrastive Loss**:
38
+ - Positive pairs: Augmented views of the same image.
39
+ - Negative pairs: Augmented views of different images.
40
+ - Loss is computed using the similarity of positive pairs while minimizing similarity with negative pairs.
41
+
42
+ 3. **Optimization**:
43
+ - The model uses the Adam optimizer with a learning rate of `3e-4` and weight decay of `1e-4`.
44
+ - Gradient clipping ensures numerical stability.
45
+
46
+ ---
47
+
48
+ ## **Results and Evaluation**
49
+ - **Training Loss**:
50
+ - Observe the training loss decreasing across epochs, indicating successful representation learning.
51
+ - **Downstream Tasks**:
52
+ - Evaluate the learned embeddings on classification or clustering tasks.
53
+
54
+ ---
55
+
56
+ ## **Acknowledgments**
57
+ - CIFAR-10 dataset: https://www.cs.toronto.edu/~kriz/cifar.html
58
+ - PyTorch: https://pytorch.org/
59
+ - ResNet-18 architecture.
60
+
61
+ ---
62
+