Update README.md

README.md

@@ -7,18 +7,97 @@ tags:
 
 ## Model description
 
-More information needed
+ShiftViT is a variation of the Vision Transformer (ViT) where the attention operation has been swapped with a shifting operation.
 
-## Intended uses & limitations
+ShiftViT model was proposed as part of the paper [When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanism](https://arxiv.org/abs/2201.10801).
+Vision Transformers have lately become very popular for computer vision problems and a lot researchers attribute their success to the attention layers. The authors of the ShiftViT paper have tried to show via the ShiftViT model that even without the attention operation, ViTs can reach SoTA results.  
 
-More information needed
+## Model Architecture
+
+The architecture for ShiftViT is inspired by the paper  [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://arxiv.org/abs/2103.14030)
+
+| ![ShiftViT Architecture](https://i.imgur.com/CHU40HX.png) |
+| :--: |
+| Figure 1: The entire architecutre of ShiftViT.
+[Source](https://arxiv.org/abs/2201.10801) |
+
+Here the authors propose a modular architecture with 4 stages. Each stage works on its own spatial size, creating a hierarchical architecture.
+
+
+Each stage in the ShiftViT architecture comprises of a Shift Block as shown in Fig 2.
+
+| ![ShiftViT block](https://i.imgur.com/IDe35vo.gif) |
+| :--: |
+| Figure 2: From the Model to a Shift Block. |
+
+The Shift Block as shown in Fig. 3, comprises of the following:
+
+Shift Operation
+Linear Normalization
+MLP Layer: stack of densely-connected layers
+
+
+1. Split the channels 
+2. Select each of the first four spilts and shift and pad them in the respective directions.
+3. After shifting and padding, we concatenate the channel back.
+
+ | ![Manim rendered animation for shift operation](https://i.imgur.com/PReeULP.gif) |
+| :--: |
+| Figure 4: The TensorFlow style shifting |
+
+## Intended uses 
+
+The ShiftViT model architecture has been designed to 
 
 ## Training and evaluation data
 
-More information needed
+The dataset used for training the model is CIFAR-10.
+The CIFAR-10 dataset is a popular dataset used for image classification. It contains images belonging to the following 10 classes:
+
+| Classes | 
+| :-- | 
+| airplane | 
+| automobile | 
+| bird | 
+| cat | 
+| deer | 
+| dog | 
+| frog | 
+| horse | 
+| ship | 
+| truck |  
+
+No. of samples used for training and evaluation are:
+- Training samples: 40000
+- Validation samples: 10000
+- Testing samples: 10000
 
 ## Training procedure
 
+1. Data Preparation:
+
+ - Data Augmentation: The augmentation steps include: rescaling, resizing, cropping and horizontal flipping.
+ 
+ 2. Building the ShiftViT Model:
+ 
+ - 
+ 
+ 3. Model Training:
+ 
+ The model is then trained using the following config:
+ 
+ | Training Config | Value |
+| :-- | :-- |
+| Optimizer | Adam |
+| Loss Function | sparse_categorical_crossentropy|
+| Metric | Accuracy |
+| Epochs | 5 |
+
+4. Model Testing:
+
+The model is tested on the test data post training achieving an accuracy of ~90%.
+
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training: