Update README.md

README.md

@@ -2,49 +2,92 @@
 tags:
 - huggan
 - gan
+task: unconditional-image-generation
+license: mit
 # See a list of available tags here:
 # https://github.com/huggingface/hub-docs/blob/main/js/src/lib/interfaces/Types.ts#L12
-# task: unconditional-image-generation or conditional-image-generation or image-to-image
-license: mit
+
 ---
 
-# MyModelName
+# Butterfly GAN
 
 ## Model description
 
-Describe the model here (what it does, what it's used for, etc.)
+Based on [paper:](https://openreview.net/forum?id=1Fqg133qRaI) *Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis* 
+
+which states:
+"Notably, the model converges from scratch with just a **few hours of training** on a single RTX-2080 GPU, and has a consistent performance, even with **less than 100 training samples**"
+
+also dubbed the Light-GAN model. This model was trained using the script [here](https://github.com/huggingface/community-events/tree/main/huggan/pytorch/lightweight_gan) which is adapted from the lucidrains [repo](https://github.com/lucidrains/lightweight-gan).
+
+Differently from the script above, I used the transforms from the official repo. Because our training images were already cropped and aligned. 
+official paper implementation [repo](https://github.com/odegeasslbc/FastGAN-pytorch)
+
+```py
+transform_list = [
+            transforms.Resize((int(im_size),int(im_size))),
+            transforms.RandomHorizontalFlip(),
+            transforms.ToTensor(),
+            transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
+        ]        
+```
 
 ## Intended uses & limitations
 
+Intended for fun & learning~
+
 #### How to use
 
 ```python
-# You can include sample code which will be formatted
+
+import torch
+from huggan.pytorch.lightweight_gan.lightweight_gan import LightweightGAN # install the community-events repo above
+
+gan = LightweightGAN.from_pretrained("ceyda/butterfly_cropped_uniq1K_512")
+gan.eval()
+batch_size = 1
+with torch.no_grad():
+        ims = gan.G(torch.randn(batch_size, gan.latent_dim)).clamp_(0., 1.)*255
+        ims = ims.permute(0,2,3,1).detach().cpu().numpy().astype(np.uint8)
+        # ims is [BxWxHxC] call Image.fromarray(ims[0])
 ```
 
 #### Limitations and bias
 
-Provide examples of latent issues and potential remediations.
+- During training I filtered the dataset to have only 1 butterfly from each species available.
+Otherwise the model generated less varied butterflies  (a few species with more images would dominate).
+- The dataset was also filtered using CLIP scores for ['pretty butterfly','one butterfly','butterfly with open wings','colorful butterfly']. 
+While this was done to eliminate images that contained no butterflies(just scientific tags, cluttered images) from the [full dataset](https://huggingface.co/datasets/ceyda/smithsonian_butterflies). 
+It is easy to imagine where this type of approach would be problematic in certain scenarios; who is to say which butterfly is "pretty" and should be in the dataset.ie; CLIP failing to identify a butterfly might exclude it from the dataset causing bias.
 
 ## Training data
 
-Describe the data you used to train the model.
-If you initialized it with pre-trained weights, add a link to the pre-trained model card or repository with description of the pre-training data.
+1000 images are used, while it was possible to increase this number, we didn't have time to manually curate the dataset.
+
+& also wanted to see if it was possible to do low data training as mention in the paper.
+ 
+More details are on the [data card](https://huggingface.co/datasets/huggan/smithsonian_butterflies_subset)
 
 ## Training procedure
 
-Preprocessing, hardware used, hyperparameters...
+Trained on 2xA4000s for ~1day. Can see good results within 7-12h.
+Importans params: "--batch_size 64 --gradient_accumulate_every 4 --image_size 512 --mixed_precision fp16"
+Training logs can be seen [here](https://wandb.ai/cceyda/butterfly-gan/runs/2e0bm7h8?workspace=user-cceyda)
 
 ## Eval results
 
+calculated FID score on 100 images. results for different checkpoints are [here](https://wandb.ai/cceyda/butterfly-gan-fid?workspace=user-cceyda)
+
+but can't say it is too meaningful (due to the shortcomings of FID score)
+
 ## Generated Images
 
-You can embed local or remote images using `![](...)`
+Play with the [demo](https://huggingface.co/spaces/huggan/butterfly-gan)
 
 ### BibTeX entry and citation info
 
-```bibtex
-@inproceedings{...,
-  year={2020}
-}
-```
+Made during the huggan sprint.
+
+Model trained by: Ceyda Cinarel https://twitter.com/ceyda_cinarel
+
+Additional contributions by Jonathan Whitaker https://twitter.com/johnowhitaker