Update README.md

## Usage:

- How to load the text encoder (with CLIP's image decoder)
- How to use the model

## Eample of model usage in these tasks:

- Zero shot classification

- Text-Image retrieval

## Citation (todo: discuss with Boat)

README.md

@@ -8,15 +8,14 @@ tags:
 <br>
 
 ## How to use
+- #### Install python package
 ```python
-from transformers import AutoModel,AutoProcessor
-model = AutoModel.from_pretrained("openthaigpt/CLIPTextCamembertModelWithProjection-contrastive", trust_remote_code=True)
-processor = AutoProcessor.from_pretrained("openthaigpt/CLIPTextCamembertModelWithProjection-contrastive", trust_remote_
-```
+pip thai2transformers==0.1.2
 
-### Preprocessing
+```
+- ### Preprocessing
 
-Texts are preprocessed with the following rules:
+Texts are preprocessed with the following rules: [process_transformers](https://github.com/vistec-AI/thai2transformers/blob/master/thai2transformers/preprocess.py)
 
 - Replace HTML forms of characters with the actual characters such asnbsp;with a space and \\\\\\\\\\\\\\\\<br /> with a line break [[Howard and Ruder, 2018]](https://arxiv.org/abs/1801.06146).
 - Remove empty brackets ((), {}, and []) than sometimes come up as a result of text extraction such as from Wikipedia.
@@ -26,5 +25,126 @@ Texts are preprocessed with the following rules:
 - Word-level tokenization using [[Phatthiyaphaibun et al., 2020]](https://zenodo.org/record/4319685#.YA4xEGQzaDU) ’s `newmm` dictionary-based maximal matching tokenizer.
 - Replace repetitive words; this is done post-tokenization unlike [[Howard and Ruder, 2018]](https://arxiv.org/abs/1801.06146). since there is no delimitation by space in Thai as in English.
 - Replace spaces with <\\\\\\\\\\\\\\\\_>. The SentencePiece tokenizer combines the spaces with other tokens. Since spaces serve as punctuation in Thai such as sentence boundaries similar to periods in English, combining it with other tokens will omit an important feature for tasks such as word tokenization and sentence breaking. Therefore, we opt to explicitly mark spaces with <\\\\\\\\\\\\\\\\_>.
+<br>
+
+- #### How to load the text encoder
+
+```python
+from transformers import AutoModel,AutoProcessor
+from thai2transformers.preprocess import process_transformers
+model = AutoModel.from_pretrained("openthaigpt/CLIPTextCamembertModelWithProjection-contrastive", trust_remote_code=True)
+processor = AutoProcessor.from_pretrained("openthaigpt/CLIPTextCamembertModelWithProjection-contrastive", trust_remote_code=True)
+
+input_text = ["This is dog",
+              "how are you today",
+              "สวัสดีครับ วันนี้อากาศร้อนมาก"]
+processed_input_text = [process_transformers(input_text_) for input_text_ in input_text ]
+
+text_tokens = processor(text=processed_input_text, padding=True, return_tensors="pt")
+embedding = model(**text_tokens).text_embeds
+
+print(embedding,embedding.shape)
+```
+- #### Output:
+```python
+tensor([[ 0.0318,  0.0341, -0.1317,  ..., -0.2763, -0.2103,  0.0968],
+        [ 0.0579, -0.1373, -0.0293,  ..., -0.3926, -0.2002, -0.0497],
+        [ 0.0303,  0.0440,  0.0217,  ..., -0.3282, -0.0100, -0.0757]],
+       grad_fn=<MmBackward0>) torch.Size([3, 512])
+```
+
+## Eample of model usage
+
+- ### Zero shot classification
+
+```python
+from torch import FloatTensor, IntTensor, Tensor
+from transformers import AutoModel, AutoProcessor, CLIPModel
+
+# Load image model and processor.
+image_processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32")
+image_model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32").to(device)
+
+# Load text model and processor.
+text_processor = AutoProcessor.from_pretrained("openthaigpt/CLIPTextCamembertModelWithProjection-contrastive", trust_remote_code=True)
+text_model = AutoModel.from_pretrained("openthaigpt/CLIPTextCamembertModelWithProjection-contrastive", trust_remote_code=True).to(device)
+
+class_labels = ['แมว','หมา', 'นก']
+label2id = {label: i for i, label in enumerate(class_labels)}
+
+inputs = text_processor(text=class_labels, padding=True, return_tensors="pt")
+inputs = {name: tensor.to(self.device) for name, tensor in inputs.items()}       
+text_embeddings = self.text_model(**inputs).text_embeds
+text_embeddings /= text_embeddings.norm(dim=1, keepdim=True)
+
+inputs = image_processor(images=images, return_tensors="pt")
+inputs = {name: tensor.to(self.device) for name, tensor in inputs.items()}
+image_embeddings = self.image_model.get_image_features(**inputs)
+image_embeddings /= image_embeddings.norm(dim=1, keepdim=True)
+
+similarities = torch.mm(image_embeddings, text_embeddings.t())
+logits = F.softmax(similarities, dim=1)
+indices = torch.argmax(logits, dim=1)
+
+logits = logits.detach().cpu()
+indices = indices.detach().cpu()
+
+predict= [class_labels[i] for i in indices ]
+
+```
+
+- ### Text-Image retrieval
+
+```python
+import faiss
+from torch import FloatTensor, IntTensor, Tensor
+from transformers import AutoModel, AutoProcessor, CLIPModel
+
+# Load image model and processor.
+image_processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32")
+image_model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32").to(device)
+
+# Load text model and processor.
+text_processor = AutoProcessor.from_pretrained("openthaigpt/CLIPTextCamembertModelWithProjection-contrastive", trust_remote_code=True)
+text_model = AutoModel.from_pretrained("openthaigpt/CLIPTextCamembertModelWithProjection-contrastive", trust_remote_code=True).to(device)
+
+text_input = ['แมวสีส้ม','หมาสีดำ', 'นกสีขาว']
+processed_input_text = [process_transformers(input_text_) for input_text_ in input_text ]
+
+
+inputs = text_processor(text=processed_input_text, padding=True, return_tensors="pt")
+inputs = {name: tensor.to(self.device) for name, tensor in inputs.items()}       
+text_embeddings = self.text_model(**inputs).text_embeds
+text_embeddings /= text_embeddings.norm(dim=1, keepdim=True)
+
+inputs = image_processor(images=images, return_tensors="pt")
+inputs = {name: tensor.to(self.device) for name, tensor in inputs.items()}
+image_embeddings = self.image_model.get_image_features(**inputs)
+image_embeddings /= image_embeddings.norm(dim=1, keepdim=True)
+
+
+n = text_embeddings.shape[1]
+
+text_index = faiss.IndexFlatIP(n)
+image_index = faiss.IndexFlatIP(n)
+text_index.add(text_embeddings)
+image_index.add(image_embeddings)
+
+
+# Get_image_search_recall_at_k
+distances, retrieved_indices = image_index.search(text_embeddings, k=5)
+recall_image_search = sum(1.0 if i in indices else 0.0
+            for i, indices in zip(range(n), retrieved_indices)
+        ) / float(n)
+
+# Get_text_search_recall_at_k
+distances, retrieved_indices = text_index.search(image_embeddings, k=5)
+recall_text_search = sum(1.0 if i in indices else 0.0
+            for i, indices in zip(range(n), retrieved_indices)
+        ) / float(n)
+
+```
+
 
-<br>
+### Authors
+* Konthee Boonmeeprakob ([email protected])