Update README.md

README.md

@@ -240,10 +240,17 @@ These prompts were run across two datasets [fairface](https://huggingface.co/dat
 The FairFace dataset is "a face image dataset which is race balanced. It contains 108,501 images from 7 different race groups: White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, and Latino. Images were collected from the YFCC-100M Flickr dataset and labelled with race, gender, and age groups". 
 The Stable Bias dataset is a dataset of synthetically generated images from the prompt "A photo portrait of a (ethnicity) (gender) at work.". 
 
-Running the above prompts across both these datasets results in two datasets containing three generated responses for each image in the dataset alongside information about the ascribed ethnicity and gender of the person depicted in each image.
-This allows for the generated response to each prompt to be compared across gender and ethnicity axis.
+Running the above prompts across both these datasets results in two datasets containing three generated responses for each image alongside information about the ascribed ethnicity and gender of the person depicted in each image.
+This allows for the generated response to each prompt to be compared across  gender and ethnicity axis.
 Our goal in performing this evaluation was to try to identify more subtle ways in which the responses generated by the model may be influenced by the gender or ethnicity of the person depicted in the input image.
 
+To surface potential biases in the outputs, we consider the following simple [TF-IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) based approach. Given a model and a prompt of interest, we:
+1. Evaluate Inverse Document Frequencies on the full set of generations for the model and prompt in questions
+2. Compute the average TFIDF vectors for all generations **for a given gender or ethnicity**
+3. Sort the terms by variance to see words that appear significantly more for a given gender or ethnicity
+
+With this approach, we can see subtle differences in the frequency of terms across gender and ethnicity. For example, for the prompt related to resumes, we see that synthetic images generated for `non-binary` are more likely to lead to resumes that include **data** or **science** than those generated for `man` or `woman`.
+When looking at the response to the arrest prompt for the FairFace dataset the term `theft` is more frequently associated with `East Asian`, `Indian`, `Black` and `Southeast Asian` compared to `White` and `Middle Eastern`.  
 
 ## Other limitations