jukofyork commited on
Commit
4fd94b3
1 Parent(s): 526f198

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -63,7 +63,7 @@ tags:
63
 
64
  #### 3. Run the model on a random sample of ~1k prompts on each of the 3 classes:
65
 
66
- - It is important that the same `'pre-prompt x prompt'` sample be used with each ```"baseline"```, ```"undesired"``` and ```"desired"``` triplet.
67
  - This takes the total number of hidden-state samples I recorded to: ```3 x 10 x 1000 = 30,000``` (per layer!).
68
  - This may seem like a lot compared to what other people are using to create control vectors with, but the theory regarding [estimation of covariance matrices](https://en.wikipedia.org/wiki/Estimation_of_covariance_matrices) shows we need at the ***very least*** a minimum of [one sample per feature](https://stats.stackexchange.com/questions/90045/how-many-samples-are-needed-to-estimate-a-p-dimensional-covariance-matrix) (and the models uploaded here have between 4k and 11.5k hidden state dimensions!).
69
 
 
63
 
64
  #### 3. Run the model on a random sample of ~1k prompts on each of the 3 classes:
65
 
66
+ - It is important that the same `'pre-prompt x prompt'` sample be used with each (```"baseline"```, ```"undesired"```, ```"desired"```) triplet.
67
  - This takes the total number of hidden-state samples I recorded to: ```3 x 10 x 1000 = 30,000``` (per layer!).
68
  - This may seem like a lot compared to what other people are using to create control vectors with, but the theory regarding [estimation of covariance matrices](https://en.wikipedia.org/wiki/Estimation_of_covariance_matrices) shows we need at the ***very least*** a minimum of [one sample per feature](https://stats.stackexchange.com/questions/90045/how-many-samples-are-needed-to-estimate-a-p-dimensional-covariance-matrix) (and the models uploaded here have between 4k and 11.5k hidden state dimensions!).
69