jukofyork
/

creative-writing-control-vectors-v1.0

creative-writing

Model card Files Files and versions Community

jukofyork commited on Jul 11

Commit

12c67ae

•

1 Parent(s): c76f23f

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -88,7 +88,7 @@ license: apache-2.0
 #### 7. Finally, we project the "direction" to reorient and scale as necessary:
-- There is no reason the ```eigenvectors``` point in the direction we want, so 50% of the time we have to flip all the signs by [projecting](https://en.wikipedia.org/wiki/Projection_(linear_algebra)) of our (differenced) "desired" dataset on to the direction and then testing the sign of the mean.
 - Due to the way the LLMs work via the "residual stream", the hidden states tend to get larger and larger as the layers progress, so to normalize this we also scale by the magnitude of the mean [projection](https://en.wikipedia.org/wiki/Projection_(linear_algebra)) of our (differenced) "desired" dataset.
 **NOTES**:

 #### 7. Finally, we project the "direction" to reorient and scale as necessary:
+- There is no reason the ```eigenvectors``` point in the direction we want, so 50% of the time we have to flip all the signs by [projecting](https://en.wikipedia.org/wiki/Projection_(linear_algebra)) our (differenced) "desired" dataset on to the (unit norm) direction and then testing the sign of the mean.
 - Due to the way the LLMs work via the "residual stream", the hidden states tend to get larger and larger as the layers progress, so to normalize this we also scale by the magnitude of the mean [projection](https://en.wikipedia.org/wiki/Projection_(linear_algebra)) of our (differenced) "desired" dataset.
 **NOTES**: