jukofyork commited on
Commit
5c8003b
1 Parent(s): 6aa7fb0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -93,7 +93,7 @@ tags:
93
 
94
  #### 7. Finally, we project the "direction" to reorient and scale as necessary:
95
 
96
- - There is no reason the ```eigenvectors``` point in the direction we want, so 50% of the time we have to flip all the signs by [projecting](https://en.wikipedia.org/wiki/Projection_(linear_algebra)) our (differenced) "desired" dataset on to the (unit norm) direction and then testing the sign of the mean.
97
  - Due to the way the LLMs work via the "residual stream", the hidden states tend to get larger and larger as the layers progress, so to normalize this we also scale by the magnitude of the mean of the same projection as above.
98
 
99
  **NOTES**:
 
93
 
94
  #### 7. Finally, we project the "direction" to reorient and scale as necessary:
95
 
96
+ - There is no reason the ```eigenvectors``` point in the direction we want, so 50% of the time we have to flip all the signs by [projecting](https://en.wikipedia.org/wiki/Projection_(linear_algebra)) our (differenced) "desired" dataset on to the (unit norm) direction and then test the sign of the mean.
97
  - Due to the way the LLMs work via the "residual stream", the hidden states tend to get larger and larger as the layers progress, so to normalize this we also scale by the magnitude of the mean of the same projection as above.
98
 
99
  **NOTES**: