File size: 976 Bytes
48b39ee 9a52c52 48b39ee |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
---
license: apache-2.0
---
## Word2Vec in a non overengineered file format
This is the [Word2Vec](https://code.google.com/archive/p/word2vec/) words embedding collection by Google(TM) converted in a sane binary format that you can read with a trivial program, without resorting to complicated libraries.
This is the format:
1. All the integers are unsigned, in little endian format.
2. Floating point numbers of the embedding vectors are stored as single precision floating point numbers, float32.
The file starts with a header of 8 bytes:
uint32_t num_entries // Number of entries: 3000000 in this file
uint32_t dimension // Embedding dimension: 300 in this file
Then each entry is stored as:
uint16_t word_len // Length of the UTF-8 encoded word.
word_len bytes // UTF-8 encoded word.
float[300] // 300 float32 numbers, 1200 bytes in total.
The license is the same as Google's original Word2Vec, Apache 2.0.
|