Context length?
#4
by
vishaal27
- opened
Hi, great work, and great paper, really enjoyed reading it?
I couldn't find what context length you trained your models at? From the InternVL2.5 repo, it seems like they used a context length of 16384 for training, do you also use the same? Thanks!
We train our model with a 6400 sequence length during multi-model training, but as the based model is trained on 16384, a longer sequence is also supported.