jmvcoelho
/

t5-base-marco-crop-pretrain-2048

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

t5-base-marco-crop-pretrain-2048 / README.md

jmvcoelho's picture

Create README.md

8b479fa verified 5 months ago

|

history blame contribute delete

359 Bytes

Model pre-trained on MS-MARCO (contrastive).

Starting model: jmvcoelho/t5-base-marco-lm-pretrain-2048

Training script: https://github.com/cxcscmu/LongEmbeddingAnalysis/blob/main/scripts/contrastive_pretrain_dr.sh
T5ForConditionalGenerationRoPE class: https://github.com/cxcscmu/LongEmbeddingAnalysis/blob/main/OpenMatch/src/openmatch/modeling/rope_t5.py