Hugging Face's logo

LilaBoualili
/
colbert-distilbert-margin_mse-T2-msmarco-encoder-only

Feature Extraction
Transformers
PyTorch
English
distilbert
dense-retrieval
knowledge-distillation
text-embeddings-inference
Inference Endpoints
Model card Files Community

Margin-MSE Trained ColBERT encoder model only

We provide the encoder model of the complete retrieval trained DistilBert-based ColBERT model. This model is trained with Margin-MSE using a 3 teacher BERT_Cat (concatenated BERT scoring) ensemble on MSMARCO-Passage, for more details check the full model card.

This encoder-only model is used as the oracle for distilling term topic embeddings in our ECIR'23 paper.

Downloads last month
11
Inference Examples
Feature Extraction
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train LilaBoualili/colbert-distilbert-margin_mse-T2-msmarco-encoder-only