|
--- |
|
language: en |
|
license: llama3.1 |
|
library_name: sentence-transformers |
|
tags: |
|
- sentence-transformers |
|
- feature-extraction |
|
- sentence-similarity |
|
- transformers |
|
datasets: |
|
- beeformer/recsys-movielens-20m |
|
- beeformer/recsys-goodbooks-10k |
|
pipeline_tag: sentence-similarity |
|
--- |
|
# Llama-goodlens-mpnet |
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and it is designed to use in recommender systems for content-base filtering and as a side information for cold-start recommendation. |
|
|
|
## Usage (Sentence-Transformers) |
|
Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed: |
|
|
|
``` |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can use the model like this: |
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
sentences = ["This is an example product description", "Each product description is converted"] |
|
model = SentenceTransformer('beeformer/Llama-goodlens-mpnet') |
|
embeddings = model.encode(sentences) |
|
print(embeddings) |
|
``` |
|
|
|
## Training procedure |
|
|
|
### Pre-training |
|
|
|
We use the pretrained [`sentence-transformers/all-mpnet-base-v2`](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) model. Please refer to the model card for more detailed information about the pre-training procedure. |
|
|
|
### Fine-tuning |
|
|
|
We use the initial model without modifying its architecture or pre-trained model parameters. |
|
However, we reduce the processed sequence length to 384 to reduce the training time of the model. |
|
|
|
### Dataset |
|
|
|
We finetuned our model on the combination of the Goodbooks-10k and the MovieLens20M datasets with item descriptions generated with [`meta-llama/Meta-Llama-3.1-8B-Instruct`](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) model. For details please see the dataset pages: [`beeformer/recsys-movielens-20m`](https://huggingface.co/datasets/beeformer/recsys-movielens-20m) and [`beeformer/recsys-goodbooks-10k`](https://huggingface.co/datasets/beeformer/recsys-goodbooks-10k). |
|
|
|
## Evaluation Results |
|
|
|
Table with results TBA. |
|
|
|
## Intended uses |
|
|
|
This model was trained as a demonstration of capabilities of the beeFormer training framework (link and details TBA) and is intended for research purposes only. |
|
|
|
## Citation |
|
|
|
Preprint available [here](https://arxiv.org/pdf/2409.10309) |
|
|
|
TBA |
