README.md

metadata

base_model: avsolatorio/GIST-Embedding-v0
datasets: []
language: []
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:5864
  - loss:ContrastiveTensionLoss
widget:
  - source_sentence: >-
      pyrex blue cup rectangular plastic cover pc  pack  original genuine pyrex 
      made in the usa genuine pyrex replacement lid nonporous surface does not
      absorb food odors flavors or stains refrigerator microwave and toprack
      dishwasher safe will not fit anchor hocking products included  pyrex pc
      blue  cup rectangle plastic lids lids only containers not included
    sentences:
      - >-
        kpywzer vintage leather sling bag bags for men women backpack shoulder
        messenger crossbody outdoor travel hiking camping tactical chest pack
        daypack brown 
      - >-
        dove beauty bar original moisturizing bar made with  moisturizing cream
        gentle for soft skin   count  oz each lock in moisture dove original
        beauty bar is the classic white bar cleanser you trust to nourish and
        pamper skin and give you softer smoother more radiantlooking skin
        effective cleanser gently wash away dirt and germs for use every day on
        body face and hands this dove beauty bar creates luxurious lather and
        wont dry your skin like ordinary bar soaps no  dermatologist recommended
        formulated with  moisturizing cream dove moisturizing beauty bar and
        gentle skin cleanser leaves skin instantly soft and smooth with lasting
        nourishment versatile cleanser the unique formulation for effective skin
        care can be enjoyed by the entire family as this allinone cleanser
        nourishes your face body and hands for beautiful skin thoughtfully made
        this beauty bar is certified crueltyfree by peta and sulfate free
        cleansers so you can feel good about switching from ordinary bar soap to
        dove beauty bar care with your cleanser our mission is to help the next
        generation of women develop a positive relationship with the way they
        look helping them raise their selfesteem
      - >-
        professional stainless steel pots and pans setply  piece induction
        cookware setinduction oven broiler safe f pots and pans silver stainless
        steel professional cookware set geanbioso ply stainless steel pots and
        pans set combines the epitome in rapid heating heat retention and
        cooking stability each piece features cast stainless steel handles
        durable rivets measurement markings easytopour edges and allcooktop
        compatibility superior qualitypots and pans are crafted from heavyduty
        ply clad stainless steelthe outer layer is  magneticconducting stainless
        steel the middle is  layer aluminum alloycore for even heat transfer
        throughout the panthe pot is formed in one piece riveted links safe and
        solid healthy and environmentally friendly physically nonstick and the
        service life of up to  years or more suitable for all cooktopsgeanbioso
        ply stainless steel cookware set is suitable for all cooktops and is
        compatible with oven electric gas ceramic halogen and induction cooktops
        it is convenient to put the pans in the dishwasheroven safe up to 
        polished technique  easytoclean cookware setthis stainless steel pans
        and pots set interior surface used mirror polished technique it make
        stainless steel cookware sets effectively avoid deformation warping and
        rust easy to wash and quick cleanup this kitchen set not only have long
        use life also make housework much more easier and decorate your kitchen
        looks shining and high class this fabulous set consists of basics and
        more the  quart covered stockpot with lid takes care of everything from
        oatmeal to bechamel sauce also included are  and inch open skillets that
        turn out perfect grilled cheese hamburgers and crepesand an quart
        covered stockpot for stews and soups dishwashersafe this set has no
        coatings to flake off or plastic to melt and will surely be family
        businessmotivated by our year familyhistory in kitchen supplywe setout
        to multigenerational familyowned manufacturers in the usand western
        europe to buildsuperior culinary toolsour aim is to enhance the
        experience of your culinary creation and let them become your best
        kitchen companion
  - source_sentence: >-
      logitech g lightspeed wireless gaming headset with suspension headband
      lightsync rgb blue voce mic technology and prog audio drivers  black total
      freedom with up to  meter wireless range and lightspeed wireless audio
      transmission keep playing for up to  hours of battery life play in stereo
      on playstationr note  if the size of the earbud tips does not match the
      size of your ear canals or the headset is not worn properly in your ears
      you may not obtain the correct sound qualities or call performance change
      the earbud tips to ones that fit more snugly in your earspecific uses for
      product  gamingpersonalstreaming personalize your headset lighting across
      the full spectrum  m colors play in colors with frontfacing dualzone
      lightsync rgb lighting and choose from preset animations or create your
      own with g hub software colorful reversible suspension headbands are
      designed for comfort during long play sessions advanced mic filters that
      make your voice sound richer cleaner and more professional customize with
      g hub and find your sound hear every audio cue with breathtaking clarity
      and get immerse in your game prog drivers are designed to significantly
      reduce distortion and reproduce precise consistent rich sound quality soft
      duallayer memory foam that conforms to your head and reduces stress points
      for longlasting comfort console compatible playstation  playstation 
      nintendo switch in dock stereo sound only via usb wireless dongle
      frequency response  hz   khz
    sentences:
      - 'levis kids pajamas '
      - >-
        bright dream placemats washable easy to clean pvc placemat for kitchen
        table heatresistand woven vinyl hard table mats x inches set of  blue
        placemats  pvc and  polyesterx inch xcm dont put tomato juice on the
        placemats its hard to clean uptemperature should not exceed  washable
        and easy to cleanhand wash in warm soapy water and wipe dryif there is
        stains stuch in the placematuse a brush and soapy water to clean it no
        machine wash ecofriendly kitchen
        accessorieswashablenonfadingnonstainwipe clean protect your table from
        scratches and stains liquid can go through placemats clean it when
        finishedperfect additon to your dinner tablebeautiful stylish placemats
        to addmore fun to your kitchen table
      - >-
        outdoor portable volleyball net system  adjustable height poles with
        soft volleyball ball pump hammer boundary line and carry bag for
        backyard beach lawn    for this volleyball net set easy conversion for
        the height design  for men height  for coplay height and  for women
        height standard volleyball net          sturdy nylon net with strengthen
        polyester ropes knit together this net could    durable  inch piece
        steel poles with black powdercoating for      the total height is
        official  instant set up designed for easy use the     makes each
        assembly step very simple set up this set just in  minutes  volleyball
        set for backyardlawnbeach this set includes standard volleyball net
        steel poles official size and weight volleyball balls and inflation pump
        with needle boundary line and a carry bag patent pending
  - source_sentence: >-
      king  queen apron set  wedding engagement gifts for couples funny kitchen
      gifts for couples husband wife boyfriend girlfriend him her bridal shower
      gifts unique valentines day gifts ideas perfect gift for couples 
      searching for a couples gift that is guaranteed to put glowing smiles on
      the faces of that special pair look no further whats better than give the
      couple a great matching apron of king  queen to help them spending the
      happy cooking time together beautiful gift for couples parents friends
      fantastic gift idea for fathers day mothers day birthday valentines day
      anniversaries weddings or christmas superior quality  practical  couples
      aprons are made of denselywoven polyester soft breathable fabric to be
      used and enjoyed for years to come designed with adjustable neck  waist
      straps comfortably fit all body shapes  sizes these aprons dont just look
      cute they add extra fun to your daily routine  quality time with your
      partner just got way better premium quality design and printing  carefully
      crafted for those who appreciate distinctive elegance and quality our peel
      proof graphics are machine washable our aprons are built to last and
      waterproof and deformable these matching aprons make a perfect idea for
      valentines day gifts christmas gifts thanksgiving gifts birthday gifts
      couple gifts engagement wedding anniversary gifts or housewarming gifts
      gifts for couples who have everything  king and queen apron gift set is
      made of denselywoven polyester fabrics do waterproof and stainproof it can
      be worn while doing housework gardening painting and baking our king and
      queen matching aprons come packaged in a beautiful box ready to be gifted
      want to be the hit of the party add to cart hasslefree guarantee  never
      worry about not having a suitable choice for engagement wedding
      anniversary gifts valentines day gifts christmas gifts housewarming gifts 
      king  queen aprons gift set is perfect couples gifts your girlfriend
      boyfriend wife husband partner parents best friend and family members will
      have a great laugh we take pride in our king  queen aprons with pockets
      and back these up with a lifetime guarantee
    sentences:
      - >-
        maven thread workout headbands for women wide headbands for women sports
        running headband for exercise gym hairband athletic workouts and yoga 
        thick non slip sweatbands for women pack fashionable  functional these
        headbands for women make the perfect accessory for a killer workout
        outfit or for adorable casual wear whether youre doing yoga playing
        sports running hitting the gym or simply lounging at home our headband
        will keep your hair out of the way and keep you looking fresh nomess
        sweatband a girls gotta sweat and when she does our thick headband
        accessories are ready for it this stretchy band features nonslip grip
        and moisturewicking material so no matter the sport youre playing or the
        exercise youre doing you can give it your all this hairband will wrap
        snugly around your head but wont be too tight cute designs our headbands
        come in a variety of cute modern designs that both girls and women love
        whether youre looking for an accessory to match your favorite sweats or
        a headband to spruce up your ponytail were sure to have a look youll
        love this basics set features  darkgray headband  lightgray headband 
        navy headband and  black headband style for any wear whether youre a
        nurse trying to keep her hair pulled back at work a runner pounding the
        pavement a biker hitting the trails or just someone looking to add a
        boho or athletic touch to your outfit these wraps are for you the cloth
        will keep thin straight curly or even thick hair back if youre going to
        get sweaty toss an extra headband in your purse comfort  quality
        combined at maven thread we take pride in creating highquality products
        that are both stylish and durable our headbands are made of comfortable
        fabric with of blend of  polyester  rayon and  spandex this fabric gives
        our bands fourway stretch for onesizefitsall sizing we also offer
        sweatbands in  widths  and 
      - >-
        disney minnie mouse girls short sleeve tshirt and shorts set for
        toddlers and big kids 
      - 'gymboree boys and toddler linen chino shorts '
  - source_sentence: >-
      ntbay flannel fuzzy toddler blanket fluffy warm and lightweight reversible
      stripes design baby plush blanket x inches pink flannel package include  x
      toddler blanket  x  inches product features our cozy baby blanket is made
      of flannel fabric which is skin friendly and soft the lightweight toddler
      blanket is not so thick but it can keep your baby warm and does not feel
      the pressure of the blanket weight product function the size of the baby
      blanket is suitable and lightweight which means that you can easily fold
      it up and pack it in bags while traveling our baby blanket can be a
      multipurpose for baby stroller covers car seat covers and all seasons
      applicable easy care machine wash cold gentle cycle only nonchlorine
      bleach when needed tumble dry low cool iron if needed satisfactory sales
      service we are so confident in the quality of our product that we offer 
      days return and replacement service we also provide it with a life time
      free customer service if you find that our products have quality problems
      just contact us and we will solve the problem for you immediately 
    sentences:
      - >-
        neutrogena original fragrancefree facial cleansing bar with glycerin
        pure  transparent gentle face wash bar soap free of harsh detergents
        dyes  hardeners  oz ounce neutrogena facial cleansing bar in original
        fragrancefree formula for gentle daily facial cleansing from a
        dermatologistrecommended skin care brand this pure cleansing bar is
        designed for all skin types and lathers up to remove excess oil and
        gently cleanse skin featuring the original neutrogena formula the face
        soap bar is glycerinrich to attract moisture and contains no harsh
        detergents dyes or hardeners that can irritate the skin our face
        cleansing bar is exceptionally cleanrinsing without leaving behind any
        poreclogging residue for clear healthierlooking skin the pure and gentle
        formula of this facial soap bar is fragrancefree hypoallergenic and
        suitable for daily use
      - 'french toast girls big vneck jumper blue gold plaid  '
      - >-
        philips norelco shaver  rechargeable cordless electric shaver with popup
        trimmer x with comfortcut blades you get a clean shave thats comfortable
        on your skin rounded caps shield  selfsharpening blades to gently cut
        hair just above skin level and help the shaver glide smoothly over your
        skin d flex heads move independently in  directions to automatically
        adjust to the curves of your face neck and jaw line popup trimmer for
        mustache and sideburns finish your look with the builtin trimmer its
        ideal for maintaining your mustache and trimming your sideburns at the
        touch of a button you can pop the heads open for an easy and thorough
        clean under the tap up to  minutes of cordless runtime  thats about 
        shaves or plug it in for instant continuous power
  - source_sentence: >-
      magic the gathering modern horizons  bundle gift edition  deluxe bundle
      with  collector booster  play boosters fullart lands  exclusive
      accessories give the gift of magicthe perfect present for any magic fan
      this gift bundle is full of exclusive accessories and the coolest cards
      from modern horizons  including a collector booster packed with rare cards
      shiny foils and altart altframe cards moderns never been more
      marvelousintroducing a heaping helping of exciting cards for modern one of
      magics most celebrated formats plus the return of competitive favorites
      theres something for every fan to love in modern horizons  a collector
      booster full of treasuresboth powerful and flashy the included collector
      booster is packed with only the coolest cards and is the only place fans
      may find foiletched cards textured foil cards or serialized cards printed
      with a unique number boosters for playing  building decksplay boosters are
      the best packs for playing magic with friends and are fun to open with a
      possibility of multiple rares and at least  shining foil card in every
      pack fullart lands  special alternateart cardevery mh bundle gift edition
      also comes with  traditional foil promo card featuring bundleexclusive
      alternate art and  of the  included land cards feature stunning fullart 
      foil  nonfoil exclusive accessorieseach gift bundle also comes with
      exclusive accessories including a special die to track your life total as
      you play and a sturdy box to store everything in bundle gift edition
      contents collector booster  play boosters  traditional foil altart card 
      land cards  foil  nonfoil  spindown life counter  card storage box and 
      reference cards
    sentences:
      - >-
        dove men  care  in  bar cleanser for body face and shaving extra fresh
        body and facial cleanser more moisturizing than bar soap to clean and
        hydrate skin  ounce pack of  hydrate your skin unlike other regular bar
        soap that can dry out skin dove mencare extra fresh body and face bar
        deeply cleans and hydrates skin leaving skin feeling smooth and
        comfortable from the  dermatologist recommended mens brand this face and
        body bar is formulated with more moisturizer than ordinary soap bar and
        washes off easily to leave skin feeling healthy  in  for bar cleanser
        for body face and shaving dove mencare extra fresh body and face bar
        formulation is so hydrating and gentle on skin that you can also use it
        to wash your face or to shave refreshing clean feel clean and
        invigorated with this hydrating body cleanser and face bar and enjoy its
        crisp refreshing scent of mandarin citrus patchouli and rosemary
        designed for men mens skin care is different men sweat more have thicker
        skin and lose hydration faster dove mencare is specifically developed
        for mens skin to protect it against skin dryness pledge for paternity
        from the  dermatologist recommended mens brand were committed to making
        paid paternity leave the new standard
      - >-
        backdrop stand xft photo video studio adjustable backdrop stand for
        parties wedding photography advertising display note balloons colorful
        decorations are not included meaningful family gathering cheerful and
        memorable birthday party romantic and warm wedding professional
        photography attractive advertising display and other uses just get  ft
        backdrop stands adjustable stable backdrop stand loading capacity up to
        lbkg good flexibility with width ftft  height  ft ft heavy duty spring
        clamp and backdrop elastic string clip to holds curtains canvas muslin
        projector screen or seamless paper prevent background slippage
        photography weight sandbag to stabilize the backdrop support system note
        this bag comes empty its capacity is about lbub the kit includes  pcs
        crossbar parts   pcs tripod bracket   pcs spring clamp   pcs backdrop
        elastic string clip   pcs sand bag   pc carry bag
      - >-
        burts bees baby baby girls mittens noscratch mitts  organic cotton set
        of  

SentenceTransformer based on avsolatorio/GIST-Embedding-v0

This is a sentence-transformers model finetuned from avsolatorio/GIST-Embedding-v0. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: avsolatorio/GIST-Embedding-v0
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("wjunwei/ecommerce_text_embedding")
# Run inference
sentences = [
    'magic the gathering modern horizons  bundle gift edition  deluxe bundle with  collector booster  play boosters fullart lands  exclusive accessories give the gift of magicthe perfect present for any magic fan this gift bundle is full of exclusive accessories and the coolest cards from modern horizons  including a collector booster packed with rare cards shiny foils and altart altframe cards moderns never been more marvelousintroducing a heaping helping of exciting cards for modern one of magics most celebrated formats plus the return of competitive favorites theres something for every fan to love in modern horizons  a collector booster full of treasuresboth powerful and flashy the included collector booster is packed with only the coolest cards and is the only place fans may find foiletched cards textured foil cards or serialized cards printed with a unique number boosters for playing  building decksplay boosters are the best packs for playing magic with friends and are fun to open with a possibility of multiple rares and at least  shining foil card in every pack fullart lands  special alternateart cardevery mh bundle gift edition also comes with  traditional foil promo card featuring bundleexclusive alternate art and  of the  included land cards feature stunning fullart  foil  nonfoil exclusive accessorieseach gift bundle also comes with exclusive accessories including a special die to track your life total as you play and a sturdy box to store everything in bundle gift edition contents collector booster  play boosters  traditional foil altart card  land cards  foil  nonfoil  spindown life counter  card storage box and  reference cards',
    'backdrop stand xft photo video studio adjustable backdrop stand for parties wedding photography advertising display note balloons colorful decorations are not included meaningful family gathering cheerful and memorable birthday party romantic and warm wedding professional photography attractive advertising display and other uses just get  ft backdrop stands adjustable stable backdrop stand loading capacity up to lbkg good flexibility with width ftft  height  ft ft heavy duty spring clamp and backdrop elastic string clip to holds curtains canvas muslin projector screen or seamless paper prevent background slippage photography weight sandbag to stabilize the backdrop support system note this bag comes empty its capacity is about lbub the kit includes  pcs crossbar parts   pcs tripod bracket   pcs spring clamp   pcs backdrop elastic string clip   pcs sand bag   pc carry bag',
    'burts bees baby baby girls mittens noscratch mitts  organic cotton set of  ',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 5,864 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	int
  details	min: 5 tokens mean: 144.3 tokens max: 512 tokens	min: 5 tokens mean: 143.08 tokens max: 512 tokens	0: ~87.50% 1: ~12.50%

• Samples:

  sentence_0	sentence_1	label
  philips norelco shaver rechargeable electric shaver with popup trimmer s comfortcut blades get a clean shave thats comfortable on your skin rounded blade caps shield selfsharpening blades to gently cut hair just above skin level and help the shaver glide smoothly over your skin experience a convenient clean shave with heads that flex and float in directions the head adjusts to the curves of your face ensuring smooth contact with your skin without a lot of pressure d flex heads follow your faces contours for a clean shave popup trimmer for mustache and sideburns finish your look with the builtin trimmer ideal for maintaining your mustache and trimming your sideburns onetouch open for easy cleaning experience a convenient clean shave with heads that flex and float in directions the head adjusts to the curves of your face ensuring smooth contact with your skin without a lot of pressure minutes of cordless shaving from an hour charge thats about shaves or plug it in for instant continuous power	philips norelco shaver rechargeable electric shaver with popup trimmer s comfortcut blades get a clean shave thats comfortable on your skin rounded blade caps shield selfsharpening blades to gently cut hair just above skin level and help the shaver glide smoothly over your skin experience a convenient clean shave with heads that flex and float in directions the head adjusts to the curves of your face ensuring smooth contact with your skin without a lot of pressure d flex heads follow your faces contours for a clean shave popup trimmer for mustache and sideburns finish your look with the builtin trimmer ideal for maintaining your mustache and trimming your sideburns onetouch open for easy cleaning experience a convenient clean shave with heads that flex and float in directions the head adjusts to the curves of your face ensuring smooth contact with your skin without a lot of pressure minutes of cordless shaving from an hour charge thats about shaves or plug it in for instant continuous power	1
  speedo girls swimsuit one piece thin straps	lace tulle flower girl long dress for wedding oneck princess dresses long sleeve pageant party gown	0
  pyrex blue cup rectangular plastic cover pc pack original genuine pyrex made in the usa genuine pyrex replacement lid nonporous surface does not absorb food odors flavors or stains refrigerator microwave and toprack dishwasher safe will not fit anchor hocking products included pyrex pc blue cup rectangle plastic lids lids only containers not included	kpywzer vintage leather sling bag bags for men women backpack shoulder messenger crossbody outdoor travel hiking camping tactical chest pack daypack brown	0

• Loss: ContrastiveTensionLoss

Training Hyperparameters

Non-Default Hyperparameters

• num_train_epochs: 5
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
0.6821	500	7.4403
1.3643	1000	4.8536
2.0464	1500	3.8646
2.7285	2000	3.8877
3.4106	2500	3.8789
4.0928	3000	3.8052
4.7749	3500	3.8385

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.0.1
• Transformers: 4.41.2
• PyTorch: 2.3.0+cu121
• Accelerate: 0.31.0
• Datasets: 2.20.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveTensionLoss

@inproceedings{carlsson2021semantic,
    title={Semantic Re-tuning with Contrastive Tension},
    author={Fredrik Carlsson and Amaru Cuba Gyllensten and Evangelia Gogoulou and Erik Ylip{"a}{"a} Hellqvist and Magnus Sahlgren},
    booktitle={International Conference on Learning Representations},
    year={2021},
    url={https://openreview.net/forum?id=Ov_sMNau-PF}
}