metadata

language:
  - en
license: apache-2.0
library_name: sentence-transformers
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:900
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
datasets: []
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
widget:
  - source_sentence: >-
      Vendor Risk Assessment View Breach Management View Privacy Policy
      Management View Privacy Center View Learn more Security Identify data risk
      and enable protection & control Data Security Posture Management View Data
      Access Intelligence & Governance View Data Risk Management View Data
      Breach Analysis View Learn more Governance Optimize Data Governance with
      granular insights into your data Data Catalog View Data Lineage View Data
      Quality View Data Controls Orchestrator View Solutions Technologies
      Covering you everywhere with 1000+ integrations across data systems.
      Snowflake View AWS View Microsoft 365 View Salesforce View Workday View
      GCP View Azure View Oracle View Learn more Regulations Automate compliance
      with global privacy regulations. US California CCPA View US California
      CPRA View European Union GDPR View Thailand’s PDPA View China PIPL View
      Canada PIPEDA View Brazil's LGPD View \+ More View Learn more Roles
      Identify data risk and enable protection & control. Privacy View Security
      View Governance View Marketing View Resources Blog Read through our
      articles written by industry experts Collateral Product brochures, white
      papers, infographics, analyst reports and more. Knowledge Center Learn
      about the data privacy, security and governance landscape. Securiti
      Education Courses and Certifications for data privacy, security and
      governance professionals. Company About Us Learn all about Securiti, our
      mission and history Partner Program Join our Partner Program Contact Us
      Contact us to learn more or schedule a demo News Coverage Read about
      Securiti in the news Press Releases Find our latest press releases Careers
      Join the
    sentences:
      - >-
        What is the purpose of tracking changes and transformations of data
        throughout its lifecycle?
      - >-
        What is the role of ePD in the European privacy regime and its relation
        to GDPR?
      - How can data governance be optimized using granular insights?
  - source_sentence: >-
      Learn more Asset and Data Discovery Discover dark and native data assets
      Learn more Data Access Intelligence & Governance Identify which users have
      access to sensitive data and prevent unauthorized access Learn more Data
      Privacy Automation PrivacyCenter.Cloud | Data Mapping | DSR Automation |
      Assessment Automation | Vendor Assessment | Breach Management | Privacy
      Notice Learn more Sensitive Data Intelligence Discover & Classify
      Structured and Unstructured Data | People Data Graph Learn more Data Flow
      Intelligence & Governance Prevent sensitive data sprawl through real-time
      streaming platforms Learn more Data Consent Automation First Party Consent
      | Third Party & Cookie Consent Learn more Data Security Posture Management
      Secure sensitive data in hybrid multicloud and SaaS environments Learn
      more Data Breach Impact Analysis & Response Analyze impact of a data
      breach and coordinate response per global regulatory obligations Learn
      more Data Catalog Automatically catalog datasets and enable users to find,
      understand, trust and access data Learn more Data Lineage Track changes
      and transformations of data throughout its lifecycle Data Controls
      Orchestrator View Data Command Center View Sensitive Data Intelligence
      View Asset Discovery Data Discovery & Classification Sensitive Data
      Catalog People Data Graph Learn more Privacy Automate compliance with
      global privacy regulations Data Mapping Automation View Data Subject
      Request Automation View People Data Graph View Assessment Automation View
      Cookie Consent View Universal Consent View Vendor Risk Assessment View
      Breach Management View Privacy Policy Management View Privacy Center View
      Learn more Security Identify data risk and enable protection & control
      Data Security Posture Management View Data Access Intelligence &
      Governance View Data Risk Management View Data Breach Analysis View Learn
      more Governance Optimize Data Governance with granular insights into your
      data Data Catalog View Data Lineage View Data Quality View Data Controls
      Orchestrator ,  View Learn more Asset and Data Discovery Discover dark and
      native data assets Learn more Data Access Intelligence & Governance
      Identify which users have access to sensitive data and prevent
      unauthorized access Learn more Data Privacy Automation PrivacyCenter.Cloud
      | Data Mapping | DSR Automation | Assessment Automation | Vendor
      Assessment | Breach Management | Privacy Notice Learn more Sensitive Data
      Intelligence Discover & Classify Structured and Unstructured Data | People
      Data Graph Learn more Data Flow Intelligence & Governance Prevent
      sensitive data sprawl through real-time streaming platforms Learn more
      Data Consent Automation First Party Consent | Third Party & Cookie Consent
      Learn more Data Security Posture Management Secure sensitive data in
      hybrid multicloud and SaaS environments Learn more Data Breach Impact
      Analysis & Response Analyze impact of a data breach and coordinate
      response per global regulatory obligations Learn more Data Catalog
      Automatically catalog datasets and enable users to find, understand, trust
      and access data Learn more Data Lineage Track changes and transformations
      of data throughout its lifecycle Data Controls Orchestrator View Data
      Command Center View Sensitive Data Intelligence View Asset Discovery Data
      Discovery & Classification Sensitive Data Catalog People Data Graph Learn
      more Privacy Automate compliance with global privacy regulations Data
      Mapping Automation View Data Subject Request Automation View People Data
      Graph View Assessment Automation View Cookie Consent View Universal
      Consent View Vendor Risk Assessment View Breach Management View Privacy
      Policy Management View Privacy Center View Learn more Security Identify
      data risk and enable protection & control Data Security Posture Management
      View Data Access Intelligence & Governance View Data Risk Management View
      Data Breach Analysis View Learn more Governance Optimize Data Governance
      with granular insights into your data Data Catalog View Data Lineage View
      Data Quality View Data Controls
    sentences:
      - >-
        What is the purpose of Asset and Data Discovery in data governance and
        security?
      - Which EU member states have strict cyber laws?
      - >-
        What is the obligation for organizations to provide Data Protection
        Impact Assessments (DPIAs) under the LGPD?
  - source_sentence: >-
      which the data is processed. **Right to Access:** Data subjects have the
      right to obtain confirmation whether or not the controller holds personal
      data about them, access their personal data, and obtain descriptions of
      data recipients. **Right to Rectification** : Under the right to
      rectification, data subjects can request the correction of their data.
      **Right to Erasure:** Data subjects have the right to request the erasure
      and destruction of the data that is no longer needed by the organization.
      **Right to Object:** The data subject has the right to prevent the data
      controller from processing personal data if such processing causes or is
      likely to cause unwarranted damage or distress to the data subject.
      **Right not to be Subjected to Automated Decision-Making** : The data
      subject has the right to not be subject to automated decision-making that
      significantly affects the individual. ## Facts related to Ghana’s Data
      Protection Act 2012 1 While processing personal data, organizations must
      comply with eight privacy principles: lawfulness of processing, data
      quality, security measures, accountability, purpose specification, purpose
      limitation, openness, and data subject participation. 2 In the event of a
      security breach, the data controller shall take measures to prevent the
      breach and notify the Commission and the data subject about the breach as
      soon as reasonably practicable after the discovery of the breach. 3 The
      DPA specifies lawful grounds for data processing, including data subject’s
      consent, the performance of a contract, the interest of data subject and
      public interest, lawful obligations, and the legitimate interest of the
      data controller. 4 The DPA requires data controllers to register with the
      Data Protection Commission (DPC). 5 The DPA provides varying fines and
      terms of imprisonment according to the severity and sensitivity of the
      violation, such as any person who sells personal data may get fined up to
      2500 penalty units or up to five years imprisonment or both. ### Forrester
      Names Securiti a Leader in the Privacy Management Wave Q4, 2021 Read the
      Report ### Securiti named a Leader in the IDC MarketScape for Data Privacy
      Compliance Software Read the Report At Securiti, our mission is to enable
      enterprises to safely harness the incredible power of data and the cloud
      by controlling the complex security, privacy and compliance risks.
      Copyright (C) 2023 Securiti Sitem
    sentences:
      - >-
        What information is required for data subjects regarding data transfers
        under the GDPR, including personal data categories, data recipients,
        retention period, and automated decision making?
      - >-
        What privacy principles must organizations follow when processing
        personal data under Ghana's Data Protection Act 2012?
      - What is the purpose of Thailand's PDPA?
  - source_sentence: >-
      consumer has the right to have his/her personal data stored or processed
      by the data controller be deleted. ## Portability The consumer has a right
      to obtain a copy of his/her personal data in a portable, technically
      feasible and readily usable format that allows the consumer to transmit
      the data to another controller without hindrance. ## Opt out The consumer
      has the right to opt out of the processing of the personal data for
      purposes of targeted advertising, the sale of personal data, or profiling
      in furtherance of decisions that produce legal or similarly significant
      effects concerning the consumer. **Time period to fulfill DSR request: **
      All data subject rights’ requests (DSR requests) must be fulfilled by the
      data controller within a 45 day period. **Extension in time period: **
      data controllers may seek for an extension of 45 days in fulfilling the
      request depending on the complexity and number of the consumer's requests.
      **Denial of DSR request: ** If a DSR request is to be denied, the data
      controller must inform the consumer of the reasons within a 45 days
      period. **Appeal against refusal: ** Consumers have a right to appeal the
      decision for refusal of grant of the DSR request. The appeal must be
      decided within 45 days but the time period can be further extended by 60
      additional days. **Limitation of DSR requests per year: ** Requests for
      data portability may be made only twice in a year. **Charges: ** DSR
      requests must be fulfilled free of charge once in a year. Any subsequent
      request within a 12 month period can be charged. **Authentication: ** A
      data controller is not to respond to a consumer request unless it can
      authenticate the request using reasonably commercial means. A data
      controller can request additional information from the consumer for the
      purposes of authenticating the request. ## Who must comply? CPA applies to
      all data controllers who conduct business in Colorado or produce or
      deliver commercial products or services that are intentionally targeted to
      residents of Colorado if they match any one or both of these conditions:
      If they control or process the personal data of 100,000 consumers or more
      during a calendar year; or If they derive revenue or receive a discount on
      the price of goods or services from the sale of personal data and process
      or control the personal data of 25,000
    sentences:
      - >-
        What is the US California CCPA and how does it relate to data privacy
        regulations?
      - >-
        What does the People Data Graph serve in terms of privacy, security, and
        governance?
      - >-
        What rights does a consumer have regarding the portability of their
        personal data?
  - source_sentence: >-
      PR and Federal Data Protection Act within Germany; To promote awareness
      within the public related to the risks, rules, safeguards, and rights
      concerning the processing of personal data; To handle all complaints
      raised by data subjects related to data processing in addition to carrying
      out investigations to find out if any data handler has breached any
      provisions of the Act; ## Penalties for Non compliance The GDPR already
      laid down some stringent penalties for companies that would be found in
      breach of the law's provisions. More importantly, as opposed to other data
      protection laws such as the CCPA and CPRA, non-compliance with the law
      also meant penalties. Germany's Federal Data Protection Act has a slightly
      more lenient take in this regard. Suppose a data handler is found to have
      fraudulently collected data, processed, shared, or sold data without
      proper consent from the data subjects, not responded or responded with
      delay to a data subject request, or failed to inform the data subject of a
      breach properly. In that case, it can be fined up to €50,000. This is in
      addition to the GDPR's €20 million or 4% of the total worldwide annual
      turnover of the preceding financial year, whichever is higher, that any
      organisation found in breach of the law is subject to. However, for this
      fine to be applied, either the data subject, the Federal Commissioner, or
      the regulatory authority must file an official complaint. ## How an
      Organization Can Operationalize the Law Data handlers processing data
      inside Germany can remain compliant with the country's data protection law
      if they fulfill the following conditions: Have a comprehensive privacy
      policy that educates all users of their rights and how to contact the
      relevant personnel within the organisation in case of a query Hire a
      competent Data Protection Officer that understands the GDPR and Federal
      Data Protection Act thoroughly and can lead compliance efforts within your
      organisation Ensure all the company's employees and staff are acutely
      aware of their responsibilities under the law Conduct regular data
      protection impact assessments as well as data mapping exercises to ensure
      maximum efficiency in your compliance efforts Notify the relevant
      authorities of a data breach as soon as possible ## How can Securiti Help
      Data privacy and compliance have become incredibly vital in earning users'
      trust globally. Most users now expect most businesses to take all the
      relevant measures to ensure the data they collect is properly stored,
      protected, and maintained. Data protection laws have made such efforts
      legally mandatory
    sentences:
      - >-
        What are the benefits of automating compliance with global privacy
        regulations for data protection and control?
      - >-
        What is required for an official complaint to be filed under Germany's
        Federal Data Protection Act?
      - Why is tracking data lineage important for data management and security?
pipeline_tag: sentence-similarity
model-index:
  - name: SentenceTransformer based on BAAI/bge-base-en-v1.5
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.08
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.31
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.47
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.65
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.08
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.10333333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.09399999999999999
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06499999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.08
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.31
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.47
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.65
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3343233273884531
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.2366031746031746
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.24981059879972897
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.09
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.29
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.46
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.65
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.09
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.09666666666666668
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.092
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06499999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.09
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.29
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.46
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.65
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3342796810716671
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.2370753968253968
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2495249393048939
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.08
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.28
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.43
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.08
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.09333333333333334
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.08599999999999998
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.059999999999999984
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.08
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.28
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.43
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.6
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3082112269933052
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.21817460317460313
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2329761521137356
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.05
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.17
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.36
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.53
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.05
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.056666666666666664
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.07200000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.05299999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.05
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.17
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.36
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.53
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.24965377482070814
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.1642142857142857
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.18144130849038587
            name: Cosine Map@100

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-base-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("MugheesAwan11/bge-base-securiti-dataset-1-v9")
# Run inference
sentences = [
    "PR and Federal Data Protection Act within Germany; To promote awareness within the public related to the risks, rules, safeguards, and rights concerning the processing of personal data; To handle all complaints raised by data subjects related to data processing in addition to carrying out investigations to find out if any data handler has breached any provisions of the Act; ## Penalties for Non compliance The GDPR already laid down some stringent penalties for companies that would be found in breach of the law's provisions. More importantly, as opposed to other data protection laws such as the CCPA and CPRA, non-compliance with the law also meant penalties. Germany's Federal Data Protection Act has a slightly more lenient take in this regard. Suppose a data handler is found to have fraudulently collected data, processed, shared, or sold data without proper consent from the data subjects, not responded or responded with delay to a data subject request, or failed to inform the data subject of a breach properly. In that case, it can be fined up to €50,000. This is in addition to the GDPR's €20 million or 4% of the total worldwide annual turnover of the preceding financial year, whichever is higher, that any organisation found in breach of the law is subject to. However, for this fine to be applied, either the data subject, the Federal Commissioner, or the regulatory authority must file an official complaint. ## How an Organization Can Operationalize the Law Data handlers processing data inside Germany can remain compliant with the country's data protection law if they fulfill the following conditions: Have a comprehensive privacy policy that educates all users of their rights and how to contact the relevant personnel within the organisation in case of a query Hire a competent Data Protection Officer that understands the GDPR and Federal Data Protection Act thoroughly and can lead compliance efforts within your organisation Ensure all the company's employees and staff are acutely aware of their responsibilities under the law Conduct regular data protection impact assessments as well as data mapping exercises to ensure maximum efficiency in your compliance efforts Notify the relevant authorities of a data breach as soon as possible ## How can Securiti Help Data privacy and compliance have become incredibly vital in earning users' trust globally. Most users now expect most businesses to take all the relevant measures to ensure the data they collect is properly stored, protected, and maintained. Data protection laws have made such efforts legally mandatory",
    "What is required for an official complaint to be filed under Germany's Federal Data Protection Act?",
    'Why is tracking data lineage important for data management and security?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.08
cosine_accuracy@3	0.31
cosine_accuracy@5	0.47
cosine_accuracy@10	0.65
cosine_precision@1	0.08
cosine_precision@3	0.1033
cosine_precision@5	0.094
cosine_precision@10	0.065
cosine_recall@1	0.08
cosine_recall@3	0.31
cosine_recall@5	0.47
cosine_recall@10	0.65
cosine_ndcg@10	0.3343
cosine_mrr@10	0.2366
cosine_map@100	0.2498

Information Retrieval

Dataset: dim_256
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.09
cosine_accuracy@3	0.29
cosine_accuracy@5	0.46
cosine_accuracy@10	0.65
cosine_precision@1	0.09
cosine_precision@3	0.0967
cosine_precision@5	0.092
cosine_precision@10	0.065
cosine_recall@1	0.09
cosine_recall@3	0.29
cosine_recall@5	0.46
cosine_recall@10	0.65
cosine_ndcg@10	0.3343
cosine_mrr@10	0.2371
cosine_map@100	0.2495

Information Retrieval

Dataset: dim_128
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.08
cosine_accuracy@3	0.28
cosine_accuracy@5	0.43
cosine_accuracy@10	0.6
cosine_precision@1	0.08
cosine_precision@3	0.0933
cosine_precision@5	0.086
cosine_precision@10	0.06
cosine_recall@1	0.08
cosine_recall@3	0.28
cosine_recall@5	0.43
cosine_recall@10	0.6
cosine_ndcg@10	0.3082
cosine_mrr@10	0.2182
cosine_map@100	0.233

Information Retrieval

Dataset: dim_64
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.05
cosine_accuracy@3	0.17
cosine_accuracy@5	0.36
cosine_accuracy@10	0.53
cosine_precision@1	0.05
cosine_precision@3	0.0567
cosine_precision@5	0.072
cosine_precision@10	0.053
cosine_recall@1	0.05
cosine_recall@3	0.17
cosine_recall@5	0.36
cosine_recall@10	0.53
cosine_ndcg@10	0.2497
cosine_mrr@10	0.1642
cosine_map@100	0.1814

Training Details

Training Dataset

Unnamed Dataset

Size: 900 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 159 tokens
mean: 445.26 tokens
max: 512 tokens

min: 7 tokens
mean: 22.05 tokens
max: 82 tokens

	positive	anchor
type	string	string
details	min: 159 tokens mean: 445.26 tokens max: 512 tokens	min: 7 tokens mean: 22.05 tokens max: 82 tokens

Samples:

positive	anchor
orra The Andorra personal data protection act came into force on May 17, 2022, by the Andorra Data Protection Authority (ADPA). Learn more about Andorra PDPA ### United Kingdom The UK Data Protection Act (DPA) 2018 is the amended version of the Data Protection Act that was passed in 1998. The DPA 2018 implements the GDPR with several additions and restrictions. Learn more about UK DPA ### Botswana The Botswana Data Protection came into effect on October 15, 2021 after the issuance of the Data Protection Act (Commencement Date) Order 2021 by the Minister of Presidential Affairs, Governance and Public Administration. Learn more about Botswana DPA ### Zambia On March 31, 2021, the Zambian parliament formally passed the Data Protection Act No. 3 of 2021 and the Electronic Communications and Transactions Act No. 4 of 2021. Learn more about Zambia DPA ### Jamaica On November 30, 2020, the First Schedule of the Data Protection Act No. 7 of 2020 came into effect following the publication of Supplement No. 160 of Volume CXLIV in the Jamaica Gazette Supplement. Learn more about Jamaica DPA ### Belarus The Law on Personal Data Protection of May 7, 2021, No. 99-Z, entered into effect within Belarus on November 15, 2021. Learn more about Belarus DPA ### Russian Federation The primary Russian law on data protection, Federal Law No. 152-FZ has been in effect since July 2006. Learn more ### Eswatini On March 4, 2022, the Eswatini Communications Commission published the Data Protection Act No. 5 of 2022, simultaneously announcing its immediate enforcement. Learn more ### Oman The Royal Decree 6/2022 promulgating the Personal Data Protection Law (PDPL) was passed on February 9, 2022. Learn more ### Sri Lanka Sri Lanka's parliament formally passed the Personal Data Protection Act (PDPA), No. 9 Of 2022, on March 19, 2022. Learn more ### Kuwait Kuwait's DPPR was formally introduced by the CITRA to ensure the Gulf country's data privacy infrastructure. Learn more ### Brunei Darussalam The draft Personal Data Protection Order is Brunei’s primary data protection law which came into effect in 2022. Learn more ### India India’	`What is the name of India's data protection law before May 17, 2022?`
the affected data subjects and regulatory authority about the breach and whether any of their information has been compromised as a result. ### Data Protection Impact Assessment There is no requirement for conducting data protection impact assessment under the PDPA. ### Record of Processing Activities A data controller must keep and maintain a record of any privacy notice, data subject request, or any other information relating to personal data processed by him in the form and manner that may be determined by the regulatory authority. ### Cross Border Data Transfer Requirements The PDPA provides that personal data can be transferred out of Malaysia only when the recipient country is specified as adequate in the Official Gazette. The personal data of data subjects can not be disclosed without the consent of the data subject. The PDPA provides the following exceptions to the cross border data transfer requirements: Where the consent of data subject is obtained for transfer; or Where the transfer is necessary for the performance of contract between the parties; The transfer is for the purpose of any legal proceedings or for the purpose of obtaining legal advice or for establishing, exercising or defending legal rights; The data user has taken all reasonable precautions and exercised all due diligence to ensure that the personal data will not in that place be processed in any manner which, if that place is Malaysia, would be a contravention of this PDPA; The transfer is necessary in order to protect the vital interests of the data subject; or The transfer is necessary as being in the public interest in circumstances as determined by the Minister. ## Data Subject Rights The data subjects or the person whose data is being collected has certain rights under the PDPA. The most prominent rights can be categorized under the following: ## Right to withdraw consent The PDPA, like some of the other landmark data protection laws such as CPRA and GDPR gives data subjects the right to revoke their consent at any time by way of written notice from having their data collected processed. ## Right to access and rectification As per this right, anyone whose data has been collected has the right to request to review their personal data and have it updated. The onus is on the data handlers to respond to such a request as soon as possible while also making it easier for data subjects on how they can request access to their personal data. ## Right to data portability Data subjects have the right to request that their data be stored in a manner where it	`What is the requirement for conducting a data protection impact assessment under the PDPA?`
more Privacy Automate compliance with global privacy regulations Data Mapping Automation View Data Subject Request Automation View People Data Graph View Assessment Automation View Cookie Consent View Universal Consent View Vendor Risk Assessment View Breach Management View Privacy Policy Management View Privacy Center View Learn more Security Identify data risk and enable protection & control Data Security Posture Management View Data Access Intelligence & Governance View Data Risk Management View Data Breach Analysis View Learn more Governance Optimize Data Governance with granular insights into your data Data Catalog View Data Lineage View Data Quality View Data Controls Orchestrator View Solutions Technologies Covering you everywhere with 1000+ integrations across data systems. Snowflake View AWS View Microsoft 365 View Salesforce View Workday View GCP View Azure View Oracle View Learn more Regulations Automate compliance with global privacy regulations. US California CCPA View US California CPRA View European Union GDPR View Thailand’s PDPA View China PIPL View Canada PIPEDA View Brazil's LGPD View + More View Learn more Roles Identify data risk and enable protection & control. Privacy View Security View Governance View Marketing View Resources Blog Read through our articles written by industry experts Collateral Product brochures, white papers, infographics, analyst reports and more. Knowledge Center Learn about the data privacy, security and governance landscape. Securiti Education Courses and Certifications for data privacy, security and governance professionals. Company About Us Learn all about	`What is Data Subject Request Automation?`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
learning_rate: 2e-05
num_train_epochs: 2
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: True
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_128_cosine_map@100	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_64_cosine_map@100
0.3448	10	7.0997	-	-	-	-
0.6897	20	5.0842	-	-	-	-
1.0	29	-	0.2367	0.2561	0.2502	0.1813
1.0345	30	4.7423	-	-	-	-
1.3793	40	3.7933	-	-	-	-
1.7241	50	3.4879	-	-	-	-
2.0	58	-	0.233	0.2495	0.2498	0.1814

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.14
Sentence Transformers: 3.0.1
Transformers: 4.41.2
PyTorch: 2.1.2+cu121
Accelerate: 0.31.0
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}