Add SetFit model

README.md

@@ -145,7 +145,7 @@ model-index:
       split: test
     metrics:
     - type: accuracy
-      value: 0.8933333333333333
+      value: 0.9066666666666666
       name: Accuracy
 ---
 
@@ -177,17 +177,17 @@ The model has been trained using an efficient few-shot learning technique that i
 - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
 
 ### Model Labels
-| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
-|:------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| 0     | <ul><li>"**Reasoning:**\n\n**Why the Answer May Be Good:**\n1. **Context Grounding:** The answer references the points made in the document, such as Coach Brian Shaw's strategy of pushing the ball after makes and misses as well as encouraging players to take the first available shot within the rhythm of the offense.\n2. **Relevance:** The answer directly addresses why the Nuggets are having an offensive outburst, highlighting the coaching strategy and players' adaptation.\n3. **Conciseness:** The answer is mostly to the point and focuses on the main question.\n\n**Why the Answer May Be Bad:**\n1. **Context Grounding:** The mention of a new training technique using virtual reality is not supported by any information within the document provided.\n2. **Conciseness:** The additional detail about the virtual reality training is unnecessary given that it is not referenced in the document and does not contribute to answering the specific question about the offensive outburst.\n   \n**Final Result:**\nBased on the evaluation criteria, the inclusion of fictitious or unsupported information about the virtual reality training significantly detracts from the answer’s credibility and relevance.\n\n****"</li><li>'Reasoning why the answer may be good:\n1. **Context Grounding:** The provided answer cites specific information about film and digital photography directly from the provided document, showing a good grounding.\n2. **Relevance:** The answer addresses the specific question by discussing different aspects such as exposure tolerance, color capture, and overall image resolution between film and digital photography.\n3. **Conciseness:** The answer is relatively concise and sticks to the main points relevant to the question without unnecessary elaboration.\n\nReasoning why the answer may be bad:\n1. **Overly Detailed:** The answer could be seen as too detailed in certain segments, which might slightly detract from conciseness.\n2. **Possible Confusion:** The mention of specific technical details like "5MP digital sensors" could confuse readers who are not familiar with the technical specifications, detracting from clarity.\n3. **Omission of Key Comparison Points:** The answer does not touch upon some of the more subjective observations made by the author, like the practical advantages in using film forcertain types of photography.\n\nFinal Result:'</li><li>'Reasoning:\n1. **Context Grounding**: The answer provided does not reference the third book of the Arcana Chronicles by Kresley Cole or even discuss any content relevant to it. Instead, it discusses an MMA event in Calgary, Alberta, Canada.\n2. **Relevance**: The answer is entirely irrelevant to the question. The question is about the main conflict in the third book of a specific book series, but the answer describes an MMA fight event.\n3. **Conciseness**: While the answer is concise in its context, it is entirely off-topic and therefore does not satisfy the conciseness criterion in a meaningful way.\n\nThe answer may be deemed bad because it does not address the question about the Arcana Chronicles at all and instead provides unrelated information about an MMA event.\n\nFinal result:'</li></ul> |
-| 1     | <ul><li>'Reasoning:\n\n1. Context Grounding:\n   - Good: The answer is supported by the document. The suggestions mentioned (getting to know the client, signing a contract, and showcasing honesty and diplomacy) are directly referenced in the text provided.\n   - Bad: There is no significant bad aspect in terms of context grounding; the answer sticks closely to the source material.\n\n2. Relevance:\n   - Good: The answer is highly relevant to the question about best practices to avoid unnecessary revisions and conflicts. It addresses client understanding, contractual agreements, and the handling of extra charges—all crucial for minimizing conflicts.\n   - Bad: There is no deviation from the topic. The answer is focused solely on the best practices, as asked in the question.\n\n3. Conciseness:\n   - Good: The answer is concise and to the point, effectively summarizing the practices without unnecessary details.\n   - Bad: The level of detail might be too succinct for some readers looking for more in-depth discussion, but this is minor given the criteria.\n\nFinal Result:'</li><li>"Reasoning for why the answer may be good:\n- The answer references the author’s emphasis on drawing from personal experiences of pain and emotion to create genuine and relatable characters, which is well-supported by the document.\n- It highlights the importance of genuineness and relatability, which aligns directly with the content provided in the document.\n- The answer stays focused on the specific question about creating a connection between the reader and the characters.\n\nReasoning for why the answer may be bad:\n- The answer could be seen as slightly verbose and might include more detail than necessary, rather than being extremely concise.\n- It does not explicitly mention the document's use of pain for romance authors specifically, which might add to the context.\n\nFinal result:"</li><li>"**Reasoning:**\n\n**Pros:**\n1. **Context Grounding:** The document explicitly states that Mauro Rubin is the CEO of JoinPad and mentions that he was speaking at the event, which directly supports the answer.\n2. **Relevance:** The answer directly and correctly responds to the question about the CEO's identity during the event.\n3. **Conciseness:** The answer is brief and to the point, providing only the necessary information.\n\n**Cons:**\n- There are no significant cons as the answer fulfillsall criteria effectively.\n\n**Final Result:**"</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+|:------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 1     | <ul><li>"Reasoning why the answer may be good:\n1. **Context Grounding**: The answer is well-supported by the provided document and directly quotes relevant information about Patricia Wallace's roles and responsibilities.\n2. **Relevance**: The answer specifically addresses the question asked, detailing the roles and responsibilities of Patricia Wallace without deviating into unrelated topics.\n3. **Conciseness**: The answer is clear, concise, and focuses on the main points relevant to the question, avoiding unnecessary information.\n\nReasoning why the answer may be bad:\n- There is no significant reason to consider the answer bad based on the given criteria. It comprehensively covers the roles and responsibilities of Patricia Wallace as mentioned in the document.\n\nFinal Result:"</li><li>'### Reasoning:\n**Why the answer may be good:**\n1. **Context Grounding:** The answer is directly taken from the document, which states that a dime is one-tenth of a dollar.\n2. **Relevance:** The answer addresses the specific question asked about the monetary value of a dime.\n3. **Conciseness:** The answer is clear and to the point, providing no more information than necessary.\n\n**Why the answer may be bad:**\n1. **Context Grounding:** The document provides additional context and details about the U.S. dollar system which were not included in the answer. However, these details are not directly necessary to answer the question.\n2. **Relevance:** No deviation or unrelated topics are present in the answer. \n3. **Conciseness:** The answer avoids unnecessary information, maintaining itsclarity and brevity. \n\n### Final Result:\n****'</li><li>'Reasoning why the answer may be good:\n- Context Grounding: The answer refers to symptoms like flu-like signs, which are detailed in the provided document. It also mentions the connection with tampon use, the presence of rashes, and the seriousness of seeking medical help, all of which are discussed in the document.\n- Relevance: The answer addresses the question by listing symptoms and highlighting the importance of recognizing them, which directly corresponds to the question asked.\n- Conciseness: The answer is relatively concise while covering most of the essential details related to recognizing TSS.\n\nReasoning why the answer may be bad:\n- Context Grounding: While the answer does mention flu-like symptoms and the association with tampon use, it lacks specific details like fever and other visible signs mentioned in the document.\n- Relevance: The mention of treatment with antibiotics is somewhat relevant but moves slightly away from the specific focus of how to recognize TSS.\n- Conciseness: The answer could be streamlined further by focusing more on the core question of identifying symptoms rather than mentioning treatment.\n\nFinal Result:'</li></ul> |
+| 0     | <ul><li>'**Reasoning:**\n\n**Why the answer may be good:**\n1. **Context Grounding:** The answer does affirm Gregory Johnson as the CEO of Franklin Templeton Investments, which is supported by the provided document.\n2. **Relevance:** The answer directly addresses the question regarding the CEO of Franklin Templeton Investments.\n3. **Conciseness:** The answer is relatively clear and to the point, providing the name of the CEO as requested.\n\n**Why the answer may be bad:**\n1. **Context Grounding:** The statement about Gregory Johnson inheriting the position from his father, Rupert H. Johnson, Sr., is not mentioned in the provided document.\n2. **Relevance:** While the primary answer is correct and relevant, the additional information about the inheritance is not relevant to the specific question asked.\n3. **Conciseness:** The answer includes unnecessary information about the inheritance of the position, which was not part of the question.\n\n**Final result:**'</li><li>'Reasoning why the answer may be good:\n1. The answer is well-supported by the provided document, mentioning key steps in diagnosis and treatment such as taking the cat to the vet, using topical antibiotics and anti-inflammatory medications, completing the full course of treatment, and isolating the infected cat.\n2. It directly addresses the specific question of how to treat conjunctivitis in cats.\n3. The answer is clear and to the point, providing practical advice on treatment.\n\nReasoning why the answer may be bad:\n1. The mention of conjunctivitis in cats often resulting from exposure to a rare type of pollen found only in the Amazon rainforest is not supported by the document. This statement is factually incorrect and detracts from the overall accuracy.\n2. It could be more concise by avoiding unnecessary information and focusing solely on the mostcritical points of treatment.\n\nFinal result:'</li><li>"Reasoning why the answer may be good: \n- The answer correctly identifies the College of Arts and Letters as Notre Dame's first college, founded in 1842, which is directly related to the question asked.\n\nReasoning why the answer may be bad:\n- The answer includes an incorrect and unsupported statement about the curriculum for time travel studies, which is not mentioned in the provided document andis irrelevant to the question.\n\nFinal result:"</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
 
 ## Evaluation
 
 ### Metrics
 | Label   | Accuracy |
 |:--------|:---------|
-| **all** | 0.8933   |
+| **all** | 0.9067   |
 
 ## Uses
 
@@ -244,12 +244,12 @@ preds = model("**Good**
 ### Training Set Metrics
 | Training set | Min | Median   | Max |
 |:-------------|:----|:---------|:----|
-| Word count   | 50  | 124.8592 | 199 |
+| Word count   | 50  | 125.2071 | 274 |
 
 | Label | Training Sample Count |
 |:------|:----------------------|
-| 0     | 34                    |
-| 1     | 37                    |
+| 0     | 95                    |
+| 1     | 103                   |
 
 ### Training Hyperparameters
 - batch_size: (16, 16)
@@ -273,10 +273,16 @@ preds = model("**Good**
 ### Training Results
 | Epoch  | Step | Training Loss | Validation Loss |
 |:------:|:----:|:-------------:|:---------------:|
-| 0.0056 | 1    | 0.1968        | -               |
-| 0.2809 | 50   | 0.2558        | -               |
-| 0.5618 | 100  | 0.2212        | -               |
-| 0.8427 | 150  | 0.0417        | -               |
+| 0.0020 | 1    | 0.1499        | -               |
+| 0.1010 | 50   | 0.2586        | -               |
+| 0.2020 | 100  | 0.2524        | -               |
+| 0.3030 | 150  | 0.1409        | -               |
+| 0.4040 | 200  | 0.0305        | -               |
+| 0.5051 | 250  | 0.015         | -               |
+| 0.6061 | 300  | 0.0097        | -               |
+| 0.7071 | 350  | 0.0107        | -               |
+| 0.8081 | 400  | 0.0054        | -               |
+| 0.9091 | 450  | 0.0047        | -               |
 
 ### Framework Versions
 - Python: 3.10.14

config_setfit.json

@@ -1,4 +1,4 @@
 {
-  "labels": null,
-  "normalize_embeddings": false
+  "normalize_embeddings": false,
+  "labels": null
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5c68953c586d0893617359046d4b09f724f2c6514dcd0f03050308558f060434
+oid sha256:d352ed759d5102f1e62eee9c053370d52ad6f8c184a3cedc635570e8e4d294a7
 size 437951328

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3a4caa74bb7ca9af6563083b07cd256483daf4871b65eea8094bcd5da1a18d98
+oid sha256:6f9d8340dc54e309118a2d4bdcbbe595c01c9b5dedd28038b8fdfd1f26cde990
 size 7007