Hugging Face's logo

Theoreticallyhugo
/
longformer-simple

Token Classification
Transformers
Safetensors
longformer
Generated from Trainer
Eval Results
Model card Files Community
Theoreticallyhugo commited on
Commit
22cef70
·
verified ·
1 Parent(s): 0360ab4

trainer: training complete at 2024-03-02 13:15:37.019190.

Browse files
Files changed (3) hide show
  1. README.md +28 -28
  2. meta_data/README_s42_e16.md +28 -28
  3. model.safetensors +1 -1
README.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
- split: train[0%:20%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.8390263857639599
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.7323
36
- - Claim: {'precision': 0.5949216896060749, 'recall': 0.5882214922571563, 'f1-score': 0.5915526191599811, 'support': 4262.0}
37
- - Majorclaim: {'precision': 0.8048654244306418, 'recall': 0.7182448036951501, 'f1-score': 0.7590920185501586, 'support': 2165.0}
38
- - O: {'precision': 0.9059811340313051, 'recall': 0.8856911228212404, 'f1-score': 0.8957212400717396, 'support': 9868.0}
39
- - Premise: {'precision': 0.8721660143268591, 'recall': 0.9057443055449037, 'f1-score': 0.8886380737396539, 'support': 13039.0}
40
- - Accuracy: 0.8390
41
- - Macro avg: {'precision': 0.7944835655987202, 'recall': 0.7744754310796127, 'f1-score': 0.7837509878803832, 'support': 29334.0}
42
- - Weighted avg: {'precision': 0.8382929152663212, 'recall': 0.8390263857639599, 'f1-score': 0.8382955111317995, 'support': 29334.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.6210 | {'precision': 0.3978787878787879, 'recall': 0.3080713280150164, 'f1-score': 0.3472626289341444, 'support': 4262.0} | {'precision': 0.5235765124555161, 'recall': 0.54364896073903, 'f1-score': 0.5334239746204397, 'support': 2165.0} | {'precision': 0.9157377442167086, 'recall': 0.7742197000405351, 'f1-score': 0.8390533194223273, 'support': 9868.0} | {'precision': 0.7896134170821731, 'recall': 0.9351944167497508, 'f1-score': 0.8562600940945159, 'support': 13039.0} | 0.7610 | {'precision': 0.6567016154082965, 'recall': 0.6402836013860831, 'f1-score': 0.6440000042678569, 'support': 29334.0} | {'precision': 0.7554909643645776, 'recall': 0.7610281584509443, 'f1-score': 0.7526914076678427, 'support': 29334.0} |
74
- | No log | 2.0 | 82 | 0.5057 | {'precision': 0.5328757225433526, 'recall': 0.34608165180666356, 'f1-score': 0.4196301564722618, 'support': 4262.0} | {'precision': 0.6262842465753424, 'recall': 0.6757505773672056, 'f1-score': 0.6500777604976672, 'support': 2165.0} | {'precision': 0.9066595059076262, 'recall': 0.8553911633563032, 'f1-score': 0.8802794869120867, 'support': 9868.0} | {'precision': 0.821313672922252, 'recall': 0.9397959966255081, 'f1-score': 0.8765692621338388, 'support': 13039.0} | 0.8057 | {'precision': 0.7217832869871433, 'recall': 0.7042548472889201, 'f1-score': 0.7066391665039636, 'support': 29334.0} | {'precision': 0.7937221895699558, 'recall': 0.8056521442694484, 'f1-score': 0.7947114837449316, 'support': 29334.0} |
75
- | No log | 3.0 | 123 | 0.4707 | {'precision': 0.542234931808183, 'recall': 0.5783669638667293, 'f1-score': 0.5597184377838329, 'support': 4262.0} | {'precision': 0.669374492282697, 'recall': 0.7612009237875289, 'f1-score': 0.7123406094661768, 'support': 2165.0} | {'precision': 0.9139037996000421, 'recall': 0.8799148763680583, 'f1-score': 0.896587330270019, 'support': 9868.0} | {'precision': 0.8872514619883041, 'recall': 0.8726896234373802, 'f1-score': 0.879910300030931, 'support': 13039.0} | 0.8241 | {'precision': 0.7531911714198065, 'recall': 0.7730430968649242, 'f1-score': 0.76213916938774, 'support': 29334.0} | {'precision': 0.8300087121591747, 'recall': 0.8241289970682485, 'f1-score': 0.8266316076408545, 'support': 29334.0} |
76
- | No log | 4.0 | 164 | 0.4995 | {'precision': 0.5606635071090047, 'recall': 0.5551384326607227, 'f1-score': 0.5578872907333177, 'support': 4262.0} | {'precision': 0.748995983935743, 'recall': 0.6891454965357968, 'f1-score': 0.7178253548231899, 'support': 2165.0} | {'precision': 0.9082793070464449, 'recall': 0.8660316173490069, 'f1-score': 0.8866524874202418, 'support': 9868.0} | {'precision': 0.8605702617953767, 'recall': 0.9050540685635402, 'f1-score': 0.8822517942583731, 'support': 13039.0} | 0.8252 | {'precision': 0.7696272649716422, 'recall': 0.7538424037772666, 'f1-score': 0.7611542318087806, 'support': 29334.0} | {'precision': 0.8248108003682995, 'recall': 0.8251517010977023, 'f1-score': 0.8244690603905188, 'support': 29334.0} |
77
- | No log | 5.0 | 205 | 0.5356 | {'precision': 0.5562700964630225, 'recall': 0.5682778038479587, 'f1-score': 0.5622098421541318, 'support': 4262.0} | {'precision': 0.7994186046511628, 'recall': 0.6351039260969977, 'f1-score': 0.7078507078507079, 'support': 2165.0} | {'precision': 0.9167929019692708, 'recall': 0.858633968382651, 'f1-score': 0.8867608581894296, 'support': 9868.0} | {'precision': 0.8533314310172635, 'recall': 0.9174016412301557, 'f1-score': 0.8842074139778985, 'support': 13039.0} | 0.8261 | {'precision': 0.7814532585251799, 'recall': 0.7448543348894408, 'f1-score': 0.760257205543042, 'support': 29334.0} | {'precision': 0.827540237126271, 'recall': 0.8260721347242108, 'f1-score': 0.8252666444817892, 'support': 29334.0} |
78
- | No log | 6.0 | 246 | 0.5402 | {'precision': 0.5901198337001712, 'recall': 0.5661661191928672, 'f1-score': 0.5778948628906718, 'support': 4262.0} | {'precision': 0.7593778591033852, 'recall': 0.766743648960739, 'f1-score': 0.7630429786256032, 'support': 2165.0} | {'precision': 0.909998948585848, 'recall': 0.877077421970004, 'f1-score': 0.8932349450436038, 'support': 9868.0} | {'precision': 0.8704605845881311, 'recall': 0.9044405245801058, 'f1-score': 0.8871252867942979, 'support': 13039.0} | 0.8359 | {'precision': 0.7824893064943839, 'recall': 0.778606928675929, 'f1-score': 0.7803245183385442, 'support': 29334.0} | {'precision': 0.8348315600763192, 'recall': 0.8359241835412832, 'f1-score': 0.8350939185438606, 'support': 29334.0} |
79
- | No log | 7.0 | 287 | 0.5522 | {'precision': 0.5645161290322581, 'recall': 0.6241201313937119, 'f1-score': 0.5928237129485181, 'support': 4262.0} | {'precision': 0.776257938446507, 'recall': 0.7339491916859122, 'f1-score': 0.7545109211775878, 'support': 2165.0} | {'precision': 0.9063338147307612, 'recall': 0.8903526550466153, 'f1-score': 0.8982721603108067, 'support': 9868.0} | {'precision': 0.8897601117925626, 'recall': 0.8789784492675818, 'f1-score': 0.8843364197530864, 'support': 13039.0} | 0.8351 | {'precision': 0.7842169985005223, 'recall': 0.7818501068484554, 'f1-score': 0.7824858035474997, 'support': 29334.0} | {'precision': 0.8397030872059231, 'recall': 0.835071930183405, 'f1-score': 0.8370881251804593, 'support': 29334.0} |
80
- | No log | 8.0 | 328 | 0.5864 | {'precision': 0.5921815889029004, 'recall': 0.5509150633505396, 'f1-score': 0.5708034520481342, 'support': 4262.0} | {'precision': 0.7887952404561229, 'recall': 0.7348729792147806, 'f1-score': 0.7608799617407939, 'support': 2165.0} | {'precision': 0.909240754094983, 'recall': 0.8944061613295501, 'f1-score': 0.9017624521072796, 'support': 9868.0} | {'precision': 0.868889703187981, 'recall': 0.909272183449651, 'f1-score': 0.888622395442962, 'support': 13039.0} | 0.8393 | {'precision': 0.7897768216604968, 'recall': 0.7723665968361304, 'f1-score': 0.7805170653347924, 'support': 29334.0} | {'precision': 0.8363489544136171, 'recall': 0.8393331969727961, 'f1-score': 0.8374380828176649, 'support': 29334.0} |
81
- | No log | 9.0 | 369 | 0.6258 | {'precision': 0.5400439384861194, 'recall': 0.6344439230408259, 'f1-score': 0.5834502103786816, 'support': 4262.0} | {'precision': 0.7136109918419923, 'recall': 0.7676674364896073, 'f1-score': 0.739652870493992, 'support': 2165.0} | {'precision': 0.9208710651142734, 'recall': 0.8656262667207134, 'f1-score': 0.8923944839114083, 'support': 9868.0} | {'precision': 0.8900330136770948, 'recall': 0.86839481555334, 'f1-score': 0.8790807810255813, 'support': 13039.0} | 0.8260 | {'precision': 0.76613975227987, 'recall': 0.7840331104511216, 'f1-score': 0.7736445864524157, 'support': 29334.0} | {'precision': 0.8365354605252965, 'recall': 0.8260380445898957, 'f1-score': 0.8303162314135053, 'support': 29334.0} |
82
- | No log | 10.0 | 410 | 0.6433 | {'precision': 0.5887546468401487, 'recall': 0.5945565462224308, 'f1-score': 0.5916413728694839, 'support': 4262.0} | {'precision': 0.765103914934751, 'recall': 0.7311778290993072, 'f1-score': 0.747756258856873, 'support': 2165.0} | {'precision': 0.9102390147166266, 'recall': 0.8837657073368463, 'f1-score': 0.8968070337806571, 'support': 9868.0} | {'precision': 0.878101644245142, 'recall': 0.9010660326712171, 'f1-score': 0.8894356334456263, 'support': 13039.0} | 0.8382 | {'precision': 0.7855498051841671, 'recall': 0.7776415288324503, 'f1-score': 0.78141007473816, 'support': 29334.0} | {'precision': 0.8385330407446147, 'recall': 0.8381741324060816, 'f1-score': 0.8381915478775454, 'support': 29334.0} |
83
- | No log | 11.0 | 451 | 0.6916 | {'precision': 0.5963211533681332, 'recall': 0.5628812763960582, 'f1-score': 0.5791188895594448, 'support': 4262.0} | {'precision': 0.7905679513184585, 'recall': 0.7200923787528868, 'f1-score': 0.7536862460720328, 'support': 2165.0} | {'precision': 0.9027949034114262, 'recall': 0.8903526550466153, 'f1-score': 0.896530612244898, 'support': 9868.0} | {'precision': 0.8699933857573308, 'recall': 0.9078917094869239, 'f1-score': 0.888538617428507, 'support': 13039.0} | 0.8380 | {'precision': 0.7899193484638372, 'recall': 0.770304504920621, 'f1-score': 0.7794685913262207, 'support': 29334.0} | {'precision': 0.8354034306270279, 'recall': 0.838003681734506, 'f1-score': 0.836318079509486, 'support': 29334.0} |
84
- | No log | 12.0 | 492 | 0.6997 | {'precision': 0.5914396887159533, 'recall': 0.5706241201313937, 'f1-score': 0.5808454740864581, 'support': 4262.0} | {'precision': 0.797138477261114, 'recall': 0.7205542725173211, 'f1-score': 0.7569141193595342, 'support': 2165.0} | {'precision': 0.9003264639869415, 'recall': 0.8943048236724767, 'f1-score': 0.8973055414336554, 'support': 9868.0} | {'precision': 0.870236945703038, 'recall': 0.8985351637395506, 'f1-score': 0.8841596860614294, 'support': 13039.0} | 0.8363 | {'precision': 0.7897853939167616, 'recall': 0.7710045950151856, 'f1-score': 0.7798062052352693, 'support': 29334.0} | {'precision': 0.8344570068256206, 'recall': 0.8363332651530647, 'f1-score': 0.8351214191174802, 'support': 29334.0} |
85
- | 0.2673 | 13.0 | 533 | 0.7149 | {'precision': 0.5794110827730118, 'recall': 0.5863444392304082, 'f1-score': 0.582857142857143, 'support': 4262.0} | {'precision': 0.7928753180661577, 'recall': 0.7196304849884526, 'f1-score': 0.7544794188861985, 'support': 2165.0} | {'precision': 0.9063998332291016, 'recall': 0.8812322659100121, 'f1-score': 0.8936388860343233, 'support': 9868.0} | {'precision': 0.8717872530084683, 'recall': 0.9000690236981364, 'f1-score': 0.8857024263235349, 'support': 13039.0} | 0.8348 | {'precision': 0.7876183717691848, 'recall': 0.7718190534567524, 'f1-score': 0.7791694685252999, 'support': 29334.0} | {'precision': 0.8351269054569442, 'recall': 0.834833299243199, 'f1-score': 0.8346862872081897, 'support': 29334.0} |
86
- | 0.2673 | 14.0 | 574 | 0.7156 | {'precision': 0.5767102058888642, 'recall': 0.6112153918348193, 'f1-score': 0.5934616698940653, 'support': 4262.0} | {'precision': 0.7826520438683948, 'recall': 0.7251732101616628, 'f1-score': 0.7528170702469431, 'support': 2165.0} | {'precision': 0.9055462885738115, 'recall': 0.8802188893392785, 'f1-score': 0.8927029804727646, 'support': 9868.0} | {'precision': 0.8822149935698615, 'recall': 0.894393741851369, 'f1-score': 0.8882626247238937, 'support': 13039.0} | 0.8360 | {'precision': 0.786780882975233, 'recall': 0.7777503082967825, 'f1-score': 0.7818110863344166, 'support': 29334.0} | {'precision': 0.8383279692260589, 'recall': 0.8359923638099134, 'f1-score': 0.8369275233262845, 'support': 29334.0} |
87
- | 0.2673 | 15.0 | 615 | 0.7311 | {'precision': 0.5837887067395264, 'recall': 0.6015954950727358, 'f1-score': 0.5925583545181419, 'support': 4262.0} | {'precision': 0.7695631301008161, 'recall': 0.7404157043879908, 'f1-score': 0.754708097928437, 'support': 2165.0} | {'precision': 0.9121858097359211, 'recall': 0.8716051884880421, 'f1-score': 0.8914339016427424, 'support': 9868.0} | {'precision': 0.8736411020104244, 'recall': 0.8998389447043484, 'f1-score': 0.8865465261249008, 'support': 13039.0} | 0.8352 | {'precision': 0.784794687146672, 'recall': 0.7783638331632793, 'f1-score': 0.7813117200535555, 'support': 29334.0} | {'precision': 0.8368128296304671, 'recall': 0.8352423808549806, 'f1-score': 0.8357461183106479, 'support': 29334.0} |
88
- | 0.2673 | 16.0 | 656 | 0.7323 | {'precision': 0.5949216896060749, 'recall': 0.5882214922571563, 'f1-score': 0.5915526191599811, 'support': 4262.0} | {'precision': 0.8048654244306418, 'recall': 0.7182448036951501, 'f1-score': 0.7590920185501586, 'support': 2165.0} | {'precision': 0.9059811340313051, 'recall': 0.8856911228212404, 'f1-score': 0.8957212400717396, 'support': 9868.0} | {'precision': 0.8721660143268591, 'recall': 0.9057443055449037, 'f1-score': 0.8886380737396539, 'support': 13039.0} | 0.8390 | {'precision': 0.7944835655987202, 'recall': 0.7744754310796127, 'f1-score': 0.7837509878803832, 'support': 29334.0} | {'precision': 0.8382929152663212, 'recall': 0.8390263857639599, 'f1-score': 0.8382955111317995, 'support': 29334.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
+ split: train[20%:40%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.8525299930594573
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.6282
36
+ - Claim: {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0}
37
+ - Majorclaim: {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0}
38
+ - O: {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0}
39
+ - Premise: {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0}
40
+ - Accuracy: 0.8525
41
+ - Macro avg: {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0}
42
+ - Weighted avg: {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.6017 | {'precision': 0.44954648526077096, 'recall': 0.36426274689940286, 'f1-score': 0.4024359299670135, 'support': 4354.0} | {'precision': 0.452642328312484, 'recall': 0.7547892720306514, 'f1-score': 0.5659112671560804, 'support': 2349.0} | {'precision': 0.8588395638629284, 'recall': 0.8666011787819253, 'f1-score': 0.8627029141404263, 'support': 10180.0} | {'precision': 0.8723285486443381, 'recall': 0.8179303125467324, 'f1-score': 0.8442540711584471, 'support': 13374.0} | 0.7641 | {'precision': 0.6583392315201303, 'recall': 0.7008958775646781, 'f1-score': 0.6688260456054919, 'support': 30257.0} | {'precision': 0.774369269779734, 'recall': 0.764120699342301, 'f1-score': 0.765274191732446, 'support': 30257.0} |
74
+ | No log | 2.0 | 82 | 0.4534 | {'precision': 0.5700215450907972, 'recall': 0.4253559944878273, 'f1-score': 0.48717611469156913, 'support': 4354.0} | {'precision': 0.6134453781512605, 'recall': 0.8701575138356747, 'f1-score': 0.719591621193452, 'support': 2349.0} | {'precision': 0.884556428434566, 'recall': 0.9069744597249508, 'f1-score': 0.8956251818799107, 'support': 10180.0} | {'precision': 0.8789847408974165, 'recall': 0.8700463586062509, 'f1-score': 0.8744927100556139, 'support': 13374.0} | 0.8185 | {'precision': 0.7367520231435101, 'recall': 0.768133581663676, 'f1-score': 0.7442214069551364, 'support': 30257.0} | {'precision': 0.8157842273466825, 'recall': 0.8184882837029448, 'f1-score': 0.8138419333500274, 'support': 30257.0} |
75
+ | No log | 3.0 | 123 | 0.4068 | {'precision': 0.5364131531168045, 'recall': 0.61070280202113, 'f1-score': 0.5711524003866395, 'support': 4354.0} | {'precision': 0.7406428885953324, 'recall': 0.7160493827160493, 'f1-score': 0.7281385281385281, 'support': 2349.0} | {'precision': 0.9368757079600453, 'recall': 0.8937131630648331, 'f1-score': 0.9147855814187321, 'support': 10180.0} | {'precision': 0.8885718576362818, 'recall': 0.8848512038283236, 'f1-score': 0.8867076277536341, 'support': 13374.0} | 0.8353 | {'precision': 0.775625901827116, 'recall': 0.7763291379075841, 'f1-score': 0.7751960344243833, 'support': 30257.0} | {'precision': 0.842663441353799, 'recall': 0.8352777869583898, 'f1-score': 0.8384354029249637, 'support': 30257.0} |
76
+ | No log | 4.0 | 164 | 0.4188 | {'precision': 0.5978233358643381, 'recall': 0.5424896646761599, 'f1-score': 0.5688139674894642, 'support': 4354.0} | {'precision': 0.7550200803212851, 'recall': 0.800340570455513, 'f1-score': 0.7770200454639389, 'support': 2349.0} | {'precision': 0.9040139616055847, 'recall': 0.9159135559921414, 'f1-score': 0.9099248560554309, 'support': 10180.0} | {'precision': 0.8849800029625241, 'recall': 0.8934499775684164, 'f1-score': 0.8891948206578361, 'support': 13374.0} | 0.8433 | {'precision': 0.7854593451884331, 'recall': 0.7880484421730577, 'f1-score': 0.7862384224166675, 'support': 30257.0} | {'precision': 0.8399725571535075, 'recall': 0.8432759361470074, 'f1-score': 0.8413577905068613, 'support': 30257.0} |
77
+ | No log | 5.0 | 205 | 0.4474 | {'precision': 0.5675675675675675, 'recall': 0.5884244372990354, 'f1-score': 0.5778078484438429, 'support': 4354.0} | {'precision': 0.728060263653484, 'recall': 0.8229033631332482, 'f1-score': 0.7725819344524381, 'support': 2349.0} | {'precision': 0.9372398001665279, 'recall': 0.8845776031434185, 'f1-score': 0.9101475641803114, 'support': 10180.0} | {'precision': 0.887240356083086, 'recall': 0.8942724689696426, 'f1-score': 0.8907425337007523, 'support': 13374.0} | 0.8415 | {'precision': 0.7800269968676663, 'recall': 0.7975444681363362, 'f1-score': 0.7878199701943362, 'support': 30257.0} | {'precision': 0.845703686302729, 'recall': 0.8414581749677761, 'f1-score': 0.8430665031306045, 'support': 30257.0} |
78
+ | No log | 6.0 | 246 | 0.4609 | {'precision': 0.6318574213311056, 'recall': 0.5211299954065227, 'f1-score': 0.5711768407803649, 'support': 4354.0} | {'precision': 0.7856852379015861, 'recall': 0.822477650063857, 'f1-score': 0.8036605657237937, 'support': 2349.0} | {'precision': 0.9054745582697692, 'recall': 0.9212180746561887, 'f1-score': 0.9132784729999514, 'support': 10180.0} | {'precision': 0.8784115523465704, 'recall': 0.9096754897562435, 'f1-score': 0.8937702027622686, 'support': 13374.0} | 0.8509 | {'precision': 0.8003571924622578, 'recall': 0.793625302470703, 'f1-score': 0.7954715205665946, 'support': 30257.0} | {'precision': 0.8448388452449266, 'recall': 0.8508774828965198, 'f1-score': 0.8469167525043788, 'support': 30257.0} |
79
+ | No log | 7.0 | 287 | 0.4865 | {'precision': 0.630808729139923, 'recall': 0.5643086816720257, 'f1-score': 0.5957085707358469, 'support': 4354.0} | {'precision': 0.7307692307692307, 'recall': 0.8573861217539378, 'f1-score': 0.7890303623898138, 'support': 2349.0} | {'precision': 0.9172843166320783, 'recall': 0.9117878192534381, 'f1-score': 0.9145278092516873, 'support': 10180.0} | {'precision': 0.8916734633350634, 'recall': 0.8992074173770002, 'f1-score': 0.8954245932764975, 'support': 13374.0} | 0.8520 | {'precision': 0.7926339349690739, 'recall': 0.8081725100141004, 'f1-score': 0.7986728339134614, 'support': 30257.0} | {'precision': 0.8502598860333094, 'recall': 0.8520011898073173, 'f1-score': 0.8504626713454606, 'support': 30257.0} |
80
+ | No log | 8.0 | 328 | 0.5096 | {'precision': 0.5821842854016196, 'recall': 0.6109324758842444, 'f1-score': 0.5962120363106579, 'support': 4354.0} | {'precision': 0.774493927125506, 'recall': 0.8143891017454236, 'f1-score': 0.7939406515874662, 'support': 2349.0} | {'precision': 0.9332583810302535, 'recall': 0.8969548133595285, 'f1-score': 0.9147465437788018, 'support': 10180.0} | {'precision': 0.8931070418341521, 'recall': 0.8971138029011515, 'f1-score': 0.8951059385258131, 'support': 13374.0} | 0.8495 | {'precision': 0.7957609088478829, 'recall': 0.804847548472587, 'f1-score': 0.8000012925506848, 'support': 30257.0} | {'precision': 0.8526655157429486, 'recall': 0.8494563241563936, 'f1-score': 0.8508490740717186, 'support': 30257.0} |
81
+ | No log | 9.0 | 369 | 0.5327 | {'precision': 0.6261045190608432, 'recall': 0.5695911805236564, 'f1-score': 0.5965123271196633, 'support': 4354.0} | {'precision': 0.7516019600452318, 'recall': 0.8488718603661133, 'f1-score': 0.7972810875649741, 'support': 2349.0} | {'precision': 0.9026343722860176, 'recall': 0.918860510805501, 'f1-score': 0.9106751691573773, 'support': 10180.0} | {'precision': 0.8932981927710844, 'recall': 0.8870195902497383, 'f1-score': 0.8901478202146019, 'support': 13374.0} | 0.8491 | {'precision': 0.7934097610407942, 'recall': 0.8060857854862523, 'f1-score': 0.7986541010141541, 'support': 30257.0} | {'precision': 0.846989457650438, 'recall': 0.8490927719205473, 'f1-score': 0.8475902474317125, 'support': 30257.0} |
82
+ | No log | 10.0 | 410 | 0.5611 | {'precision': 0.6031589338598223, 'recall': 0.5613229214515388, 'f1-score': 0.5814894123245301, 'support': 4354.0} | {'precision': 0.7980400511291009, 'recall': 0.7973605789697744, 'f1-score': 0.7977001703577512, 'support': 2349.0} | {'precision': 0.9056966897613549, 'recall': 0.9245579567779961, 'f1-score': 0.9150301380517207, 'support': 10180.0} | {'precision': 0.8834100698054359, 'recall': 0.8894870644534171, 'f1-score': 0.8864381520119226, 'support': 13374.0} | 0.8469 | {'precision': 0.7975764361389286, 'recall': 0.7931821304131816, 'f1-score': 0.7951644681864812, 'support': 30257.0} | {'precision': 0.8439524293048358, 'recall': 0.8469114585054698, 'f1-score': 0.8452864874840641, 'support': 30257.0} |
83
+ | No log | 11.0 | 451 | 0.5648 | {'precision': 0.5827433628318585, 'recall': 0.6049609554432706, 'f1-score': 0.5936443542934415, 'support': 4354.0} | {'precision': 0.7425330812854443, 'recall': 0.8361004682843763, 'f1-score': 0.7865438526231477, 'support': 2349.0} | {'precision': 0.938811369509044, 'recall': 0.8922396856581533, 'f1-score': 0.9149332661798036, 'support': 10180.0} | {'precision': 0.890064843109488, 'recall': 0.8929265739494542, 'f1-score': 0.8914934119667053, 'support': 13374.0} | 0.8468 | {'precision': 0.7885381641839586, 'recall': 0.8065569208338136, 'f1-score': 0.7966537212657745, 'support': 30257.0} | {'precision': 0.8507883056171393, 'recall': 0.8468453580989523, 'f1-score': 0.8483713709144507, 'support': 30257.0} |
84
+ | No log | 12.0 | 492 | 0.6091 | {'precision': 0.6024649589173514, 'recall': 0.5725769407441433, 'f1-score': 0.5871408384361753, 'support': 4354.0} | {'precision': 0.749317738791423, 'recall': 0.8182205193699447, 'f1-score': 0.7822547822547823, 'support': 2349.0} | {'precision': 0.9307965499746321, 'recall': 0.9010805500982318, 'f1-score': 0.9156975293236835, 'support': 10180.0} | {'precision': 0.8842981239506533, 'recall': 0.9057873485868102, 'f1-score': 0.8949137517083441, 'support': 13374.0} | 0.8495 | {'precision': 0.791719342908515, 'recall': 0.7994163396997825, 'f1-score': 0.7950017254307464, 'support': 30257.0} | {'precision': 0.8489074193741941, 'recall': 0.8494563241563936, 'f1-score': 0.848871502724331, 'support': 30257.0} |
85
+ | 0.2687 | 13.0 | 533 | 0.6049 | {'precision': 0.6140061306295685, 'recall': 0.5980707395498392, 'f1-score': 0.605933682373473, 'support': 4354.0} | {'precision': 0.7614920874152223, 'recall': 0.8603661132396765, 'f1-score': 0.8079152508494902, 'support': 2349.0} | {'precision': 0.921222343486457, 'recall': 0.9120825147347741, 'f1-score': 0.9166296460832224, 'support': 10180.0} | {'precision': 0.8964089437627042, 'recall': 0.8903095558546433, 'f1-score': 0.8933488389541209, 'support': 13374.0} | 0.8533 | {'precision': 0.798282376323488, 'recall': 0.8152072308447333, 'f1-score': 0.8059568545650766, 'support': 30257.0} | {'precision': 0.8536452482623537, 'recall': 0.8532570975311499, 'f1-score': 0.8531898518226914, 'support': 30257.0} |
86
+ | 0.2687 | 14.0 | 574 | 0.6213 | {'precision': 0.6094716801523085, 'recall': 0.588194763435921, 'f1-score': 0.5986442262739597, 'support': 4354.0} | {'precision': 0.759737755495565, 'recall': 0.8386547467007237, 'f1-score': 0.7972480777013357, 'support': 2349.0} | {'precision': 0.9096567149664495, 'recall': 0.918860510805501, 'f1-score': 0.914235449347603, 'support': 10180.0} | {'precision': 0.8965778890659383, 'recall': 0.8835053088081352, 'f1-score': 0.88999359771024, 'support': 13374.0} | 0.8494 | {'precision': 0.7938610099200654, 'recall': 0.8073038324375702, 'f1-score': 0.8000303377582847, 'support': 30257.0} | {'precision': 0.8490399487645354, 'recall': 0.8494232739531348, 'f1-score': 0.8490241579089998, 'support': 30257.0} |
87
+ | 0.2687 | 15.0 | 615 | 0.6240 | {'precision': 0.6123274631128034, 'recall': 0.5909508497932935, 'f1-score': 0.6014492753623188, 'support': 4354.0} | {'precision': 0.7543659832953683, 'recall': 0.8458918688803746, 'f1-score': 0.7975115392333936, 'support': 2349.0} | {'precision': 0.9263157894736842, 'recall': 0.9076620825147348, 'f1-score': 0.9168940709501364, 'support': 10180.0} | {'precision': 0.8923843522237096, 'recall': 0.8971885748467175, 'f1-score': 0.894780014914243, 'support': 13374.0} | 0.8527 | {'precision': 0.7963483970263914, 'recall': 0.8104233440087801, 'f1-score': 0.802658725115023, 'support': 30257.0} | {'precision': 0.8527852243327482, 'recall': 0.8526621938724923, 'f1-score': 0.8524584166415128, 'support': 30257.0} |
88
+ | 0.2687 | 16.0 | 656 | 0.6282 | {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0} | {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0} | {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0} | {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0} | 0.8525 | {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0} | {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0} |
89
 
90
 
91
  ### Framework versions
meta_data/README_s42_e16.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
- split: train[0%:20%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.8390263857639599
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.7323
36
- - Claim: {'precision': 0.5949216896060749, 'recall': 0.5882214922571563, 'f1-score': 0.5915526191599811, 'support': 4262.0}
37
- - Majorclaim: {'precision': 0.8048654244306418, 'recall': 0.7182448036951501, 'f1-score': 0.7590920185501586, 'support': 2165.0}
38
- - O: {'precision': 0.9059811340313051, 'recall': 0.8856911228212404, 'f1-score': 0.8957212400717396, 'support': 9868.0}
39
- - Premise: {'precision': 0.8721660143268591, 'recall': 0.9057443055449037, 'f1-score': 0.8886380737396539, 'support': 13039.0}
40
- - Accuracy: 0.8390
41
- - Macro avg: {'precision': 0.7944835655987202, 'recall': 0.7744754310796127, 'f1-score': 0.7837509878803832, 'support': 29334.0}
42
- - Weighted avg: {'precision': 0.8382929152663212, 'recall': 0.8390263857639599, 'f1-score': 0.8382955111317995, 'support': 29334.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.6210 | {'precision': 0.3978787878787879, 'recall': 0.3080713280150164, 'f1-score': 0.3472626289341444, 'support': 4262.0} | {'precision': 0.5235765124555161, 'recall': 0.54364896073903, 'f1-score': 0.5334239746204397, 'support': 2165.0} | {'precision': 0.9157377442167086, 'recall': 0.7742197000405351, 'f1-score': 0.8390533194223273, 'support': 9868.0} | {'precision': 0.7896134170821731, 'recall': 0.9351944167497508, 'f1-score': 0.8562600940945159, 'support': 13039.0} | 0.7610 | {'precision': 0.6567016154082965, 'recall': 0.6402836013860831, 'f1-score': 0.6440000042678569, 'support': 29334.0} | {'precision': 0.7554909643645776, 'recall': 0.7610281584509443, 'f1-score': 0.7526914076678427, 'support': 29334.0} |
74
- | No log | 2.0 | 82 | 0.5057 | {'precision': 0.5328757225433526, 'recall': 0.34608165180666356, 'f1-score': 0.4196301564722618, 'support': 4262.0} | {'precision': 0.6262842465753424, 'recall': 0.6757505773672056, 'f1-score': 0.6500777604976672, 'support': 2165.0} | {'precision': 0.9066595059076262, 'recall': 0.8553911633563032, 'f1-score': 0.8802794869120867, 'support': 9868.0} | {'precision': 0.821313672922252, 'recall': 0.9397959966255081, 'f1-score': 0.8765692621338388, 'support': 13039.0} | 0.8057 | {'precision': 0.7217832869871433, 'recall': 0.7042548472889201, 'f1-score': 0.7066391665039636, 'support': 29334.0} | {'precision': 0.7937221895699558, 'recall': 0.8056521442694484, 'f1-score': 0.7947114837449316, 'support': 29334.0} |
75
- | No log | 3.0 | 123 | 0.4707 | {'precision': 0.542234931808183, 'recall': 0.5783669638667293, 'f1-score': 0.5597184377838329, 'support': 4262.0} | {'precision': 0.669374492282697, 'recall': 0.7612009237875289, 'f1-score': 0.7123406094661768, 'support': 2165.0} | {'precision': 0.9139037996000421, 'recall': 0.8799148763680583, 'f1-score': 0.896587330270019, 'support': 9868.0} | {'precision': 0.8872514619883041, 'recall': 0.8726896234373802, 'f1-score': 0.879910300030931, 'support': 13039.0} | 0.8241 | {'precision': 0.7531911714198065, 'recall': 0.7730430968649242, 'f1-score': 0.76213916938774, 'support': 29334.0} | {'precision': 0.8300087121591747, 'recall': 0.8241289970682485, 'f1-score': 0.8266316076408545, 'support': 29334.0} |
76
- | No log | 4.0 | 164 | 0.4995 | {'precision': 0.5606635071090047, 'recall': 0.5551384326607227, 'f1-score': 0.5578872907333177, 'support': 4262.0} | {'precision': 0.748995983935743, 'recall': 0.6891454965357968, 'f1-score': 0.7178253548231899, 'support': 2165.0} | {'precision': 0.9082793070464449, 'recall': 0.8660316173490069, 'f1-score': 0.8866524874202418, 'support': 9868.0} | {'precision': 0.8605702617953767, 'recall': 0.9050540685635402, 'f1-score': 0.8822517942583731, 'support': 13039.0} | 0.8252 | {'precision': 0.7696272649716422, 'recall': 0.7538424037772666, 'f1-score': 0.7611542318087806, 'support': 29334.0} | {'precision': 0.8248108003682995, 'recall': 0.8251517010977023, 'f1-score': 0.8244690603905188, 'support': 29334.0} |
77
- | No log | 5.0 | 205 | 0.5356 | {'precision': 0.5562700964630225, 'recall': 0.5682778038479587, 'f1-score': 0.5622098421541318, 'support': 4262.0} | {'precision': 0.7994186046511628, 'recall': 0.6351039260969977, 'f1-score': 0.7078507078507079, 'support': 2165.0} | {'precision': 0.9167929019692708, 'recall': 0.858633968382651, 'f1-score': 0.8867608581894296, 'support': 9868.0} | {'precision': 0.8533314310172635, 'recall': 0.9174016412301557, 'f1-score': 0.8842074139778985, 'support': 13039.0} | 0.8261 | {'precision': 0.7814532585251799, 'recall': 0.7448543348894408, 'f1-score': 0.760257205543042, 'support': 29334.0} | {'precision': 0.827540237126271, 'recall': 0.8260721347242108, 'f1-score': 0.8252666444817892, 'support': 29334.0} |
78
- | No log | 6.0 | 246 | 0.5402 | {'precision': 0.5901198337001712, 'recall': 0.5661661191928672, 'f1-score': 0.5778948628906718, 'support': 4262.0} | {'precision': 0.7593778591033852, 'recall': 0.766743648960739, 'f1-score': 0.7630429786256032, 'support': 2165.0} | {'precision': 0.909998948585848, 'recall': 0.877077421970004, 'f1-score': 0.8932349450436038, 'support': 9868.0} | {'precision': 0.8704605845881311, 'recall': 0.9044405245801058, 'f1-score': 0.8871252867942979, 'support': 13039.0} | 0.8359 | {'precision': 0.7824893064943839, 'recall': 0.778606928675929, 'f1-score': 0.7803245183385442, 'support': 29334.0} | {'precision': 0.8348315600763192, 'recall': 0.8359241835412832, 'f1-score': 0.8350939185438606, 'support': 29334.0} |
79
- | No log | 7.0 | 287 | 0.5522 | {'precision': 0.5645161290322581, 'recall': 0.6241201313937119, 'f1-score': 0.5928237129485181, 'support': 4262.0} | {'precision': 0.776257938446507, 'recall': 0.7339491916859122, 'f1-score': 0.7545109211775878, 'support': 2165.0} | {'precision': 0.9063338147307612, 'recall': 0.8903526550466153, 'f1-score': 0.8982721603108067, 'support': 9868.0} | {'precision': 0.8897601117925626, 'recall': 0.8789784492675818, 'f1-score': 0.8843364197530864, 'support': 13039.0} | 0.8351 | {'precision': 0.7842169985005223, 'recall': 0.7818501068484554, 'f1-score': 0.7824858035474997, 'support': 29334.0} | {'precision': 0.8397030872059231, 'recall': 0.835071930183405, 'f1-score': 0.8370881251804593, 'support': 29334.0} |
80
- | No log | 8.0 | 328 | 0.5864 | {'precision': 0.5921815889029004, 'recall': 0.5509150633505396, 'f1-score': 0.5708034520481342, 'support': 4262.0} | {'precision': 0.7887952404561229, 'recall': 0.7348729792147806, 'f1-score': 0.7608799617407939, 'support': 2165.0} | {'precision': 0.909240754094983, 'recall': 0.8944061613295501, 'f1-score': 0.9017624521072796, 'support': 9868.0} | {'precision': 0.868889703187981, 'recall': 0.909272183449651, 'f1-score': 0.888622395442962, 'support': 13039.0} | 0.8393 | {'precision': 0.7897768216604968, 'recall': 0.7723665968361304, 'f1-score': 0.7805170653347924, 'support': 29334.0} | {'precision': 0.8363489544136171, 'recall': 0.8393331969727961, 'f1-score': 0.8374380828176649, 'support': 29334.0} |
81
- | No log | 9.0 | 369 | 0.6258 | {'precision': 0.5400439384861194, 'recall': 0.6344439230408259, 'f1-score': 0.5834502103786816, 'support': 4262.0} | {'precision': 0.7136109918419923, 'recall': 0.7676674364896073, 'f1-score': 0.739652870493992, 'support': 2165.0} | {'precision': 0.9208710651142734, 'recall': 0.8656262667207134, 'f1-score': 0.8923944839114083, 'support': 9868.0} | {'precision': 0.8900330136770948, 'recall': 0.86839481555334, 'f1-score': 0.8790807810255813, 'support': 13039.0} | 0.8260 | {'precision': 0.76613975227987, 'recall': 0.7840331104511216, 'f1-score': 0.7736445864524157, 'support': 29334.0} | {'precision': 0.8365354605252965, 'recall': 0.8260380445898957, 'f1-score': 0.8303162314135053, 'support': 29334.0} |
82
- | No log | 10.0 | 410 | 0.6433 | {'precision': 0.5887546468401487, 'recall': 0.5945565462224308, 'f1-score': 0.5916413728694839, 'support': 4262.0} | {'precision': 0.765103914934751, 'recall': 0.7311778290993072, 'f1-score': 0.747756258856873, 'support': 2165.0} | {'precision': 0.9102390147166266, 'recall': 0.8837657073368463, 'f1-score': 0.8968070337806571, 'support': 9868.0} | {'precision': 0.878101644245142, 'recall': 0.9010660326712171, 'f1-score': 0.8894356334456263, 'support': 13039.0} | 0.8382 | {'precision': 0.7855498051841671, 'recall': 0.7776415288324503, 'f1-score': 0.78141007473816, 'support': 29334.0} | {'precision': 0.8385330407446147, 'recall': 0.8381741324060816, 'f1-score': 0.8381915478775454, 'support': 29334.0} |
83
- | No log | 11.0 | 451 | 0.6916 | {'precision': 0.5963211533681332, 'recall': 0.5628812763960582, 'f1-score': 0.5791188895594448, 'support': 4262.0} | {'precision': 0.7905679513184585, 'recall': 0.7200923787528868, 'f1-score': 0.7536862460720328, 'support': 2165.0} | {'precision': 0.9027949034114262, 'recall': 0.8903526550466153, 'f1-score': 0.896530612244898, 'support': 9868.0} | {'precision': 0.8699933857573308, 'recall': 0.9078917094869239, 'f1-score': 0.888538617428507, 'support': 13039.0} | 0.8380 | {'precision': 0.7899193484638372, 'recall': 0.770304504920621, 'f1-score': 0.7794685913262207, 'support': 29334.0} | {'precision': 0.8354034306270279, 'recall': 0.838003681734506, 'f1-score': 0.836318079509486, 'support': 29334.0} |
84
- | No log | 12.0 | 492 | 0.6997 | {'precision': 0.5914396887159533, 'recall': 0.5706241201313937, 'f1-score': 0.5808454740864581, 'support': 4262.0} | {'precision': 0.797138477261114, 'recall': 0.7205542725173211, 'f1-score': 0.7569141193595342, 'support': 2165.0} | {'precision': 0.9003264639869415, 'recall': 0.8943048236724767, 'f1-score': 0.8973055414336554, 'support': 9868.0} | {'precision': 0.870236945703038, 'recall': 0.8985351637395506, 'f1-score': 0.8841596860614294, 'support': 13039.0} | 0.8363 | {'precision': 0.7897853939167616, 'recall': 0.7710045950151856, 'f1-score': 0.7798062052352693, 'support': 29334.0} | {'precision': 0.8344570068256206, 'recall': 0.8363332651530647, 'f1-score': 0.8351214191174802, 'support': 29334.0} |
85
- | 0.2673 | 13.0 | 533 | 0.7149 | {'precision': 0.5794110827730118, 'recall': 0.5863444392304082, 'f1-score': 0.582857142857143, 'support': 4262.0} | {'precision': 0.7928753180661577, 'recall': 0.7196304849884526, 'f1-score': 0.7544794188861985, 'support': 2165.0} | {'precision': 0.9063998332291016, 'recall': 0.8812322659100121, 'f1-score': 0.8936388860343233, 'support': 9868.0} | {'precision': 0.8717872530084683, 'recall': 0.9000690236981364, 'f1-score': 0.8857024263235349, 'support': 13039.0} | 0.8348 | {'precision': 0.7876183717691848, 'recall': 0.7718190534567524, 'f1-score': 0.7791694685252999, 'support': 29334.0} | {'precision': 0.8351269054569442, 'recall': 0.834833299243199, 'f1-score': 0.8346862872081897, 'support': 29334.0} |
86
- | 0.2673 | 14.0 | 574 | 0.7156 | {'precision': 0.5767102058888642, 'recall': 0.6112153918348193, 'f1-score': 0.5934616698940653, 'support': 4262.0} | {'precision': 0.7826520438683948, 'recall': 0.7251732101616628, 'f1-score': 0.7528170702469431, 'support': 2165.0} | {'precision': 0.9055462885738115, 'recall': 0.8802188893392785, 'f1-score': 0.8927029804727646, 'support': 9868.0} | {'precision': 0.8822149935698615, 'recall': 0.894393741851369, 'f1-score': 0.8882626247238937, 'support': 13039.0} | 0.8360 | {'precision': 0.786780882975233, 'recall': 0.7777503082967825, 'f1-score': 0.7818110863344166, 'support': 29334.0} | {'precision': 0.8383279692260589, 'recall': 0.8359923638099134, 'f1-score': 0.8369275233262845, 'support': 29334.0} |
87
- | 0.2673 | 15.0 | 615 | 0.7311 | {'precision': 0.5837887067395264, 'recall': 0.6015954950727358, 'f1-score': 0.5925583545181419, 'support': 4262.0} | {'precision': 0.7695631301008161, 'recall': 0.7404157043879908, 'f1-score': 0.754708097928437, 'support': 2165.0} | {'precision': 0.9121858097359211, 'recall': 0.8716051884880421, 'f1-score': 0.8914339016427424, 'support': 9868.0} | {'precision': 0.8736411020104244, 'recall': 0.8998389447043484, 'f1-score': 0.8865465261249008, 'support': 13039.0} | 0.8352 | {'precision': 0.784794687146672, 'recall': 0.7783638331632793, 'f1-score': 0.7813117200535555, 'support': 29334.0} | {'precision': 0.8368128296304671, 'recall': 0.8352423808549806, 'f1-score': 0.8357461183106479, 'support': 29334.0} |
88
- | 0.2673 | 16.0 | 656 | 0.7323 | {'precision': 0.5949216896060749, 'recall': 0.5882214922571563, 'f1-score': 0.5915526191599811, 'support': 4262.0} | {'precision': 0.8048654244306418, 'recall': 0.7182448036951501, 'f1-score': 0.7590920185501586, 'support': 2165.0} | {'precision': 0.9059811340313051, 'recall': 0.8856911228212404, 'f1-score': 0.8957212400717396, 'support': 9868.0} | {'precision': 0.8721660143268591, 'recall': 0.9057443055449037, 'f1-score': 0.8886380737396539, 'support': 13039.0} | 0.8390 | {'precision': 0.7944835655987202, 'recall': 0.7744754310796127, 'f1-score': 0.7837509878803832, 'support': 29334.0} | {'precision': 0.8382929152663212, 'recall': 0.8390263857639599, 'f1-score': 0.8382955111317995, 'support': 29334.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: simple
20
+ split: train[20%:40%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.8525299930594573
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.6282
36
+ - Claim: {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0}
37
+ - Majorclaim: {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0}
38
+ - O: {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0}
39
+ - Premise: {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0}
40
+ - Accuracy: 0.8525
41
+ - Macro avg: {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0}
42
+ - Weighted avg: {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.6017 | {'precision': 0.44954648526077096, 'recall': 0.36426274689940286, 'f1-score': 0.4024359299670135, 'support': 4354.0} | {'precision': 0.452642328312484, 'recall': 0.7547892720306514, 'f1-score': 0.5659112671560804, 'support': 2349.0} | {'precision': 0.8588395638629284, 'recall': 0.8666011787819253, 'f1-score': 0.8627029141404263, 'support': 10180.0} | {'precision': 0.8723285486443381, 'recall': 0.8179303125467324, 'f1-score': 0.8442540711584471, 'support': 13374.0} | 0.7641 | {'precision': 0.6583392315201303, 'recall': 0.7008958775646781, 'f1-score': 0.6688260456054919, 'support': 30257.0} | {'precision': 0.774369269779734, 'recall': 0.764120699342301, 'f1-score': 0.765274191732446, 'support': 30257.0} |
74
+ | No log | 2.0 | 82 | 0.4534 | {'precision': 0.5700215450907972, 'recall': 0.4253559944878273, 'f1-score': 0.48717611469156913, 'support': 4354.0} | {'precision': 0.6134453781512605, 'recall': 0.8701575138356747, 'f1-score': 0.719591621193452, 'support': 2349.0} | {'precision': 0.884556428434566, 'recall': 0.9069744597249508, 'f1-score': 0.8956251818799107, 'support': 10180.0} | {'precision': 0.8789847408974165, 'recall': 0.8700463586062509, 'f1-score': 0.8744927100556139, 'support': 13374.0} | 0.8185 | {'precision': 0.7367520231435101, 'recall': 0.768133581663676, 'f1-score': 0.7442214069551364, 'support': 30257.0} | {'precision': 0.8157842273466825, 'recall': 0.8184882837029448, 'f1-score': 0.8138419333500274, 'support': 30257.0} |
75
+ | No log | 3.0 | 123 | 0.4068 | {'precision': 0.5364131531168045, 'recall': 0.61070280202113, 'f1-score': 0.5711524003866395, 'support': 4354.0} | {'precision': 0.7406428885953324, 'recall': 0.7160493827160493, 'f1-score': 0.7281385281385281, 'support': 2349.0} | {'precision': 0.9368757079600453, 'recall': 0.8937131630648331, 'f1-score': 0.9147855814187321, 'support': 10180.0} | {'precision': 0.8885718576362818, 'recall': 0.8848512038283236, 'f1-score': 0.8867076277536341, 'support': 13374.0} | 0.8353 | {'precision': 0.775625901827116, 'recall': 0.7763291379075841, 'f1-score': 0.7751960344243833, 'support': 30257.0} | {'precision': 0.842663441353799, 'recall': 0.8352777869583898, 'f1-score': 0.8384354029249637, 'support': 30257.0} |
76
+ | No log | 4.0 | 164 | 0.4188 | {'precision': 0.5978233358643381, 'recall': 0.5424896646761599, 'f1-score': 0.5688139674894642, 'support': 4354.0} | {'precision': 0.7550200803212851, 'recall': 0.800340570455513, 'f1-score': 0.7770200454639389, 'support': 2349.0} | {'precision': 0.9040139616055847, 'recall': 0.9159135559921414, 'f1-score': 0.9099248560554309, 'support': 10180.0} | {'precision': 0.8849800029625241, 'recall': 0.8934499775684164, 'f1-score': 0.8891948206578361, 'support': 13374.0} | 0.8433 | {'precision': 0.7854593451884331, 'recall': 0.7880484421730577, 'f1-score': 0.7862384224166675, 'support': 30257.0} | {'precision': 0.8399725571535075, 'recall': 0.8432759361470074, 'f1-score': 0.8413577905068613, 'support': 30257.0} |
77
+ | No log | 5.0 | 205 | 0.4474 | {'precision': 0.5675675675675675, 'recall': 0.5884244372990354, 'f1-score': 0.5778078484438429, 'support': 4354.0} | {'precision': 0.728060263653484, 'recall': 0.8229033631332482, 'f1-score': 0.7725819344524381, 'support': 2349.0} | {'precision': 0.9372398001665279, 'recall': 0.8845776031434185, 'f1-score': 0.9101475641803114, 'support': 10180.0} | {'precision': 0.887240356083086, 'recall': 0.8942724689696426, 'f1-score': 0.8907425337007523, 'support': 13374.0} | 0.8415 | {'precision': 0.7800269968676663, 'recall': 0.7975444681363362, 'f1-score': 0.7878199701943362, 'support': 30257.0} | {'precision': 0.845703686302729, 'recall': 0.8414581749677761, 'f1-score': 0.8430665031306045, 'support': 30257.0} |
78
+ | No log | 6.0 | 246 | 0.4609 | {'precision': 0.6318574213311056, 'recall': 0.5211299954065227, 'f1-score': 0.5711768407803649, 'support': 4354.0} | {'precision': 0.7856852379015861, 'recall': 0.822477650063857, 'f1-score': 0.8036605657237937, 'support': 2349.0} | {'precision': 0.9054745582697692, 'recall': 0.9212180746561887, 'f1-score': 0.9132784729999514, 'support': 10180.0} | {'precision': 0.8784115523465704, 'recall': 0.9096754897562435, 'f1-score': 0.8937702027622686, 'support': 13374.0} | 0.8509 | {'precision': 0.8003571924622578, 'recall': 0.793625302470703, 'f1-score': 0.7954715205665946, 'support': 30257.0} | {'precision': 0.8448388452449266, 'recall': 0.8508774828965198, 'f1-score': 0.8469167525043788, 'support': 30257.0} |
79
+ | No log | 7.0 | 287 | 0.4865 | {'precision': 0.630808729139923, 'recall': 0.5643086816720257, 'f1-score': 0.5957085707358469, 'support': 4354.0} | {'precision': 0.7307692307692307, 'recall': 0.8573861217539378, 'f1-score': 0.7890303623898138, 'support': 2349.0} | {'precision': 0.9172843166320783, 'recall': 0.9117878192534381, 'f1-score': 0.9145278092516873, 'support': 10180.0} | {'precision': 0.8916734633350634, 'recall': 0.8992074173770002, 'f1-score': 0.8954245932764975, 'support': 13374.0} | 0.8520 | {'precision': 0.7926339349690739, 'recall': 0.8081725100141004, 'f1-score': 0.7986728339134614, 'support': 30257.0} | {'precision': 0.8502598860333094, 'recall': 0.8520011898073173, 'f1-score': 0.8504626713454606, 'support': 30257.0} |
80
+ | No log | 8.0 | 328 | 0.5096 | {'precision': 0.5821842854016196, 'recall': 0.6109324758842444, 'f1-score': 0.5962120363106579, 'support': 4354.0} | {'precision': 0.774493927125506, 'recall': 0.8143891017454236, 'f1-score': 0.7939406515874662, 'support': 2349.0} | {'precision': 0.9332583810302535, 'recall': 0.8969548133595285, 'f1-score': 0.9147465437788018, 'support': 10180.0} | {'precision': 0.8931070418341521, 'recall': 0.8971138029011515, 'f1-score': 0.8951059385258131, 'support': 13374.0} | 0.8495 | {'precision': 0.7957609088478829, 'recall': 0.804847548472587, 'f1-score': 0.8000012925506848, 'support': 30257.0} | {'precision': 0.8526655157429486, 'recall': 0.8494563241563936, 'f1-score': 0.8508490740717186, 'support': 30257.0} |
81
+ | No log | 9.0 | 369 | 0.5327 | {'precision': 0.6261045190608432, 'recall': 0.5695911805236564, 'f1-score': 0.5965123271196633, 'support': 4354.0} | {'precision': 0.7516019600452318, 'recall': 0.8488718603661133, 'f1-score': 0.7972810875649741, 'support': 2349.0} | {'precision': 0.9026343722860176, 'recall': 0.918860510805501, 'f1-score': 0.9106751691573773, 'support': 10180.0} | {'precision': 0.8932981927710844, 'recall': 0.8870195902497383, 'f1-score': 0.8901478202146019, 'support': 13374.0} | 0.8491 | {'precision': 0.7934097610407942, 'recall': 0.8060857854862523, 'f1-score': 0.7986541010141541, 'support': 30257.0} | {'precision': 0.846989457650438, 'recall': 0.8490927719205473, 'f1-score': 0.8475902474317125, 'support': 30257.0} |
82
+ | No log | 10.0 | 410 | 0.5611 | {'precision': 0.6031589338598223, 'recall': 0.5613229214515388, 'f1-score': 0.5814894123245301, 'support': 4354.0} | {'precision': 0.7980400511291009, 'recall': 0.7973605789697744, 'f1-score': 0.7977001703577512, 'support': 2349.0} | {'precision': 0.9056966897613549, 'recall': 0.9245579567779961, 'f1-score': 0.9150301380517207, 'support': 10180.0} | {'precision': 0.8834100698054359, 'recall': 0.8894870644534171, 'f1-score': 0.8864381520119226, 'support': 13374.0} | 0.8469 | {'precision': 0.7975764361389286, 'recall': 0.7931821304131816, 'f1-score': 0.7951644681864812, 'support': 30257.0} | {'precision': 0.8439524293048358, 'recall': 0.8469114585054698, 'f1-score': 0.8452864874840641, 'support': 30257.0} |
83
+ | No log | 11.0 | 451 | 0.5648 | {'precision': 0.5827433628318585, 'recall': 0.6049609554432706, 'f1-score': 0.5936443542934415, 'support': 4354.0} | {'precision': 0.7425330812854443, 'recall': 0.8361004682843763, 'f1-score': 0.7865438526231477, 'support': 2349.0} | {'precision': 0.938811369509044, 'recall': 0.8922396856581533, 'f1-score': 0.9149332661798036, 'support': 10180.0} | {'precision': 0.890064843109488, 'recall': 0.8929265739494542, 'f1-score': 0.8914934119667053, 'support': 13374.0} | 0.8468 | {'precision': 0.7885381641839586, 'recall': 0.8065569208338136, 'f1-score': 0.7966537212657745, 'support': 30257.0} | {'precision': 0.8507883056171393, 'recall': 0.8468453580989523, 'f1-score': 0.8483713709144507, 'support': 30257.0} |
84
+ | No log | 12.0 | 492 | 0.6091 | {'precision': 0.6024649589173514, 'recall': 0.5725769407441433, 'f1-score': 0.5871408384361753, 'support': 4354.0} | {'precision': 0.749317738791423, 'recall': 0.8182205193699447, 'f1-score': 0.7822547822547823, 'support': 2349.0} | {'precision': 0.9307965499746321, 'recall': 0.9010805500982318, 'f1-score': 0.9156975293236835, 'support': 10180.0} | {'precision': 0.8842981239506533, 'recall': 0.9057873485868102, 'f1-score': 0.8949137517083441, 'support': 13374.0} | 0.8495 | {'precision': 0.791719342908515, 'recall': 0.7994163396997825, 'f1-score': 0.7950017254307464, 'support': 30257.0} | {'precision': 0.8489074193741941, 'recall': 0.8494563241563936, 'f1-score': 0.848871502724331, 'support': 30257.0} |
85
+ | 0.2687 | 13.0 | 533 | 0.6049 | {'precision': 0.6140061306295685, 'recall': 0.5980707395498392, 'f1-score': 0.605933682373473, 'support': 4354.0} | {'precision': 0.7614920874152223, 'recall': 0.8603661132396765, 'f1-score': 0.8079152508494902, 'support': 2349.0} | {'precision': 0.921222343486457, 'recall': 0.9120825147347741, 'f1-score': 0.9166296460832224, 'support': 10180.0} | {'precision': 0.8964089437627042, 'recall': 0.8903095558546433, 'f1-score': 0.8933488389541209, 'support': 13374.0} | 0.8533 | {'precision': 0.798282376323488, 'recall': 0.8152072308447333, 'f1-score': 0.8059568545650766, 'support': 30257.0} | {'precision': 0.8536452482623537, 'recall': 0.8532570975311499, 'f1-score': 0.8531898518226914, 'support': 30257.0} |
86
+ | 0.2687 | 14.0 | 574 | 0.6213 | {'precision': 0.6094716801523085, 'recall': 0.588194763435921, 'f1-score': 0.5986442262739597, 'support': 4354.0} | {'precision': 0.759737755495565, 'recall': 0.8386547467007237, 'f1-score': 0.7972480777013357, 'support': 2349.0} | {'precision': 0.9096567149664495, 'recall': 0.918860510805501, 'f1-score': 0.914235449347603, 'support': 10180.0} | {'precision': 0.8965778890659383, 'recall': 0.8835053088081352, 'f1-score': 0.88999359771024, 'support': 13374.0} | 0.8494 | {'precision': 0.7938610099200654, 'recall': 0.8073038324375702, 'f1-score': 0.8000303377582847, 'support': 30257.0} | {'precision': 0.8490399487645354, 'recall': 0.8494232739531348, 'f1-score': 0.8490241579089998, 'support': 30257.0} |
87
+ | 0.2687 | 15.0 | 615 | 0.6240 | {'precision': 0.6123274631128034, 'recall': 0.5909508497932935, 'f1-score': 0.6014492753623188, 'support': 4354.0} | {'precision': 0.7543659832953683, 'recall': 0.8458918688803746, 'f1-score': 0.7975115392333936, 'support': 2349.0} | {'precision': 0.9263157894736842, 'recall': 0.9076620825147348, 'f1-score': 0.9168940709501364, 'support': 10180.0} | {'precision': 0.8923843522237096, 'recall': 0.8971885748467175, 'f1-score': 0.894780014914243, 'support': 13374.0} | 0.8527 | {'precision': 0.7963483970263914, 'recall': 0.8104233440087801, 'f1-score': 0.802658725115023, 'support': 30257.0} | {'precision': 0.8527852243327482, 'recall': 0.8526621938724923, 'f1-score': 0.8524584166415128, 'support': 30257.0} |
88
+ | 0.2687 | 16.0 | 656 | 0.6282 | {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0} | {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0} | {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0} | {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0} | 0.8525 | {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0} | {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0} |
89
 
90
 
91
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:68d85248bdc4a5c3a14f28ad13e9587c84018077aa9588862cabff40e2ddf443
3
  size 592324828
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a911b67406fd260f9f05f6fbae536e6a3a5469f0ba1dcfc3b423b3d941155906
3
  size 592324828