trainer: training complete at 2024-02-19 20:46:24.071770.

README.md

@@ -22,7 +22,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: 0.837507614031316
+      value: 0.8431688702569063
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,17 +32,17 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4465
-- B-claim: {'precision': 0.5551020408163265, 'recall': 0.49097472924187724, 'f1-score': 0.5210727969348659, 'support': 277.0}
-- B-majorclaim: {'precision': 0.696969696969697, 'recall': 0.6524822695035462, 'f1-score': 0.673992673992674, 'support': 141.0}
-- B-premise: {'precision': 0.7235213204951857, 'recall': 0.8205928237129485, 'f1-score': 0.7690058479532164, 'support': 641.0}
-- I-claim: {'precision': 0.6258634982039237, 'recall': 0.5552831576366757, 'f1-score': 0.5884645362431801, 'support': 4079.0}
-- I-majorclaim: {'precision': 0.7485029940119761, 'recall': 0.7961783439490446, 'f1-score': 0.7716049382716049, 'support': 2041.0}
-- I-premise: {'precision': 0.8608122758735719, 'recall': 0.9010912265386294, 'f1-score': 0.8804913418067046, 'support': 11455.0}
-- O: {'precision': 0.9317375886524822, 'recall': 0.906522911051213, 'f1-score': 0.918957320072135, 'support': 9275.0}
-- Accuracy: 0.8375
-- Macro avg: {'precision': 0.7346442021461661, 'recall': 0.7318750659477049, 'f1-score': 0.731941350753483, 'support': 27909.0}
-- Weighted avg: {'precision': 0.8348158563134491, 'recall': 0.837507614031316, 'f1-score': 0.8354599902087165, 'support': 27909.0}
+- Loss: 0.4438
+- B-claim: {'precision': 0.5970149253731343, 'recall': 0.5776173285198556, 'f1-score': 0.5871559633027523, 'support': 277.0}
+- B-majorclaim: {'precision': 0.6540880503144654, 'recall': 0.7375886524822695, 'f1-score': 0.6933333333333332, 'support': 141.0}
+- B-premise: {'precision': 0.7533039647577092, 'recall': 0.8003120124804992, 'f1-score': 0.7760968229954615, 'support': 641.0}
+- I-claim: {'precision': 0.6253114100647733, 'recall': 0.6153468987496935, 'f1-score': 0.6202891387618931, 'support': 4079.0}
+- I-majorclaim: {'precision': 0.7570308898109728, 'recall': 0.8045075943165115, 'f1-score': 0.7800475059382421, 'support': 2041.0}
+- I-premise: {'precision': 0.8764452113891286, 'recall': 0.8867743343518114, 'f1-score': 0.881579518333695, 'support': 11455.0}
+- O: {'precision': 0.9354231280460789, 'recall': 0.9105121293800539, 'f1-score': 0.9227995410588428, 'support': 9275.0}
+- Accuracy: 0.8432
+- Macro avg: {'precision': 0.7426596542508946, 'recall': 0.7618084214686707, 'f1-score': 0.7516145462463172, 'support': 27909.0}
+- Weighted avg: {'precision': 0.8438834099280034, 'recall': 0.8431688702569063, 'f1-score': 0.8433686534034867, 'support': 27909.0}
 
 ## Model description
 
@@ -67,18 +67,19 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 6
+- num_epochs: 7
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                                             | B-majorclaim                                                                                                       | B-premise                                                                                                         | I-claim                                                                                                              | I-majorclaim                                                                                                       | I-premise                                                                                                           | O                                                                                                                  | Accuracy | Macro avg                                                                                                              | Weighted avg                                                                                                        |
-|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:----------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
-| No log        | 1.0   | 41   | 0.7283          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                                | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                               | {'precision': 0.8, 'recall': 0.1747269890795632, 'f1-score': 0.28681177976952626, 'support': 641.0}               | {'precision': 0.43048740595354923, 'recall': 0.32262809512135326, 'f1-score': 0.3688340807174888, 'support': 4079.0} | {'precision': 0.612691466083151, 'recall': 0.13718765311122, 'f1-score': 0.2241793434747798, 'support': 2041.0}    | {'precision': 0.7836619287788477, 'recall': 0.8952422522915757, 'f1-score': 0.83574426469989, 'support': 11455.0}   | {'precision': 0.7604082728982003, 'recall': 0.9156873315363881, 'f1-score': 0.8308550185873605, 'support': 9275.0} | 0.7330   | {'precision': 0.48389272481624973, 'recall': 0.34935318873430005, 'f1-score': 0.36377492674986367, 'support': 27909.0} | {'precision': 0.7004513073364416, 'recall': 0.7329535275359204, 'f1-score': 0.6960300066518306, 'support': 27909.0} |
-| No log        | 2.0   | 82   | 0.5262          | {'precision': 0.25, 'recall': 0.0036101083032490976, 'f1-score': 0.007117437722419929, 'support': 277.0}            | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                               | {'precision': 0.5701754385964912, 'recall': 0.8112324492979719, 'f1-score': 0.6696716033483581, 'support': 641.0} | {'precision': 0.5822991508817766, 'recall': 0.4371169404265751, 'f1-score': 0.499369836157401, 'support': 4079.0}    | {'precision': 0.6713729308666018, 'recall': 0.6756491915727585, 'f1-score': 0.6735042735042734, 'support': 2041.0} | {'precision': 0.8282341604432667, 'recall': 0.9003928415539066, 'f1-score': 0.862807428475824, 'support': 11455.0}  | {'precision': 0.884974533106961, 'recall': 0.8991913746630728, 'f1-score': 0.8920263115674635, 'support': 9275.0}  | 0.8004   | {'precision': 0.5410080305564425, 'recall': 0.5324561294025049, 'f1-score': 0.5149281272536771, 'support': 27909.0}    | {'precision': 0.7838247141399024, 'recall': 0.800351141208929, 'f1-score': 0.7882685135577218, 'support': 27909.0}  |
-| No log        | 3.0   | 123  | 0.4882          | {'precision': 0.3162393162393162, 'recall': 0.26714801444043323, 'f1-score': 0.2896281800391389, 'support': 277.0}  | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                               | {'precision': 0.6475609756097561, 'recall': 0.828393135725429, 'f1-score': 0.7268993839835729, 'support': 641.0}  | {'precision': 0.5622895622895623, 'recall': 0.3684726648688404, 'f1-score': 0.4452014218009479, 'support': 4079.0}   | {'precision': 0.7279187817258883, 'recall': 0.7025967662910338, 'f1-score': 0.7150336574420344, 'support': 2041.0} | {'precision': 0.8009592326139089, 'recall': 0.9330423395896988, 'f1-score': 0.8619702407355135, 'support': 11455.0} | {'precision': 0.9306417051990526, 'recall': 0.8897035040431267, 'f1-score': 0.9097122698710176, 'support': 9275.0} | 0.8055   | {'precision': 0.5693727962396407, 'recall': 0.569908060708366, 'f1-score': 0.564063593410318, 'support': 27909.0}      | {'precision': 0.7914520785180175, 'recall': 0.8055465978716543, 'f1-score': 0.7930409622719756, 'support': 27909.0} |
-| No log        | 4.0   | 164  | 0.4589          | {'precision': 0.4944649446494465, 'recall': 0.48375451263537905, 'f1-score': 0.48905109489051096, 'support': 277.0} | {'precision': 0.7263157894736842, 'recall': 0.48936170212765956, 'f1-score': 0.5847457627118644, 'support': 141.0} | {'precision': 0.733044733044733, 'recall': 0.7925117004680188, 'f1-score': 0.7616191904047976, 'support': 641.0}  | {'precision': 0.6057815298030187, 'recall': 0.5805344447168423, 'f1-score': 0.5928893340010015, 'support': 4079.0}   | {'precision': 0.6938000843525939, 'recall': 0.8059774620284175, 'f1-score': 0.7456935630099728, 'support': 2041.0} | {'precision': 0.8850901340911109, 'recall': 0.8701003928415539, 'f1-score': 0.8775312555027294, 'support': 11455.0} | {'precision': 0.9134171232140939, 'recall': 0.9167654986522911, 'f1-score': 0.9150882479552304, 'support': 9275.0} | 0.8311   | {'precision': 0.7217020483755258, 'recall': 0.7055722447814518, 'f1-score': 0.7095169212108725, 'support': 27909.0}    | {'precision': 0.831521700022212, 'recall': 0.8310580816224157, 'f1-score': 0.8307726680976919, 'support': 27909.0}  |
-| No log        | 5.0   | 205  | 0.4503          | {'precision': 0.5175097276264592, 'recall': 0.48014440433212996, 'f1-score': 0.49812734082397003, 'support': 277.0} | {'precision': 0.6929824561403509, 'recall': 0.5602836879432624, 'f1-score': 0.6196078431372548, 'support': 141.0}  | {'precision': 0.7054886211512718, 'recall': 0.8221528861154446, 'f1-score': 0.7593659942363112, 'support': 641.0} | {'precision': 0.606861499364676, 'recall': 0.5854376072566806, 'f1-score': 0.5959570751185427, 'support': 4079.0}    | {'precision': 0.7482582443102648, 'recall': 0.7893189612934836, 'f1-score': 0.7682403433476394, 'support': 2041.0} | {'precision': 0.8633981403212172, 'recall': 0.8916630292448713, 'f1-score': 0.8773029847541336, 'support': 11455.0} | {'precision': 0.938915812013975, 'recall': 0.8982210242587602, 'f1-score': 0.9181176989199912, 'support': 9275.0}  | 0.8342   | {'precision': 0.7247735001326021, 'recall': 0.7181745143492331, 'f1-score': 0.719531325762549, 'support': 27909.0}     | {'precision': 0.8346602140306427, 'recall': 0.8342470170912609, 'f1-score': 0.833997433789052, 'support': 27909.0}  |
-| No log        | 6.0   | 246  | 0.4465          | {'precision': 0.5551020408163265, 'recall': 0.49097472924187724, 'f1-score': 0.5210727969348659, 'support': 277.0}  | {'precision': 0.696969696969697, 'recall': 0.6524822695035462, 'f1-score': 0.673992673992674, 'support': 141.0}    | {'precision': 0.7235213204951857, 'recall': 0.8205928237129485, 'f1-score': 0.7690058479532164, 'support': 641.0} | {'precision': 0.6258634982039237, 'recall': 0.5552831576366757, 'f1-score': 0.5884645362431801, 'support': 4079.0}   | {'precision': 0.7485029940119761, 'recall': 0.7961783439490446, 'f1-score': 0.7716049382716049, 'support': 2041.0} | {'precision': 0.8608122758735719, 'recall': 0.9010912265386294, 'f1-score': 0.8804913418067046, 'support': 11455.0} | {'precision': 0.9317375886524822, 'recall': 0.906522911051213, 'f1-score': 0.918957320072135, 'support': 9275.0}   | 0.8375   | {'precision': 0.7346442021461661, 'recall': 0.7318750659477049, 'f1-score': 0.731941350753483, 'support': 27909.0}     | {'precision': 0.8348158563134491, 'recall': 0.837507614031316, 'f1-score': 0.8354599902087165, 'support': 27909.0}  |
+| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                                           | B-majorclaim                                                                                                        | B-premise                                                                                                         | I-claim                                                                                                              | I-majorclaim                                                                                                        | I-premise                                                                                                           | O                                                                                                                  | Accuracy | Macro avg                                                                                                            | Weighted avg                                                                                                        |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
+| No log        | 1.0   | 41   | 0.7886          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                              | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                                | {'precision': 0.8888888888888888, 'recall': 0.0748829953198128, 'f1-score': 0.1381294964028777, 'support': 641.0} | {'precision': 0.47000821692686934, 'recall': 0.1402304486393724, 'f1-score': 0.21601208459214502, 'support': 4079.0} | {'precision': 0.5424354243542435, 'recall': 0.0720235178833905, 'f1-score': 0.12716262975778547, 'support': 2041.0} | {'precision': 0.7775630122158652, 'recall': 0.8779572239196858, 'f1-score': 0.8247160605190865, 'support': 11455.0} | {'precision': 0.6536142336038115, 'recall': 0.9466307277628032, 'f1-score': 0.7732957548000705, 'support': 9275.0} | 0.7024   | {'precision': 0.4760728251413826, 'recall': 0.3016749876464378, 'f1-score': 0.29704514658170933, 'support': 27909.0} | {'precision': 0.6651369922726568, 'recall': 0.7024257408004586, 'f1-score': 0.6395296795513288, 'support': 27909.0} |
+| No log        | 2.0   | 82   | 0.5373          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                              | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                                | {'precision': 0.5765472312703583, 'recall': 0.828393135725429, 'f1-score': 0.679897567221511, 'support': 641.0}   | {'precision': 0.5732105732105732, 'recall': 0.47315518509438587, 'f1-score': 0.5183991404781091, 'support': 4079.0}  | {'precision': 0.5972927241962775, 'recall': 0.6918177364037237, 'f1-score': 0.6410896708286039, 'support': 2041.0}  | {'precision': 0.858668504004823, 'recall': 0.870362287210825, 'f1-score': 0.8644758519032342, 'support': 11455.0}   | {'precision': 0.8686365992742353, 'recall': 0.903288409703504, 'f1-score': 0.8856236786469344, 'support': 9275.0}  | 0.7962   | {'precision': 0.4963365188508953, 'recall': 0.538145250591124, 'f1-score': 0.5127837012969133, 'support': 27909.0}   | {'precision': 0.7818058448922789, 'recall': 0.7961947758787488, 'f1-score': 0.7874004427160499, 'support': 27909.0} |
+| No log        | 3.0   | 123  | 0.4911          | {'precision': 0.34893617021276596, 'recall': 0.296028880866426, 'f1-score': 0.3203125, 'support': 277.0}          | {'precision': 0.8333333333333334, 'recall': 0.03546099290780142, 'f1-score': 0.06802721088435375, 'support': 141.0} | {'precision': 0.6662371134020618, 'recall': 0.8065522620904836, 'f1-score': 0.7297106563161608, 'support': 641.0} | {'precision': 0.6018223234624146, 'recall': 0.3238538857563128, 'f1-score': 0.421102964615875, 'support': 4079.0}    | {'precision': 0.7116279069767442, 'recall': 0.7496325330720235, 'f1-score': 0.7301360057265569, 'support': 2041.0}  | {'precision': 0.7889374090247453, 'recall': 0.9463116542994325, 'f1-score': 0.8604881921016073, 'support': 11455.0} | {'precision': 0.9330078346769615, 'recall': 0.8859299191374663, 'f1-score': 0.908859639420418, 'support': 9275.0}  | 0.8066   | {'precision': 0.6977002987270037, 'recall': 0.5776814468757066, 'f1-score': 0.5769481670092816, 'support': 27909.0}  | {'precision': 0.7968542338095115, 'recall': 0.8066215199398044, 'f1-score': 0.7904444769227739, 'support': 27909.0} |
+| No log        | 4.0   | 164  | 0.4471          | {'precision': 0.5464285714285714, 'recall': 0.5523465703971119, 'f1-score': 0.5493716337522441, 'support': 277.0} | {'precision': 0.6544117647058824, 'recall': 0.6312056737588653, 'f1-score': 0.6425992779783394, 'support': 141.0}   | {'precision': 0.7510917030567685, 'recall': 0.8049921996879875, 'f1-score': 0.7771084337349398, 'support': 641.0} | {'precision': 0.6000949893137022, 'recall': 0.619514586908556, 'f1-score': 0.6096501809408926, 'support': 4079.0}    | {'precision': 0.7037037037037037, 'recall': 0.8005879470847623, 'f1-score': 0.7490258996103598, 'support': 2041.0}  | {'precision': 0.8949800652410294, 'recall': 0.8622435617634221, 'f1-score': 0.8783068783068784, 'support': 11455.0} | {'precision': 0.9216195734545848, 'recall': 0.9178436657681941, 'f1-score': 0.9197277441659464, 'support': 9275.0} | 0.8352   | {'precision': 0.7246186244148918, 'recall': 0.7412477436241284, 'f1-score': 0.7322557212128, 'support': 27909.0}     | {'precision': 0.838766973613019, 'recall': 0.835178616216991, 'f1-score': 0.8365729339666597, 'support': 27909.0}   |
+| No log        | 5.0   | 205  | 0.4553          | {'precision': 0.5725490196078431, 'recall': 0.5270758122743683, 'f1-score': 0.5488721804511277, 'support': 277.0} | {'precision': 0.608433734939759, 'recall': 0.7163120567375887, 'f1-score': 0.6579804560260587, 'support': 141.0}    | {'precision': 0.7355021216407355, 'recall': 0.8112324492979719, 'f1-score': 0.7715133531157271, 'support': 641.0} | {'precision': 0.5901240035429584, 'recall': 0.6533464084334396, 'f1-score': 0.6201279813845259, 'support': 4079.0}   | {'precision': 0.7180370210934137, 'recall': 0.8172464478196962, 'f1-score': 0.7644362969752522, 'support': 2041.0}  | {'precision': 0.8847149103239047, 'recall': 0.8655608904408555, 'f1-score': 0.8750330950489806, 'support': 11455.0} | {'precision': 0.9455065827132226, 'recall': 0.8904582210242588, 'f1-score': 0.9171571349250416, 'support': 9275.0} | 0.8339   | {'precision': 0.7221239134088339, 'recall': 0.7544617551468827, 'f1-score': 0.7364457854181019, 'support': 27909.0}  | {'precision': 0.8417519193793559, 'recall': 0.8339245404708159, 'f1-score': 0.8369775321954073, 'support': 27909.0} |
+| No log        | 6.0   | 246  | 0.4431          | {'precision': 0.5860805860805861, 'recall': 0.5776173285198556, 'f1-score': 0.5818181818181819, 'support': 277.0} | {'precision': 0.6503067484662577, 'recall': 0.75177304964539, 'f1-score': 0.6973684210526316, 'support': 141.0}     | {'precision': 0.7481804949053857, 'recall': 0.8018720748829953, 'f1-score': 0.7740963855421686, 'support': 641.0} | {'precision': 0.634337807039757, 'recall': 0.614121108114734, 'f1-score': 0.6240657698056801, 'support': 4079.0}     | {'precision': 0.7280740414279419, 'recall': 0.809407153356198, 'f1-score': 0.7665893271461718, 'support': 2041.0}   | {'precision': 0.8748292349726776, 'recall': 0.8944565691837626, 'f1-score': 0.8845340354815039, 'support': 11455.0} | {'precision': 0.9417344173441734, 'recall': 0.8991913746630728, 'f1-score': 0.9199713198389498, 'support': 9275.0} | 0.8428   | {'precision': 0.7376490471766827, 'recall': 0.7640626654808583, 'f1-score': 0.7497776343836123, 'support': 27909.0}  | {'precision': 0.8442738869920543, 'recall': 0.8428463936364614, 'f1-score': 0.8431306326473246, 'support': 27909.0} |
+| No log        | 7.0   | 287  | 0.4438          | {'precision': 0.5970149253731343, 'recall': 0.5776173285198556, 'f1-score': 0.5871559633027523, 'support': 277.0} | {'precision': 0.6540880503144654, 'recall': 0.7375886524822695, 'f1-score': 0.6933333333333332, 'support': 141.0}   | {'precision': 0.7533039647577092, 'recall': 0.8003120124804992, 'f1-score': 0.7760968229954615, 'support': 641.0} | {'precision': 0.6253114100647733, 'recall': 0.6153468987496935, 'f1-score': 0.6202891387618931, 'support': 4079.0}   | {'precision': 0.7570308898109728, 'recall': 0.8045075943165115, 'f1-score': 0.7800475059382421, 'support': 2041.0}  | {'precision': 0.8764452113891286, 'recall': 0.8867743343518114, 'f1-score': 0.881579518333695, 'support': 11455.0}  | {'precision': 0.9354231280460789, 'recall': 0.9105121293800539, 'f1-score': 0.9227995410588428, 'support': 9275.0} | 0.8432   | {'precision': 0.7426596542508946, 'recall': 0.7618084214686707, 'f1-score': 0.7516145462463172, 'support': 27909.0}  | {'precision': 0.8438834099280034, 'recall': 0.8431688702569063, 'f1-score': 0.8433686534034867, 'support': 27909.0} |
 
 
 ### Framework versions

meta_data/README_s42_e7.md

@@ -0,0 +1,90 @@
+---
+license: apache-2.0
+base_model: allenai/longformer-base-4096
+tags:
+- generated_from_trainer
+datasets:
+- essays_su_g
+metrics:
+- accuracy
+model-index:
+- name: longformer-full_labels
+  results:
+  - task:
+      name: Token Classification
+      type: token-classification
+    dataset:
+      name: essays_su_g
+      type: essays_su_g
+      config: full_labels
+      split: test
+      args: full_labels
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.8431688702569063
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# longformer-full_labels
+
+This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.4438
+- B-claim: {'precision': 0.5970149253731343, 'recall': 0.5776173285198556, 'f1-score': 0.5871559633027523, 'support': 277.0}
+- B-majorclaim: {'precision': 0.6540880503144654, 'recall': 0.7375886524822695, 'f1-score': 0.6933333333333332, 'support': 141.0}
+- B-premise: {'precision': 0.7533039647577092, 'recall': 0.8003120124804992, 'f1-score': 0.7760968229954615, 'support': 641.0}
+- I-claim: {'precision': 0.6253114100647733, 'recall': 0.6153468987496935, 'f1-score': 0.6202891387618931, 'support': 4079.0}
+- I-majorclaim: {'precision': 0.7570308898109728, 'recall': 0.8045075943165115, 'f1-score': 0.7800475059382421, 'support': 2041.0}
+- I-premise: {'precision': 0.8764452113891286, 'recall': 0.8867743343518114, 'f1-score': 0.881579518333695, 'support': 11455.0}
+- O: {'precision': 0.9354231280460789, 'recall': 0.9105121293800539, 'f1-score': 0.9227995410588428, 'support': 9275.0}
+- Accuracy: 0.8432
+- Macro avg: {'precision': 0.7426596542508946, 'recall': 0.7618084214686707, 'f1-score': 0.7516145462463172, 'support': 27909.0}
+- Weighted avg: {'precision': 0.8438834099280034, 'recall': 0.8431688702569063, 'f1-score': 0.8433686534034867, 'support': 27909.0}
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 7
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                                           | B-majorclaim                                                                                                        | B-premise                                                                                                         | I-claim                                                                                                              | I-majorclaim                                                                                                        | I-premise                                                                                                           | O                                                                                                                  | Accuracy | Macro avg                                                                                                            | Weighted avg                                                                                                        |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
+| No log        | 1.0   | 41   | 0.7886          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                              | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                                | {'precision': 0.8888888888888888, 'recall': 0.0748829953198128, 'f1-score': 0.1381294964028777, 'support': 641.0} | {'precision': 0.47000821692686934, 'recall': 0.1402304486393724, 'f1-score': 0.21601208459214502, 'support': 4079.0} | {'precision': 0.5424354243542435, 'recall': 0.0720235178833905, 'f1-score': 0.12716262975778547, 'support': 2041.0} | {'precision': 0.7775630122158652, 'recall': 0.8779572239196858, 'f1-score': 0.8247160605190865, 'support': 11455.0} | {'precision': 0.6536142336038115, 'recall': 0.9466307277628032, 'f1-score': 0.7732957548000705, 'support': 9275.0} | 0.7024   | {'precision': 0.4760728251413826, 'recall': 0.3016749876464378, 'f1-score': 0.29704514658170933, 'support': 27909.0} | {'precision': 0.6651369922726568, 'recall': 0.7024257408004586, 'f1-score': 0.6395296795513288, 'support': 27909.0} |
+| No log        | 2.0   | 82   | 0.5373          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                              | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                                | {'precision': 0.5765472312703583, 'recall': 0.828393135725429, 'f1-score': 0.679897567221511, 'support': 641.0}   | {'precision': 0.5732105732105732, 'recall': 0.47315518509438587, 'f1-score': 0.5183991404781091, 'support': 4079.0}  | {'precision': 0.5972927241962775, 'recall': 0.6918177364037237, 'f1-score': 0.6410896708286039, 'support': 2041.0}  | {'precision': 0.858668504004823, 'recall': 0.870362287210825, 'f1-score': 0.8644758519032342, 'support': 11455.0}   | {'precision': 0.8686365992742353, 'recall': 0.903288409703504, 'f1-score': 0.8856236786469344, 'support': 9275.0}  | 0.7962   | {'precision': 0.4963365188508953, 'recall': 0.538145250591124, 'f1-score': 0.5127837012969133, 'support': 27909.0}   | {'precision': 0.7818058448922789, 'recall': 0.7961947758787488, 'f1-score': 0.7874004427160499, 'support': 27909.0} |
+| No log        | 3.0   | 123  | 0.4911          | {'precision': 0.34893617021276596, 'recall': 0.296028880866426, 'f1-score': 0.3203125, 'support': 277.0}          | {'precision': 0.8333333333333334, 'recall': 0.03546099290780142, 'f1-score': 0.06802721088435375, 'support': 141.0} | {'precision': 0.6662371134020618, 'recall': 0.8065522620904836, 'f1-score': 0.7297106563161608, 'support': 641.0} | {'precision': 0.6018223234624146, 'recall': 0.3238538857563128, 'f1-score': 0.421102964615875, 'support': 4079.0}    | {'precision': 0.7116279069767442, 'recall': 0.7496325330720235, 'f1-score': 0.7301360057265569, 'support': 2041.0}  | {'precision': 0.7889374090247453, 'recall': 0.9463116542994325, 'f1-score': 0.8604881921016073, 'support': 11455.0} | {'precision': 0.9330078346769615, 'recall': 0.8859299191374663, 'f1-score': 0.908859639420418, 'support': 9275.0}  | 0.8066   | {'precision': 0.6977002987270037, 'recall': 0.5776814468757066, 'f1-score': 0.5769481670092816, 'support': 27909.0}  | {'precision': 0.7968542338095115, 'recall': 0.8066215199398044, 'f1-score': 0.7904444769227739, 'support': 27909.0} |
+| No log        | 4.0   | 164  | 0.4471          | {'precision': 0.5464285714285714, 'recall': 0.5523465703971119, 'f1-score': 0.5493716337522441, 'support': 277.0} | {'precision': 0.6544117647058824, 'recall': 0.6312056737588653, 'f1-score': 0.6425992779783394, 'support': 141.0}   | {'precision': 0.7510917030567685, 'recall': 0.8049921996879875, 'f1-score': 0.7771084337349398, 'support': 641.0} | {'precision': 0.6000949893137022, 'recall': 0.619514586908556, 'f1-score': 0.6096501809408926, 'support': 4079.0}    | {'precision': 0.7037037037037037, 'recall': 0.8005879470847623, 'f1-score': 0.7490258996103598, 'support': 2041.0}  | {'precision': 0.8949800652410294, 'recall': 0.8622435617634221, 'f1-score': 0.8783068783068784, 'support': 11455.0} | {'precision': 0.9216195734545848, 'recall': 0.9178436657681941, 'f1-score': 0.9197277441659464, 'support': 9275.0} | 0.8352   | {'precision': 0.7246186244148918, 'recall': 0.7412477436241284, 'f1-score': 0.7322557212128, 'support': 27909.0}     | {'precision': 0.838766973613019, 'recall': 0.835178616216991, 'f1-score': 0.8365729339666597, 'support': 27909.0}   |
+| No log        | 5.0   | 205  | 0.4553          | {'precision': 0.5725490196078431, 'recall': 0.5270758122743683, 'f1-score': 0.5488721804511277, 'support': 277.0} | {'precision': 0.608433734939759, 'recall': 0.7163120567375887, 'f1-score': 0.6579804560260587, 'support': 141.0}    | {'precision': 0.7355021216407355, 'recall': 0.8112324492979719, 'f1-score': 0.7715133531157271, 'support': 641.0} | {'precision': 0.5901240035429584, 'recall': 0.6533464084334396, 'f1-score': 0.6201279813845259, 'support': 4079.0}   | {'precision': 0.7180370210934137, 'recall': 0.8172464478196962, 'f1-score': 0.7644362969752522, 'support': 2041.0}  | {'precision': 0.8847149103239047, 'recall': 0.8655608904408555, 'f1-score': 0.8750330950489806, 'support': 11455.0} | {'precision': 0.9455065827132226, 'recall': 0.8904582210242588, 'f1-score': 0.9171571349250416, 'support': 9275.0} | 0.8339   | {'precision': 0.7221239134088339, 'recall': 0.7544617551468827, 'f1-score': 0.7364457854181019, 'support': 27909.0}  | {'precision': 0.8417519193793559, 'recall': 0.8339245404708159, 'f1-score': 0.8369775321954073, 'support': 27909.0} |
+| No log        | 6.0   | 246  | 0.4431          | {'precision': 0.5860805860805861, 'recall': 0.5776173285198556, 'f1-score': 0.5818181818181819, 'support': 277.0} | {'precision': 0.6503067484662577, 'recall': 0.75177304964539, 'f1-score': 0.6973684210526316, 'support': 141.0}     | {'precision': 0.7481804949053857, 'recall': 0.8018720748829953, 'f1-score': 0.7740963855421686, 'support': 641.0} | {'precision': 0.634337807039757, 'recall': 0.614121108114734, 'f1-score': 0.6240657698056801, 'support': 4079.0}     | {'precision': 0.7280740414279419, 'recall': 0.809407153356198, 'f1-score': 0.7665893271461718, 'support': 2041.0}   | {'precision': 0.8748292349726776, 'recall': 0.8944565691837626, 'f1-score': 0.8845340354815039, 'support': 11455.0} | {'precision': 0.9417344173441734, 'recall': 0.8991913746630728, 'f1-score': 0.9199713198389498, 'support': 9275.0} | 0.8428   | {'precision': 0.7376490471766827, 'recall': 0.7640626654808583, 'f1-score': 0.7497776343836123, 'support': 27909.0}  | {'precision': 0.8442738869920543, 'recall': 0.8428463936364614, 'f1-score': 0.8431306326473246, 'support': 27909.0} |
+| No log        | 7.0   | 287  | 0.4438          | {'precision': 0.5970149253731343, 'recall': 0.5776173285198556, 'f1-score': 0.5871559633027523, 'support': 277.0} | {'precision': 0.6540880503144654, 'recall': 0.7375886524822695, 'f1-score': 0.6933333333333332, 'support': 141.0}   | {'precision': 0.7533039647577092, 'recall': 0.8003120124804992, 'f1-score': 0.7760968229954615, 'support': 641.0} | {'precision': 0.6253114100647733, 'recall': 0.6153468987496935, 'f1-score': 0.6202891387618931, 'support': 4079.0}   | {'precision': 0.7570308898109728, 'recall': 0.8045075943165115, 'f1-score': 0.7800475059382421, 'support': 2041.0}  | {'precision': 0.8764452113891286, 'recall': 0.8867743343518114, 'f1-score': 0.881579518333695, 'support': 11455.0}  | {'precision': 0.9354231280460789, 'recall': 0.9105121293800539, 'f1-score': 0.9227995410588428, 'support': 9275.0} | 0.8432   | {'precision': 0.7426596542508946, 'recall': 0.7618084214686707, 'f1-score': 0.7516145462463172, 'support': 27909.0}  | {'precision': 0.8438834099280034, 'recall': 0.8431688702569063, 'f1-score': 0.8433686534034867, 'support': 27909.0} |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.2.0+cu121
+- Datasets 2.17.0
+- Tokenizers 0.15.2

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dd60df080273f3c94c25d35461b709a413509d1562662829757b8e99ff04826e
+oid sha256:869856a027d05b58aac2546bb5d494fa44c208b65675b409fae0d2aa9376c7c2
 size 592330980