roberta_large_hostel_ner

This model is a fine-tuned version of FacebookAI/roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.0480
• Precision: 0.6916
• Recall: 0.7347
• F1: 0.7125
• Accuracy: 0.8223

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50.0

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	307	0.6049	0.5460	0.6836	0.6071	0.8031
0.7077	2.0	614	0.5622	0.5902	0.7044	0.6423	0.8194
0.7077	3.0	921	0.6149	0.6159	0.7155	0.6620	0.8174
0.3967	4.0	1228	0.6661	0.5917	0.7185	0.6490	0.8112
0.2371	5.0	1535	0.7497	0.6154	0.7145	0.6612	0.8126
0.2371	6.0	1842	0.8418	0.6138	0.7293	0.6666	0.8059
0.1496	7.0	2149	0.8446	0.6258	0.7231	0.6710	0.8190
0.1496	8.0	2456	0.9823	0.6399	0.7229	0.6789	0.8150
0.1073	9.0	2763	0.9789	0.6372	0.7235	0.6776	0.8163
0.0792	10.0	3070	1.0675	0.6607	0.7254	0.6915	0.8219
0.0792	11.0	3377	1.1495	0.6471	0.7306	0.6863	0.8129
0.0584	12.0	3684	1.1720	0.6313	0.7254	0.6751	0.8122
0.0584	13.0	3991	1.2905	0.6484	0.7246	0.6844	0.8080
0.0476	14.0	4298	1.3109	0.6515	0.7258	0.6867	0.8143
0.0321	15.0	4605	1.3268	0.6500	0.7256	0.6857	0.8123
0.0321	16.0	4912	1.4593	0.6482	0.7218	0.6830	0.8089
0.027	17.0	5219	1.4810	0.6559	0.7268	0.6895	0.8117
0.0242	18.0	5526	1.4636	0.6321	0.7193	0.6729	0.8098
0.0242	19.0	5833	1.5093	0.6640	0.7301	0.6955	0.8187
0.0188	20.0	6140	1.4944	0.6690	0.7240	0.6954	0.8178
0.0188	21.0	6447	1.5568	0.6550	0.7232	0.6874	0.8155
0.0164	22.0	6754	1.6352	0.6786	0.7215	0.6994	0.8176
0.0118	23.0	7061	1.6460	0.6674	0.7327	0.6985	0.8188
0.0118	24.0	7368	1.6089	0.6781	0.7300	0.7031	0.8223
0.0112	25.0	7675	1.7131	0.6635	0.7340	0.6970	0.8162
0.0112	26.0	7982	1.7572	0.6759	0.7313	0.7025	0.8185
0.0083	27.0	8289	1.7329	0.6726	0.7228	0.6968	0.8197
0.006	28.0	8596	1.8310	0.6684	0.7337	0.6995	0.8172
0.006	29.0	8903	1.8690	0.6692	0.7368	0.7014	0.8162
0.0059	30.0	9210	1.9132	0.6785	0.7283	0.7025	0.8173
0.0049	31.0	9517	1.8567	0.6856	0.7294	0.7068	0.8223
0.0049	32.0	9824	1.9176	0.6773	0.7320	0.7036	0.8217
0.0044	33.0	10131	1.9170	0.6843	0.7340	0.7083	0.8214
0.0044	34.0	10438	1.9416	0.6810	0.7371	0.7080	0.8196
0.004	35.0	10745	1.8975	0.6654	0.7332	0.6977	0.8215
0.0038	36.0	11052	1.9453	0.6877	0.7373	0.7116	0.8177
0.0038	37.0	11359	1.9305	0.6787	0.7342	0.7054	0.8179
0.002	38.0	11666	1.9255	0.6745	0.7313	0.7017	0.8202
0.002	39.0	11973	1.9737	0.6816	0.7329	0.7063	0.8196
0.0016	40.0	12280	1.9903	0.6838	0.7339	0.7080	0.8190
0.0018	41.0	12587	1.9903	0.6882	0.7365	0.7115	0.8224
0.0018	42.0	12894	1.9753	0.6802	0.7364	0.7072	0.8228
0.001	43.0	13201	2.0004	0.6904	0.7345	0.7118	0.8222
0.0007	44.0	13508	2.0058	0.6825	0.7357	0.7081	0.8225
0.0007	45.0	13815	2.0355	0.6874	0.7357	0.7107	0.8228
0.0006	46.0	14122	2.0481	0.6912	0.7346	0.7122	0.8226
0.0006	47.0	14429	2.0460	0.6900	0.7338	0.7112	0.8220
0.0004	48.0	14736	2.0553	0.6911	0.7364	0.7130	0.8224
0.0003	49.0	15043	2.0499	0.6918	0.7346	0.7125	0.8224
0.0003	50.0	15350	2.0480	0.6916	0.7347	0.7125	0.8223

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 3.1.0
• Tokenizers 0.21.0