BioHackathon_Lipids-tapt_grouped_llrd-LR_2e-05

This model is a fine-tuned version of microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9814
• Accuracy: 0.7729

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 3407
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.06
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.1166	1.0	29	1.0691	0.7638
1.1241	2.0	58	1.0680	0.7633
1.1143	3.0	87	1.0080	0.7744
1.1051	4.0	116	1.0305	0.7686
1.0955	5.0	145	1.0445	0.7660
1.0724	6.0	174	1.0239	0.7744
1.0624	7.0	203	1.0370	0.7692
1.0677	8.0	232	0.9991	0.7723
1.0773	9.0	261	1.0491	0.7634
1.0402	10.0	290	0.9966	0.7767
1.0531	11.0	319	1.0393	0.7629
1.057	12.0	348	0.9912	0.7764
1.0384	13.0	377	1.0113	0.7719
1.0446	14.0	406	0.9877	0.7779
1.0166	15.0	435	1.0201	0.7726
1.0443	16.0	464	1.0053	0.7710
1.0316	17.0	493	0.9932	0.7750
1.0414	18.0	522	1.0127	0.7756
1.0125	19.0	551	0.9969	0.7753
1.0228	20.0	580	0.9773	0.7771
1.0185	21.0	609	0.9927	0.7705
1.0141	22.0	638	0.9924	0.7717
0.9937	23.0	667	1.0443	0.7690
0.9783	24.0	696	1.0183	0.7721
0.9984	25.0	725	0.9907	0.7765
0.9976	26.0	754	0.9765	0.7751
0.9883	27.0	783	1.0047	0.7697
0.9853	28.0	812	1.0013	0.7693
0.9915	29.0	841	0.9874	0.7762
0.9716	30.0	870	0.9950	0.7758
0.9682	31.0	899	0.9853	0.7744
0.9426	32.0	928	0.9986	0.7688
0.9499	33.0	957	1.0205	0.7718
0.9462	34.0	986	1.0001	0.7735
0.943	35.0	1015	0.9643	0.7783
0.9302	36.0	1044	0.9802	0.7765
0.9462	37.0	1073	0.9950	0.7748
0.9501	38.0	1102	0.9890	0.7727
0.9463	39.0	1131	1.0448	0.7618
0.9475	40.0	1160	1.0014	0.7775
0.9476	41.0	1189	0.9993	0.7746
0.924	42.0	1218	1.0299	0.7667
0.9262	43.0	1247	1.0259	0.7698
0.9337	44.0	1276	1.0585	0.7629
0.9121	45.0	1305	1.0140	0.7732
0.9068	46.0	1334	1.0237	0.7739
0.903	47.0	1363	1.0173	0.7706
0.93	48.0	1392	1.0091	0.7681
0.9286	49.0	1421	0.9986	0.7685
0.9034	50.0	1450	1.0186	0.7724
0.8983	51.0	1479	0.9912	0.7748
0.8936	52.0	1508	1.0157	0.7693
0.9052	53.0	1537	0.9931	0.7754
0.8906	54.0	1566	1.0099	0.7707
0.8781	55.0	1595	0.9913	0.7722
0.8954	56.0	1624	1.0015	0.7755
0.8734	57.0	1653	0.9900	0.7755
0.8701	58.0	1682	1.0097	0.7706
0.8732	59.0	1711	0.9912	0.7749
0.8618	60.0	1740	0.9636	0.7798
0.872	61.0	1769	0.9952	0.7694
0.8635	62.0	1798	1.0329	0.7668
0.8654	63.0	1827	1.0099	0.7689
0.8739	64.0	1856	0.9887	0.7753
0.8657	65.0	1885	1.0044	0.7741
0.8629	66.0	1914	0.9935	0.7722
0.8645	67.0	1943	0.9585	0.7788
0.8646	68.0	1972	0.9810	0.7743
0.8605	69.0	2001	1.0097	0.7668
0.8569	70.0	2030	0.9732	0.7735
0.8636	71.0	2059	1.0018	0.7715
0.8683	72.0	2088	0.9725	0.7765
0.8818	73.0	2117	0.9742	0.7776
0.8476	74.0	2146	1.0173	0.7740
0.8457	75.0	2175	0.9999	0.7736
0.8591	76.0	2204	1.0032	0.7736
0.8641	77.0	2233	0.9888	0.7788
0.8441	78.0	2262	0.9899	0.7732
0.8493	79.0	2291	0.9808	0.7793
0.8735	80.0	2320	0.9472	0.7807
0.8508	81.0	2349	1.0416	0.7729
0.8582	82.0	2378	0.9904	0.7741
0.8453	83.0	2407	0.9928	0.7730
0.8426	84.0	2436	0.9761	0.7735
0.8628	85.0	2465	1.0195	0.7714
0.8529	86.0	2494	0.9920	0.7749
0.8533	87.0	2523	1.0019	0.7701
0.8477	88.0	2552	1.0178	0.7682
0.8328	89.0	2581	1.0003	0.7744
0.8427	90.0	2610	0.9917	0.7755
0.8473	91.0	2639	0.9900	0.7743
0.8518	92.0	2668	0.9994	0.7677
0.8307	93.0	2697	0.9798	0.7764
0.857	94.0	2726	0.9803	0.7759
0.8394	95.0	2755	1.0033	0.7707
0.845	96.0	2784	1.0312	0.7672
0.8586	96.5614	2800	0.9814	0.7729

Framework versions

• Transformers 4.48.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.2
• Tokenizers 0.21.0