medhi-corneille/whisper-tiny-lb

Whisper tiny LB - AKABI

This model is a fine-tuned version of openai/whisper-tiny on the google/fleurs dataset. It achieves the following results on the evaluation set:

• Loss: 1.4215
• Wer Ortho: 62.8649
• Wer: 60.1867

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• training_steps: 4000

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
0.9979	1.37	250	1.5394	73.1448	73.3298
0.6784	2.75	500	1.2998	66.9095	64.8060
0.3773	4.12	750	1.2317	63.9250	61.5385
0.2906	5.49	1000	1.2117	63.0759	60.3958
0.2052	6.87	1250	1.2157	64.1913	62.0685
0.1155	8.24	1500	1.2432	61.6791	59.6130
0.0912	9.62	1750	1.2684	63.0056	60.3229
0.0698	10.99	2000	1.2937	63.6788	60.9598
0.0396	12.36	2250	1.3224	62.7996	60.2451
0.0309	13.74	2500	1.3480	62.1514	59.4622
0.0205	15.11	2750	1.3696	62.1715	59.5303
0.017	16.48	3000	1.3895	62.0761	59.8074
0.0151	17.86	3250	1.4016	62.7745	60.0360
0.0125	19.23	3500	1.4126	62.8900	60.5952
0.012	20.6	3750	1.4202	63.0709	60.3909
0.0115	21.98	4000	1.4215	62.8649	60.1867

Framework versions

• Transformers 4.32.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3