Are there any multilingual unit-based HiFi-GAN vocoders?
#12
by
pilkyu
- opened
If I only change the vocoder to a multilingual one, can it be used for other languages as well?
Besides mhubert_vp_en_es_fr_it3_400k_layer11_km1000_lj, are there any other languages available?
pilkyu
changed discussion title from
Are there any multilingual unit-based HiFi-GAN vocoders
to Are there any multilingual unit-based HiFi-GAN vocoders?
Support the question.🧐
UPD: I think, we can’t just swap in a “multilingual vocoder” and magically get new languages. It only works if the entire unit pipeline matches: the talker must predict the same discrete unit inventory that the vocoder expects, and that unit inventory must actually cover the new languages. That means (1) our unit extractor / k-means codebook and (2) the talker’s output layer must align with (3) the vocoder’s training. If those aren’t aligned, it won’t work.