In August 2023, Meta revealed a translation model of AI “all in one” capable of understanding about 100 different languages.
Nicknamed Seamlessm4T (Multilingual and Multimodal Mass Automatic Translation), this is the goaly attempt to create a 'universal translator' similar to Babel fish in the classic science fiction series of Douglas Adams. The Galactapic Auto -Stop Guide.
The team behind the SeamlessM4T tool has detailed its work in an article in the magazine. Naturerevealing that the advanced system offers a solution all in one for text translations to text, voice to text, voice to voice and text to voice in an impressive and growing range of languages.
More than 400 years of unprocessed audio
SeamlessM4T, which is used, among other things, to copy videos automatically on Facebook and Instagram, currently admits voice to voice from 101 to 36 languages, voice translation to text from 101 to 96 languages, translation of text to -Traduction of Text for 96 languages, voice text translation from 96 to 36 languages and automatic voice recognition for 96 languages. This unified approach exceeds the limitations of traditional waterfall systems, which often require separate subsystems for voice recognition, translation and text synthesis to voice.
When optimizing these processes, Meta says that SeamlessM4T surpasses existing models, achieving Bleu scores (bilingual alternate evaluation) up to 23 % higher in translation precision and demonstrating impressive resistance to background noise and the variations of speakers.
To create SeamlessM4T, Meta began with 4 million hours (more than 400 years) of unprocessed multilingual audio from a publicly available web data repository. The team developed Seamlessalign, a multimodal corpus that contains more than 470,000 hours of aligned voice and combined the data set with avant -garde automatic learning techniques, including sound incorporations (multimodal and agnostic representations at the prayer level), which allow multilingual coding and independent of the modality. For text and voice.
Goal says that addressing social and ethical challenges through the use of safeguards, SeamlessM4T can be a valuable tool for global communication. These safeguards reduce gender bias (errors in the grammatical determination of the genre) and mitigate the problem of additional toxicity (where offensive words appear in translations but not in the original source).