We know that communication depends more on what you are saying. Just as you say it is often important. That's why Google's latest AI translator's prototype not only translates the words that come from your mouth, but also the tone and rhythm of your voice.
The system is called Translatotron, and Google researchers explain how it works in a recent blog post. They do not say that Translatotron will come in commercial products any time soon, but this can happen in time. As explained by Google's translation chief at The Verge earlier this year, the company's goal at this moment was to add more conviction to its translation tools, creating more awards speech.
You can hear what this sound is in the audio sample below. The first clip is the input; the second is the mainstream; and the third tries to get the voice of the original speaker.
|Translation of translatotron|
|Translation of translatableron with tones|
If you can hear it, it is not a continuous translation, but it is wonderful however. You can listen to many other audio samples from Translatotron here.
Although speaker's hypocrisy is the most impressive of laypeople, the Translatotron's attraction for AI engineers translates speech directly from audio input to audio output without translates this to standard mediated text.
The type AI model is known as an end-to-end system, since no stops for subsidiary activities or actions. Google says that doing a concurrency translation results in faster results while avoiding the risk of entering errors in multiple translation steps.
Perhaps more interesting, the data that the model is processing is not raw audio. Instead, it uses spectrogram data, or detailed sound visualization. In effect, this means that we are translating from one language to another with pictures, which are entertaining.
As usual in Google's translation efforts, there is reason to be skeptical about how such systems will work in the wild. The company often opens ambitious new speeches and translation tools, and they often perform less than anticipated. Still: the future begins, and the AI translation just gets better.