Bilingual spoken flash cards for Google Assistant

Question

I am developing a bilingual (Japanese/English) flashcard app for the Google Assistant so that it would work for Google Home.

The google assistant will say a word in Japanese, and check that the user replies with the correct translation in English, or vice-versa.

Simply starting from Google Sheet data does not work, since the assistant is not able to read the Japanese input. Is there a way to achieve this?

Essentially, the question is, is it possible to alternate between text-to-speech in one language and speech recognition in a different one?

Answer 1

I don't think the Google Assistant can alternate between multiple languages in the middle of a conversation. So far I've only seen one example of a language learning app, see:

https://tech.busuu.com/creating-a-new-busuu-experience-for-the-google-assistant-on-google-home-e62ba200e000

But they seem to have the same problem as you, being restricted by the limited language options of the Assistant. In the Busuu assistant app they have only one language you can learn, which language that is depends on your locale.

The Google Assistant is a conversation between you and Google. You can now prompt the Google Assistant in French (“Parler à busuu ”) or German (“Sprich mit busuu”) and learn English as if you were having a real conversation with a language teacher. You can already prompt in English (“Ok Google, talk to busuu”) to learn Spanish.

As for the problem of outputting speech in one language while the Assistant locale is set to another language they use SSML to play an audio sample containing the word/sentence in the "foreign" language.

So it looks like a 2-way language learning app might not be possible with the current state of the Assistant but I hope this will help you.

Also take look at: https://dialogflow.com/docs/multi-language.