How to implement an Assistant with Google Assist API

Question

I have been checking out and reading about Google Now on Tap (from http://developer.android.com/training/articles/assistant.html).

It was very interesting to find from that article that Now on Tap is based on Google's Assist API bundled with Marshmallow and it seems possible for us to develop our own assistant (the term Google used in the article to refer to app like Now on Tap) using the API.

However, the mentioned article only very briefly discusses how to use Assist API and I couldn't find any additional information about how to use it to develop a custom assistant even after spending a few days searching for it on the Internet. No documentation and no example.

I was wondering if any of you have experience with Assist API that you could share? Any help appreciated.

Thanks

Answer 1

You can definitely implement a personal assistant just like the Google Now on Tap using the Assist API starting Android 6.0. The official developer (http://developer.android.com/training/articles/assistant.html) guide tells exactly how you should implement it.

Some developers may wish to implement their own assistant. As shown in Figure 2, the active assistant app can be selected by the Android user. The assistant app must provide an implementation of VoiceInteractionSessionService and VoiceInteractionSession as shown in this example and it requires the BIND_VOICE_INTERACTION permission. It can then receive the text and view hierarchy represented as an instance of the AssistStructure in onHandleAssist(). The assistant receives the screenshot through onHandleScreenshot().

Commonsware has four demos for basic Assist API usage. The TapOffNow (https://github.com/commonsguy/cw-omnibus/tree/master/Assist/TapOffNow) should be enough to get you started.

You don't have to use the onHandleScreenshot() to get the relevant textual data, the AssistStructure in onHandleAssist() will give you a root ViewNode which usually contains all you can see on the screen.

You probably need to also implement some sorts of function to quickly locate the specific ViewNode that you want to focus on using recursive search on the children from this root ViewNode.

Answer 2

There is a complete example here but it's too complicated to start. This is my example works on android 7.1.1

AndroidManifest.xml

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.eaydin79.voiceinteraction">
    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:theme="@style/AppTheme" >
        <service 
            android:name="voiceInteractionService"
            android:permission="android.permission.BIND_VOICE_INTERACTION" >
            <meta-data 
                android:name="android.voice_interaction"
                android:resource="@xml/interaction_service" />
            <intent-filter>
                <action android:name="android.service.voice.VoiceInteractionService" />
            </intent-filter>
        </service>
        <service 
            android:name="voiceInteractionSessionService"
            android:permission="android.permission.BIND_VOICE_INTERACTION" >
        </service>
    </application>
</manifest>

this is interaction_service.xml file stored in res\xml folder

<?xml version="1.0" encoding="utf-8"?>
<voice-interaction-service xmlns:android="http://schemas.android.com/apk/res/android"
    android:sessionService="com.eaydin79.voiceinteraction.voiceInteractionSessionService"
    android:recognitionService="com.eaydin79.voiceinteraction.voiceInteractionService"
    android:supportsAssist="true" />

voiceInteractionService.java

package com.eaydin79.voiceinteraction;
import android.service.voice.VoiceInteractionService;
import android.service.voice.VoiceInteractionSession;

public class voiceInteractionService extends VoiceInteractionService {
    @Override
    public void onReady() {
        super.onReady();
    }
}

voiceInteractionSessionService.java

package com.eaydin79.voiceinteraction;
import android.os.Bundle;
import android.service.voice.VoiceInteractionSession;
import android.service.voice.VoiceInteractionSessionService;

public class voiceInteractionSessionService extends VoiceInteractionSessionService {    
    @Override
    public VoiceInteractionSession onNewSession(Bundle bundle) {
         return new voiceInteractionSession(this);
    }
}

voiceInteractionSession.java

package com.eaydin79.voiceinteraction;
import android.app.VoiceInteractor;
import android.content.Context;
import android.os.Bundle;
import android.service.voice.VoiceInteractionSession;
import android.media.AudioManager;

public class voiceInteractionSession extends VoiceInteractionSession {
   
    voiceInteractionSession(Context context) {
        super(context);
    }

    @Override
    public void onShow(Bundle args, int showFlags) {
        super.onShow(args, showFlags);
        //whatever you want to do when you hold the home button 
        //i am using it to show volume control slider
        AudioManager audioManager = (AudioManager) getContext().getSystemService(Context.AUDIO_SERVICE);
        if (audioManager != null) audioManager.adjustStreamVolume(AudioManager.STREAM_MUSIC, AudioManager.ADJUST_SAME, AudioManager.FLAG_SHOW_UI);
        hide();
    }

}