Is there an API or SDK to create a remote control application on SmartPhone for Android TV

Question

I have been tasked to create an application for android mobile to control an Android TV, preferably the dashboard/landingpage outside of any apps (settings included). It doesn't really matter if it's via bluetooth or wifi, although I have found that bluetooth is not possible as the HID profile is needed, and that profile is only available on API 28 (I need to support from API 19 up)

There are some apps on the play store that already have this functionality. Most connect via Wifi to the Android TV, also pairing with it.

By analysing the APK Files I found out some options, i.e.

• some use the connectSDK library

• others use what seems to be a native google package that I can't seem to find

  import com.google.android.tv.support.remote.Discovery;
  import com.google.android.tv.support.remote.core.Client;
  import com.google.android.tv.remote.BuildInfo;
  

I found that a couple of years ago the Anymote Protocol could be used as well, but that one only works with Google TV, not Android TV.

The problems I am facing right now is that the connectSDK library isn't being maintained and does not contain any code for Android TV connections. The native google package cannot be found anywhere, not sure if it's included in a specific Jar file, or maybe some obscured/hidden dependency?

I could try to create a connection to a specific socket with Android TV, I know for example that the ServiceType is "_androidtvremote._tcp." and that the port number is 6466. But I'm not sure what would be the best way to implement this.

What I'm looking for are some pointers or ideas how I could tackle this problem. Maybe some references as well.

Answer 1

EDIT on December 2021: I created a new documentation for the new protocol v2.

---

EDIT on September 2021: Google is deploying a new version of the "Android TV Remote Control" (from v4.x to v5), and this version is not compatible with the legacy pairing system. For now it's necessary to keep a version < 5 to make it work.

---

We spent some time to find how to connect and control an Android/Google TV (by reverse engineering), and I'm sharing here the result of our findings. For a more recent/updated version, you can check this wiki page.

I develop in PHP so I'll share the code in PHP (the Java code can be found by decompiling some Android apps using https://github.com/skylot/jadx)

---

Thanks to @hubertlejaune for his tremendous help.

The Android TV (aka server in this document) should have 2 open ports: 6466 and 6467.

To know more about the Android TV, we can enter the below Linux command:

openssl s_client -connect SERVER_IP:6467 -prexit -state -debug

Which will return some information, including the server's public certificate.

If you only want the server's public certificate:

openssl s_client -showcerts -connect SERVER_IP:6467 </dev/null 2>/dev/null|openssl x509 -outform PEM > server.pem

Pairing

The pairing protocol will happen on port 6467.

Client's certificate

It's required to generate our own (client) certificate.

In PHP we can do it with the below code:

<?php
// the commande line is: php generate_key.php > client.pem

// certificate details (Distinguished Name)
// (OpenSSL applies defaults to missing fields)
$dn = array(
  "commonName" => "atvremote",
  "countryName" => "US",
  "stateOrProvinceName" => "California",
  "localityName" => "Montain View",
  "organizationName" => "Google Inc.",
  "organizationalUnitName" => "Android",
  "emailAddress" => "[email protected]"
);

// create certificate which is valid for ~10 years
$privkey = openssl_pkey_new();
$cert = openssl_csr_new($dn, $privkey);
$cert = openssl_csr_sign($cert, null, $privkey, 3650);

// export public key
openssl_x509_export($cert, $out);
echo $out;

// export private key
$passphrase = null;
openssl_pkey_export($privkey, $out, $passphrase);
echo $out;

It will generate a file called client.pem that contains both the public and the private keys for our client.

Connection to the server

You need to open a TLS/SSL connection to the server using port 6467.

In PHP, you could use https://github.com/reactphp/socket:

<?php
use React\EventLoop\Factory;
use React\Socket\Connector;
use React\Socket\SecureConnector;
use React\Socket\ConnectionInterface;

require __DIR__ . '/./vendor/autoload.php';

$host = 'SERVER_IP';
$loop = Factory::create();
$tcpConnector = new React\Socket\TcpConnector($loop);
$dnsResolverFactory = new React\Dns\Resolver\Factory();
$dns = $dnsResolverFactory->createCached('8.8.8.8', $loop);
$dnsConnector = new React\Socket\DnsConnector($tcpConnector, $dns);

$connector = new SecureConnector($dnsConnector, $loop, array(
  'allow_self_signed' => true,
  'verify_peer' => false,
  'verify_peer_name' => false,
  'dns' => false,
  'local_cert' => 'client.pem'
));

$connector->connect('tls://' . $host . ':6467')->then(function (ConnectionInterface $connection) use ($host) {
  $connection->on('data', function ($data) use ($connection) {
    $dataLen = strlen($data);
    echo "data recv => ".$data." (".strlen($data).")\n";
    // deal with the messages received from the server
  });
  
  // below we can send the first message
  $connection->write(/* first message here */);
}, 'printf');

$loop->run();
?>

Protocol

⚠️ Attention, each message is sent as a JSON string, but with two components/parts:

• (first) we send the length of the message (JSON string) on 4 bytes,
• (second) we send the message (JSON string) itself.

PAIRING_REQUEST(10)

As soon as we are connected to the server, we send a PAIRING_REQUEST(10) message (type = 10).

The first message to send is:

{"protocol_version":1,"payload":{"service_name":"androidtvremote","client_name":"CLIENT_NAME"},"type":10,"status":200}

The server returns a PAIRING_REQUEST_ACK(11) message with type is 11 and status is 200:

{"protocol_version":1,"payload":{},"type":11,"status":200}

OPTIONS(20)

Then the client replies with a OPTIONS(20) message (type = 20):

{"protocol_version":1,"payload":{"output_encodings":[{"symbol_length":4,"type":3}],"input_encodings":[{"symbol_length":4,"type":3}],"preferred_role":1},"type":20,"status":200}

The server returns a OPTIONS(20) message with type is 20 and status is 200.

CONFIGURATION(30)

Then the client replies with a CONFIGURATION(30) message (type = 30):

{"protocol_version":1,"payload":{"encoding":{"symbol_length":4,"type":3},"client_role":1},"type":30,"status":200}

The server returns a CONFIGURATION_ACK(31) message with type is 31 and status is 200.

🎉 The code appears on the TV screen!

SECRET(40)

Then the client replies with a SECRET(40) message (type = 40):

{"protocol_version":1,"payload":{"secret":"encodedSecret"},"type":40,"status":200}

At this stage, the TV screen shows a code with 4 characters (e.g. 4D35).

To find the encodedSecret:

• we use a SHA-256 hash;
• we add the client public key's modulus to the hash;
• we add the client public key's exponent to the hash;
• we add the server public key's modulus to the hash;
• we add the server public key's exponent to the hash;
• we add the last 2 characters of the code to the hash (in the example it's 35).

The result of the hash is then encoded in base64.

The server returns a SECRET_ACK(41) message with type is 41 and status is 200, as well as an encoded secret that permits to verify – we didn't try to decode it, but it's probably the first 2 characters of the code:

{"protocol_version":1,"payload":{"secret":"encodedSecretAck"},"type":41,"status":200}

PHP Code

(you can find some Java code that produces pretty much the same)

Here is the related PHP code:

<?php
use React\EventLoop\Factory;
use React\Socket\Connector;
use React\Socket\SecureConnector;
use React\Socket\ConnectionInterface;

require __DIR__ . '/./vendor/autoload.php';

$host = 'SERVER_IP';
$loop = Factory::create();
$tcpConnector = new React\Socket\TcpConnector($loop);
$dnsResolverFactory = new React\Dns\Resolver\Factory();
$dns = $dnsResolverFactory->createCached('8.8.8.8', $loop);
$dnsConnector = new React\Socket\DnsConnector($tcpConnector, $dns);

// get the server's public certificate
exec("openssl s_client -showcerts -connect ".escapeshellcmd($host).":6467 </dev/null 2>/dev/null|openssl x509 -outform PEM > server.pem");

$connector = new SecureConnector($dnsConnector, $loop, array(
  'allow_self_signed' => true,
  'verify_peer' => false,
  'verify_peer_name' => false,
  'dns' => false,
  'local_cert' => 'client.pem'
));

// return the message's length on 4 bytes
function getLen($len) {
  return chr($len>>24 & 0xFF).chr($len>>16 & 0xFF).chr($len>>8 & 0xFF).chr($len & 0xFF);
}

// connect to the server
$connector->connect('tls://' . $host . ':6467')->then(function (ConnectionInterface $connection) use ($host) {
  $connection->on('data', function ($data) use ($connection) {
    $dataLen = strlen($data);
    echo "data recv => ".$data." (".strlen($data).")\n";

    // the first response from the server is the message's size on 4 bytes (that looks like a char to convert to decimal) – we can ignore it
    // only look at messages longer than 4 bytes
    if ($dataLen > 4) {
      // decode the JSON string
      $res = json_decode($data);
      // check the status is 200
      if ($res->status === 200) {
        // check at which step we are
        switch($res->type) {
          case 11:{
            // message to send:
            // {"protocol_version":1,"payload":{"output_encodings":[{"symbol_length":4,"type":3}],"input_encodings":[{"symbol_length":4,"type":3}],"preferred_role":1},"type":20,"status":200}
            $json = new stdClass();
            $json->protocol_version = 1;
            $json->payload = new stdClass();
            $json->payload->output_encodings = [];
            $encoding = new stdClass();
            $encoding->symbol_length = 4;
            $encoding->type = 3;
            array_push($json->payload->output_encodings, $encoding);
            $json->payload->input_encodings = [];
            $encoding = new stdClass();
            $encoding->symbol_length = 4;
            $encoding->type = 3;
            array_push($json->payload->input_encodings, $encoding);
            $json->payload->preferred_role = 1;
            $json->type = 20;
            $json->status = 200;
            $payload = json_encode($json);
            $payloadLen = strlen($payload);
            $connection->write(getLen($payloadLen));
            $connection->write($payload);
            break;
          }
          case 20:{
            // message to send:
            // {"protocol_version":1,"payload":{"encoding":{"symbol_length":4,"type":3},"client_role":1},"type":30,"status":200}
            $json = new stdClass();
            $json->protocol_version = 1;
            $json->payload = new stdClass();
            $json->payload->encoding = new stdClass();
            $json->payload->encoding->symbol_length = 4;
            $json->payload->encoding->type = 3;
            $json->payload->client_role = 1;
            $json->type = 30;
            $json->status = 200;
            $payload = json_encode($json);
            $payloadLen = strlen($payload);
            $connection->write(getLen($payloadLen));
            $connection->write($payload);
            break;
          }
          case 31:{
            // when we arrive here, the TV screen displays a code with 4 characters
            // message to send:
            // {"protocol_version":1,"payload":{"secret":"encodedSecret"},"type":40,"status":200}
            $json = new stdClass();
            $json->protocol_version = 1;
            $json->payload = new stdClass();
            // get the code... here we'll let the user to enter it in the console
            $code = readline("Code: ");

            // get the client's certificate
            $clientPub = openssl_get_publickey(file_get_contents("client.pem"));
            $clientPubDetails = openssl_pkey_get_details($clientPub);
            // get the server's certificate
            $serverPub = openssl_get_publickey(file_get_contents("public.key"));
            $serverPubDetails = openssl_pkey_get_details($serverPub);

            // get the client's certificate modulus
            $clientModulus = $clientPubDetails['rsa']['n'];
            // get the client's certificate exponent
            $clientExponent = $clientPubDetails['rsa']['e'];
            // get the server's certificate modulus
            $serverModulus = $serverPubDetails['rsa']['n'];
            // get the server's certificate exponent
            $serverExponent = $serverPubDetails['rsa']['e'];

            // use SHA-256
            $ctxHash = hash_init('sha256');
            hash_update($ctxHash, $clientModulus);
            hash_update($ctxHash, $clientExponent);
            hash_update($ctxHash, $serverModulus);
            hash_update($ctxHash, $serverExponent);
            // only keep the last two characters of the code
            $codeBin = hex2bin(substr($code, 2));
            hash_update($ctxHash, $codeBin);
            $alpha = hash_final($ctxHash, true);
            
            // encode in base64
            $json->payload->secret = base64_encode($alpha);
            $json->type = 40;
            $json->status = 200;
            $payload = json_encode($json);
            $payloadLen = strlen($payload);

            $connection->write(getLen($payloadLen));
            $connection->write($payload);
            break;
          }
        }
      }
    }
  });

  // send the first message to the server
  // {"protocol_version":1,"payload":{"service_name":"androidtvremote","client_name":"TEST"},"type":10,"status":200}
  $json = new stdClass();
  $json->protocol_version = 1;
  $json->payload = new stdClass();
  $json->payload->service_name = "androidtvremote";
  $json->payload->client_name = "interface Web";
  $json->type = 10;
  $json->status = 200;
  $payload = json_encode($json);
  $payloadLen = strlen($payload);

  // send the message size
  $connection->write(getLen($payloadLen));
  // send the message
  $connection->write($payload);
}, 'printf');

$loop->run();
?>

Send Commands

Now that the client is paired with the server, we'll use port 6466 to send the commands.
Please, note we'll use an array of bytes for the commands.

Configuration message

An initial message must be sent:

[1,0,0,21,0,0,0,1,0,0,0,1,32,3,0,0,0,0,0,0,4,116,101,115,116]

The server will respond with an array of bytes that should start with [1,7,0

Commands

You must send two messages to execute one command.

The format is:

[1,2,0,{SIZE=16},0,0,0,0,0,0,0, {COUNTER} ,0,0,0, {PRESS=0} ,0,0,0,{KEYCODE}]
[1,2,0,{SIZE=16},0,0,0,0,0,0,0,{COUNTER+1},0,0,0,{RELEASE=1},0,0,0,{KEYCODE}]

The {KEYCODE} can be found on https://developer.android.com/reference/android/view/KeyEvent.

For example, if we want to send a VOLUME_UP:

[1,2,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,24]
[1,2,0,16,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,24]

PHP Code

And here some PHP code:

<?php
use React\EventLoop\Factory;
use React\Socket\Connector;
use React\Socket\SecureConnector;
use React\Socket\ConnectionInterface;

require __DIR__ . '/./vendor/autoload.php';

$host = 'SERVER_IP';
$loop = Factory::create();
$tcpConnector = new React\Socket\TcpConnector($loop);
$dnsResolverFactory = new React\Dns\Resolver\Factory();
$dns = $dnsResolverFactory->createCached('8.8.8.8', $loop);
$dnsConnector = new React\Socket\DnsConnector($tcpConnector, $dns);

$connector = new SecureConnector($dnsConnector, $loop, array(
  'allow_self_signed' => true,
  'verify_peer' => false,
  'verify_peer_name' => false,
  'dns' => false,
  'local_cert' => 'client.pem'
));

// convert the array of bytes
function toMsg($arr) {
  $chars = array_map("chr", $arr);
  return join($chars);
}

// connect to the server
$connector->connect('tls://' . $host . ':6466')->then(function (ConnectionInterface $connection) use ($host) {
  $connection->on('data', function ($data) use ($connection) {
    // convert the data received to an array of bytes
    $dataLen = strlen($data);
    $arr = [];
    for ($i=0; $i<$dataLen;$i++) {
      $arr[] = ord($data[$i]);
    }
    $str = "[".implode(",", $arr)."]";
    echo "data recv => ".$data." ".$str." (".strlen($data).")\n";

    // if we receive [1,20,0,0] it means the server sent a ping
    if (strpos($str, "[1,20,0,0]") === 0) {
      // we can reply with a PONG [1,21,0,0] if we want
      // $connection->write(toMsg([1,21,0,0]));
    }
    else if (strpos($str, "[1,7,0,") === 0) {
      // we can send the command, here it's a VOLUME_UP
      $connection->write(toMsg([1,2,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,24]));
      $connection->write(toMsg([1,2,0,16,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,24]));
    }
  });

  // send the first message (configuration) to the server
  $arr = [1,0,0,21,0,0,0,1,0,0,0,1,32,3,0,0,0,0,0,0,4,116,101,115,116];
  $connection->write(toMsg($arr));
}, 'printf');

$loop->run();
?>

Answer 2

So, I found the answer what I was looking for.

If you are a Google Partner (and only then), and have an account with those privileges, you can simply download the jar file at this location. Documentation can be found there as well and the SDK exists for Android and iOS.

Not much information is available how to use it. But by looking over the different classes it can become clear.