How to inspect clients that are connected to a GRPC server

Question

To provide better debugging information for my GRPC server/client setup, I am trying to find an API for grpc.server that allows me to inspect what clients are connected to the server.

The most promising question I have found is question, which gives a starting point on how to do this in Java GRPC. However, the Java API doesn't exist in the Python GRPC implementation.

So far, I keep track of unique peers using the context.peer() method in a grpc.ServicerContext. If a peer hasn't sent a request in a while (a timeout I set to 2 seconds) I assume the client has disconnected.

I've started looking at the python-grpc source code but I haven't made any headway.

If anyone knows of a similar API in python I could use, that would be appreciated! Even an internal API would be sufficient for these debugging needs.

Answer 1

I found some more documentation and examples of channelz, other people have been suggesting it and it seems like what you want.

Here's a pull request that gives an example of channelz being used. It only uses the GetServer channelz API, so you'll have to adapt it.

And here's a unit test that uses channelz, which tests APIs that are probably relevant GetTopChannels API.

Answer 2

There isn't a native API for this, but you have all of the pieces you need. Here's a modified version of the helloworld example from the repo.

class PeerSet(object):
    def __init__(self):
        self._peers_lock = threading.RLock()
        self._peers = {}

    def connect(self, peer):
        print("Peer {} connecting".format(peer))
        with self._peers_lock:
            if peer not in self._peers:
                self._peers[peer] = 1
            else:
                self._peers[peer] += 1

    def disconnect(self, peer):
        print("Peer {} disconnecting".format(peer))
        with self._peers_lock:
            if peer not in self._peers:
                raise RuntimeError("Tried to disconnect peer '{}' but it was never connected.".format(peer))
            self._peers[peer] -= 1
            if self._peers[peer] == 0:
                del self._peers[peer]

    def peers(self):
        with self._peers_lock:
            return self._peers.keys()


class Greeter(helloworld_pb2_grpc.GreeterServicer):

    def __init__(self):
        self._peer_set = PeerSet()

    def _record_peer(self, context):
        def _unregister_peer():
            self._peer_set.disconnect(context.peer())
        context.add_callback(_unregister_peer)
        self._peer_set.connect(context.peer())

    def SayHello(self, request, context):
        self._record_peer(context)
        for i in range(10):
            print("[thread {}] Peers: {}".format(threading.currentThread().ident, self._peer_set.peers()))
            time.sleep(1)
        return helloworld_pb2.HelloReply(message='Hello, %s!' % request.name)

This will get you output like the following:

[thread 139905506195200] Peers: [u'ipv6:[::1]:57940', u'ipv6:[::1]:57930', u'ipv6:[::1]:57926', u'ipv6:[::1]:57920', u'ipv6:[::1]:57934']

As Ding commented above, channelz may also be a good fit for you.

Answer 3

There is one tool grpcdebug, it could inspect clients that are connected to a GRPC server.

grpcdebug is a command line interface focusing on simplifying the debugging process of gRPC applications. grpcdebug fetches the internal states of the gRPC library from the application via gRPC protocol and provide a human-friendly UX to browse them. Currently, it supports Channelz/Health Checking/CSDS (aka. admin services)

grpcdebug is an gRPC service admin CLI

Usage:
  grpcdebug <target address> [flags] <command>

Available Commands:
  channelz    Display gRPC states in human readable way.
  health      Check health status of the target service (default "").
  help        Help about any command
  xds         Fetch xDS related information.