How to listen to firestore through RPC

Question

I want to listen to real time changes in firestore and I am also only allowed to use Go. Since firestore SDK for Go doesn't have any option to listen for real time changes, I decided to use the firestore v1beta1 sdk.

I have written the following code to do that

func TestRPCHandler(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
c, err := firestore.NewClient(context.Background())

databaseName := "projects/[project_name]/databases/(default)"
if err != nil {
    panic(err)
}

stream, err := client.Listen(context.Background())
if err != nil {
    panic(err)
}

request := &firestorepb.ListenRequest{
    Database:             databaseName,
    TargetChange:         &firestorepb.ListenRequest_AddTarget{
        AddTarget: &firestorepb.Target{
            TargetType:           &firestorepb.Target_Documents{
                Documents: &firestorepb.Target_DocumentsTarget{
                    Documents:    []string{"projects/[project_name]/databases/(default)/[collection_name]"} ,
                },
            },
        },
    },
}

if err := stream.Send(request); err != nil {
    panic(err)
}

if err := stream.CloseSend(); err != nil {
    panic(err)
}

for {
    resp, err := stream.Recv()
    if err == io.EOF {
        break
    }
    if err != nil {
        panic(err)
    }
}

   }

When I am doing this, the code does not detect any changes I bring about manually in the database. stream.Recv() just returns EOF and exits immediately. I even tried manually waiting by adding time.Sleep() but that does not help either.

Answer 1

You don't need the beta SDK or hacks to make this happen, I found the solution, it's pretty easy actually.

The https://firebase.google.com/docs/firestore/query-data/listen documentation does not contain an example for Go.

The source code of the firestore client API for Go has an unexported watchStream which we cannot directly use: https://github.com/googleapis/google-cloud-go/blob/master/firestore/watch.go#L130

Deep search of the repository shows that this is actually used on the DocumentSnapshotIterator and QuerySnapshotIterator at: https://github.com/googleapis/google-cloud-go/blob/master/firestore/docref.go#L644 and: https://github.com/googleapis/google-cloud-go/blob/master/firestore/query.go#L716.

The Collection contains a Snapshots method which returns the snapshot iterator that we want, after that all is easy, we just make an infivitive loop through its Next method.

Example:

cols, err := client.Collections(context.Background()).GetAll()

for _, col := range cols {
    iter := col.Snapshots(context.Background())
    defer iter.Stop()

    for {
        doc, err := iter.Next()
        if err != nil {
            if err == iterator.Done {
                break
            }
            return err
        }

        for _, change := range doc.Changes {
            // access the change.Doc returns the Document,
            // which contains Data() and DataTo(&p) methods.
            switch change.Kind {
            case firestore.DocumentAdded:
                // on added it returns the existing ones.
                isNew := change.Doc.CreateTime.After(l.startTime)
                // [...]
            case firestore.DocumentModified:
                // [...]
            case firestore.DocumentRemoved:
                // [...]
            }
        }
    }
}

Yours, Gerasimos Maropoulos aka @kataras