How to log custom metrics in Google Container Engine to Stackdriver?

Question

I'm currently exploring more and more of Google Cloud Platform and try to evaluate how I can change my default stack. I like the idea of using all the built-in tooling instead of having to go to various Saas products. One of the more open questions has been how to make use of Stackdriver Monitoring. I understand that the built-in monitoring uses Heapster with the Google Cloud Sink. However, how do I expose my own custom metrics to Stackdriver for use? I don't mean the logs-based metrics, but the real thing?

I know I could always install the agent myself onto the cluster, but it appears to me that as part of Google Container Engine, it is already running, so how could I push metrics to it?

Answer 1

The fluentd pod pre-installed in GKE is in kube-system namespace and managed by k8s, so you shouldn't mess with it. Currently it seems only proxying stdout/stderr to Stackdriver. But according to this post, k8s team is working on providing accessible node-wide fluentd port out of the box.

This document has details about setting up custom logging.