Can you get a static external IP address for Google Cloud Composer / Airflow?

Question

I know how to assign a static external IP address to a Compute Engine, but can this be done with Google Cloud Composer (Airflow)? I'd imagine most companies need that functionality since they'd generally be writing back to a warehouse that is possibly behind a firewall, but I can't find any doc's on how to do this.

Answer 1

Now is possible. You will need to configure the Cloud Composer in a private VPC and expose the cluster using a Cloud NAT.

The documentation has some information on how to do this: https://cloud.google.com/composer/docs/concepts/private-ip

Answer 2

It's not possible to assign a static IP to the underlying GKE cluster in a Composer environment. The endpoint @kaxil mentioned is the Kubernetes master endpoint but not the GKE nodes.

If the intent is to let all outgoing network connections from Composer tasks use the same external IP, consider setting up a GCE VM with a static IP and adding a proxy service for all workloads running inside your Composer GKE cluster.

More information: https://groups.google.com/forum/#!topic/kubernetes-users/zNytc8GVB5s.