Google Cloud Composer - Deploying Docker Image

Question

Definitely missing something, and could use some quick assistance!

Simply, how do you deploy a Docker image to an Airflow DAG for running jobs? Does anyone have a simple example of deploying a Google container and running it via Airflow/Composer?

Answer 1

You can use the Docker Operator, included in the core Airflow repository.

If pulling an image from a private registry, you'll need to set a connection config with the relevant credentials and pass it to the docker_conn_id param.