Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I select my Airflow or Python version with Cloud Composer?

I am using Cloud Composer and I noticed that it selects the version of Apache Airflow and Python (2.7.x) for me. I want to use a different version of Airflow and/or Python. How can I change this?

like image 393
James Avatar asked May 01 '18 19:05

James


People also ask

How does the airflow web interface work in Cloud Composer?

Each Cloud Composer environment has a web server that runs the Airflow web interface. The web server is a part of Cloud Composer environment architecture. The web server parses the DAG definition files in the dags/ folder and must be able to access a DAG's data and resources to load the DAG and serve HTTP requests.

What version of Python is supported by Cloud Composer?

Supported Python Versions. Cloud Composer supports Python 2 and Python 3. Environments with different Python versions can run in the same Google Cloud Platform project. You can specify the major Python version. For the major version you specify, Cloud Composer installs the latest supported major.minor.patch version.

What do you like most about Cloud Composer?

Maybe the best feature of cloud composer is how easy it is to actually access the AirFlow UI. In the virtual environments I created I’d always have to scroll through lines of output to find a destination URL for the AirFlow UI. In Cloud Composer, the link for an AirFlow UI is provided and very clearly indicated.

Can I change the Airflow version deployed by composer?

New versions of Airflow are usually deployed by Composer within a a few weeks of their stable release. The Airflow version deployed and the Python version installed cannot be changed at this time.


1 Answers

Cloud Composer deploys the latest stable build of Airflow. New versions of Airflow are usually deployed by Composer within a a few weeks of their stable release. The Airflow version deployed and the Python version installed cannot be changed at this time. A future release of Cloud Composer may offer the ability to select the Airflow and/or Python version for new environments.

like image 97
James Avatar answered Oct 13 '22 08:10

James