Airflow installation failure beam[gcp]

Question

Airflow installation with command is failing

sudo pip3 install apache-airflow[gcp_api]

Everything was working fine yesterday. Today I see the following error:

Could not find a version that satisfies the requirement apache-beam[gcp]==2.3.0 (from google-cloud-dataflow->apache-airflow[gcp_api]) (from versions: 0.6.0, 2.0.0, 2.1.0, 2.1.1, 2.2.0)
No matching distribution found for apache-beam[gcp]==2.3.0 (from google-cloud-dataflow->apache-airflow[gcp_api])

Can someone help me on this?

Thanks in advance

Answer 1

I faced the same problem :(

Why? most likely it happened because in the new version(2.3.0) of apache-beam they actually added the restriction for python3 https://pypi.python.org/pypi/apache-beam/2.3.0 Requires Python: >=2.7,<3.0 the previous packages didn't have this restriction, that is why it was working before(if you didn't use dataflow from gcp).

• probably you have the last version of https://pypi.python.org/pypi/google-cloud-dataflow/2.3.0 which has updated apache-beam package

How to fix?

uninstall google-cloud-dataflow

pip3 uninstall google-cloud-dataflow

and install version 2.2.0 which has the old version of apache beam

pip install google-cloud-dataflow==2.2.0

Fixed this problem for me, I hope it will help you as well

Answer 2

This has been resolved in the master branch at Apache Airflow Github by Pull Request #3273 .

You can install the latest development branch using the below commands.

pip install git+https://github.com/apache/incubator-airflow
pip install git+https://github.com/apache/incubator-airflow#egg=apache-airflow[gcp_api]