Issues after Apache Airflow migration from 1.9.0 to 1.10.1

Question

I just upgraded my Airflow install from 1.9.0 to 1.10.1.

I use docker to install and run Airflow. So I just updated my DockerFile with these lines:

ENV SLUGIFY_USES_TEXT_UNIDECODE yes

RUN pip install apache-airflow[crypto,celery,postgres,hive,jdbc]==1.10.1

And then run docker build, then docker-compose with the new image.

So far, so good.

In the airflow.cfg, I add the line:

rbac = True

Because I want to create users with specific roles, and allow them to access only their DAGs

The docker containers are running without errors, an error happens when I click on a DAG name in the UI, or when I try to launch a DAG:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.5/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.5/site-packages/flask_admin/base.py", line 69, in inner
    return self._run_view(f, *args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/flask_admin/base.py", line 368, in _run_view
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/flask_login/utils.py", line 261, in decorated_view
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/airflow/www/utils.py", line 372, in view_func
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/airflow/www/utils.py", line 278, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/airflow/www/views.py", line 1345, in tree
    session, start_date=min_date, end_date=base_date)
  File "/usr/local/lib/python3.5/site-packages/airflow/models.py", line 3753, in get_task_instances
    tis = tis.order_by(TI.execution_date).all()
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2703, in all
    return list(self)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2878, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) column task_instance.executor_config does not exist
LINE 1: ...ued_dttm, task_instance.pid AS task_instance_pid, task_insta...

Thanks for any help.

Answer 1

Try airflow upgradedb. This will create missing columns in your metadb.

For folks facing the same issue while upgrading to Airflow >=2.0.0 Try airflow db upgrade