Is there any way to get the exception details on the airflow on_failure_callback
?
I've noticed it's not part of context
. I'd like to create a generic exception handling mechanism which posts to Slack information about the errors, including details about the exception. I've now managed to trigger/execute the callback and post to Slack, but can't post the exception details.
Thanks.
An on_failure_callback
can be supplied to the DAG and/or individual tasks.
In the first case (supplying to the DAG), there is no 'exception'
in the context
(the argument Airflow calls your on_failure_callback
with).
In the second case (supplying to a task), there is.
The contained object should be a python Exception
.
It's surprisingly non-intuitive to get something like a stack trace from that, but from this answer I use the following to get a fairly readable stack trace:
import traceback
...
exception = context.get('exception')
formatted_exception = ''.join(
traceback.format_exception(etype=type(exception),
value=exception, tb=exception.__traceback__
)
).strip()
The error is added to the context here. So you can actually get that simply by doing:
context.get("exception")
Unfortunately it doesn't look like you can get a stack trace or something like that from the context.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With