Why does GitHub check not reflect Azure Pipelines build status?

Question

I am trying to add Azure Pipelines configuration to an existing project, bundler/bundler. Here is the PR that adds the configuration: https://github.com/bundler/bundler/pull/6899

As one of the maintainers set up the bundler/bundler project on Azure Pipelines, this PR already triggers a build: https://dev.azure.com/bundler/bundler/_build/results?buildId=11 Note that the build has a green checkmark and is marked as finished.

(Also note that there are loads of tests failing in the build, as this wasn't tested on Windows before. To make the build succeed anyway - and not all PRs and commits get the red "x" on Github while I am working on fixing the tests, I added || exit 0 at the end of the test command - which works fine on Azure Pipelines)

A feature of Azure Pipelines' GitHub integration is that the build results are shown in Github via a feature called "Check": https://github.com/bundler/bundler/pull/6899/checks

(A shorter version of that is also included at the end of the PR page: https://github.com/bundler/bundler/pull/6899#partial-pull-merging)

Unfortunately, this check doesn't reflect the build status on Azure Pipelines and is still shown as "in progress":

and

Any idea why the GitHub check doesn't reflect the build status on Azure Pipelines?

---

What is confusing me further, is that the integration with Azure Pipelines actually worked just fine (check correctly reflects the build status) in the Pull Request that was automatically created by Azure Pipelines when creating the bundler/bundler project: https://github.com/bundler/bundler/pull/6955

But: It also can't really be the Azure Pipelines configuration I created in my PR, because the same configuration also works just fine in my fork: https://github.com/janpio/bundler/pull/6#partial-timeline (see the green checkmark for the bundler task). (On the other hand: Here Azure Pipelines doesn't use the "Check" feature of Github at all)

Answer 1

Setting the following configuration worked for me: