I want to understand Github Checks API so that I can use them to retrieve data. On following Github documentation https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api I am able to derive that check runs are associated with the sha of the change and at each commit on branch check suite is created. Checks API helps in getting all this information. But I want more clarity on three of them in terms of differences. Can anyone please explain these three terms using simple example and terms?
So, first of all, the GitHub Commit Statuses API is separate from the GitHub Checks API (includes suites & runs), so let's look at them individually first then I'll explain the differences.
Before we get into it, I want to differentiate a PR Check from a Check Run, to avoid confusion. A PR Check describes the current state (trying not to say status here 😉) of a given job or task running in CI or elsewhere on a specific PR commit. These can be created via either a Commit Status or a Check Run. All the items in the pink box below are PR Checks, notice the Hide all checks button.
I see this as the simple, all-purpose API for reporting PR Checks for a given commit. It's easy and doesn't require jumping through hoops to just display a simple result of a PR Check for a given commit. This API also came before the check API so it's a little less powerful.
context
as the identifier.status
options, only allow error
, failure
, pending
, success
, no conclusion
subset option to define completed jobs like Check Runs.output
logs. This is not too important as you could just link to the URL where the actual PR Check was run such as in CircleCI, Jenkins, etc. But the user is not always authorized to view these runs so the output could be helpful for open source repos that have non-public CI.The Checks API is the latest and greatest tool for displaying task results on commits, which can essentially do everything the Commit Statuses API can do and more. Check Runs belong to one Check Suite, one Check Suite can have many Check Runs. You can only have one Check suite per commit (i.e. head_sha
) per GitHub App, attempting to create another for a given App will just return the previously created Check Suite. Thus, a new Check Run is automatically assigned to the Check Suite based on the authenticated GitHub App, you cannot manually assign Run to Suites.
Contrary to statuses, Check runs are identified by an auto-generated check_run_id
and not a context
string.
I haven't touched too much on the Check Suites API because they are really just a grouping of Check Runs which is pretty self-explanatory and they don't affect any of the PR Checks UI, only the grouping of Check Runs in the checks tab. One thing to note is that by default you can create a Check Run without having to first create a Check suite and GitHub will just create a new Check Suite for you.
status
/conclusion
for a run.status
conclusion
and duration of the check run task.One edge case you likely won't come across but is good to know: If you are creating a new Check Run with the exact same
name
of an existing check run under the same authenticated app. The resulting behavior is a little strange but doing this will create the new Check Run under the same name, but will not delete the existing Check Run. However, the Check Suite will not see or link to the existing Check Run, even in the PR UI. BUT!! If you change thename
of either such that the runs now have uniquename
s, it will be linked up again. It seems GitHub just does a sort by date and then filters by unique names when looking up Check Runs in a Check Suite. This does not apply with identicalnames
from different authenticated apps.
Below is a mapping of sorts to compare similar options between the Commit Statuses API and the Check Runs API. They are not exactly 1:1 but similar.
Commit Status | Check Run | Option Desciption |
---|---|---|
sha |
head_sha |
These are equivalent with the minor exception that the Commit Status is linked to the sha directly, whereas the head_sha is linked to the Check Suite that the Check Run belongs to. |
context |
name |
The context is used as an identifier but both define the title of the PR Check in the PR UI. Because Check Runs are tracked by the check_run_id the name option may change without creating a new Check Run. This is not the case for context , changing the context will always create a new Commit Status. You may not have duplicate names for Check Runs created with the same GitHub App, see note above. |
context * |
external_id |
The external_id is meant for keeping track of Check Runs without having to store ids or always keep the name constant. This is only somewhat similar to the context option but only for the purpose of identifying the Check Run. |
description |
output.title |
The main difference here is that description gives you the full space to work with, where output.title will be displayed after the auto-generated status string. |
target_url |
details_url |
These are somewhat equivalent, the first difference is that target_url will not show unless defined whereas details_url will default to the check run URL in the PR UI. The other difference is the Check Runs Details button on the PR Checks UI will always link the the Checks page which will present a link to the details_url if defined. |
state |
status |
These are very similar but have slightly different allowed values but effectively appear the same in the PR UI. |
N/A | conclusion |
This just shows the increased power of the PR Checks API where you have more granular control over the PR Check status, though not a big difference in the UI, see example variations below. |
N/A | started_at |
No comparable option for Commit Statuses. |
N/A | completed_at |
No comparable option for Commit Statuses. |
N/A | actions |
No comparable option for Commit Statuses. |
N/A | output |
No comparable option for Commit Statuses. |
N/A | output.annotations |
No comparable option for Commit Statuses. |
* Only somewhat similar, not a direct equivalent.
I've taken a simple PR Check and highlighted the differences between the elements of the UI between the Commit Status API and the Check Runs API. Very similar but slight differences.
Below are example variations to relate the options to their impact on the PR Checks UI.
If a check run is in an incomplete state for more than 14 days, then the Check Run's
conclusion
becomesstale
. Only GitHub can mark Check Runs asstale
.
The Checks Tab in the PR UI will display all created Check Suites and their child Check Runs and allows you to set the output
option to control the content of this page. This page would also show images
and annotations
if you defined those too.
One final note: You cannot DELETE any Commit Status, Check Run or Check Suite once created, only updating is allowed. With these statuses being so ephemeral this is not that big of a deal but in case you were wondering.
I found two more quirks to explain. First, the details_url
for Check Runs does not set the value of the Details
button, instead the Details
button actually always redirects to the Checks page which has a link to the details_url
if defined.
Second, once a Check Run status
is set to 'completed'
, there is not way to un-complete the Check run. But you can get around this by creating a new Check Run with the same name
and set that status
to something other than 'completed'
. See this edge case with duplicate-name
d Check Runs explained above in detail.
As mentioned above, the Check Runs API keeps track of the duration of the run. It is important to note that the started_at
time is set only once whenever a check run is created, regardless of the defined status
. For example, if I trigger a CI build and set all jobs status
to queued
, the jobs run duration will be inaccurate. A good way to fix this is to always set the started_at
time whenever you set the status
to in_progress
. Something as simple as const started_at = new Date().toISOString()
(javascript) will do the trick.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With