Capture multiline output of script and exit code

Question

I have a Python script that runs whenever a pull request is created. This script does some checks on a file in the repo and prints out the results, along with an exit code (0 if the file looks ok and 1 if the file has issues). I then want to capture the output of the script and auto comment on the pull request with the results.

For a script that fails my check (exit code of 1) I can capture the output or have the exit code fail the step, but not both. Capturing the multiline output into $GITHUB_OUTPUT appears to disregard the Python script's exit code.

Here is my workflow (auto comment step is currently a placeholder until I can get the run script step to work)

name: Test Data

on:
  pull_request

jobs:
  check_data:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4

      - name: Get branch name
        shell: bash
        run: echo "BRANCH=${{ github.event.pull_request.head.ref }}" >> "$GITHUB_ENV"

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'

      - name: Install dependencies
        run: pip install pandas

      - name: Run script
        id: check_data
        shell: bash
        run: |
          {
            echo "results<<EOF"
            echo "$(python tools/check_data.py $BRANCH)"
            echo "EOF"
          } >> "$GITHUB_OUTPUT"

      - name: Auto comment
        shell: bash
        run: echo "${{ steps.check_data.outputs.results }}"

I want the Run script step to fail if the Python script exits with a 1. This works fine if I:

- name: Run script
        run: python tools/check_data.py $BRANCH

But this will obviously not capture the stdout for use in the next step.

Does anyone have thoughts of how I can both capture the output and exit the step with a failure if the script fails? I could have two steps and run the script twice but that's not very efficient.

Answer 1

Here is how you could do this; I've added a comment action as well:

name: Check Data and Show Message in PR

on:
  push:
    branches: [main, develop]
  pull_request:

env:
  PYTHON_VERSION: "3.10"

permissions:
  # perms required by the marocchino/sticky-pull-request-comment action
  contents: read
  pull-requests: write

jobs:
  check-data:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    env:
      # see https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#good-practices-for-mitigating-script-injection-attacks for why we use an env and not templating
      # see https://laurentsenta.com/digital-garden/programming/github-actions-tips/ for why I picked this value (I use a push trigger too)
      BRANCH: ${{ github.head_ref || github.ref_name }}
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}
      # - name: Install dependencies
      #   run: pip install pandas
      - name: Run script
        id: check_data
        shell: bash
        # We use a file instead of an HEREDOC, see https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#multiline-strings
        run: |
          python tools/exit_and_output.py 1 10 | tee comment.txt
      - uses: marocchino/sticky-pull-request-comment@331f8f5b4215f0445d3c07b4967662a32a2d3e31 # v2.9.0
        # on failure and success and if it's a PR -- this is different from `if: always()`
        if: (success() || failure()) && github.event.pull_request
        with:
          recreate: true
          header: check-data
          path: comment.txt
      - if: (success() || failure()) && !github.event.pull_request
        run: |
          cat comment.txt >> "${GITHUB_STEP_SUMMARY}"

A few things to note:

I use a declarative env instead of your echo BRANCH=.... > GITHUB_ENV script. I find this more explicit and it's usually safer.

I use ${{ github.head_ref || github.ref_name }} to get the branch name, just because I wanted to trigger this workflow during push to main as well. But you don't have to.

I use a file instead of the HEREDOC (<< EOF) I find this more maintainable: you don't have to juggle through pipefail, exit codes, and all. Also, you'll avoid weird bugs if your output contains "GEOFFRAY", for example. Doc

I added a comment action; I like that one; it auto-updates.

I use if: (success() || failure()), which stops on job cancellation (not always()), and that makes the rest of the check more readable.

Finally, for the sake of completeness, I also output a job summary on push to main and develop (you can see the result on this GitHub job).

I set this up this in a demo repo if you want to see the output: PR, and the results for many different cases.