gitlab runner throws "Cleaning up file based variables 00:01 ERROR: Job failed: exit code 1" at the end

Question

Even though all my steps pass successfully , Gitlab CI shows this - "Cleaning up file based variables 00:01 ERROR: Job failed: exit code 1"

and fails the job at the very end . Also interestingly , this only happens for my master branch . It runs successfully on other branches. Has anyone faced this issue and found a resolution ?

    - >
     for dir in $(git log -m -1 --name-only -r --pretty="format:" "$CI_COMMIT_SHA"); do 
     if [[ -f "$dir" ]]; then 
     SERVICE=$(echo "$dir")
     # helm install the service
     fi
     done
    - echo "deployed" 

Answer 1

Overview

This drove me crazy and I'm still not sure what the appropriate answer is. I just ran into this issue myself and sunk hours into this issue. I think GitLab messed something up with command substitution (shows a new release yesterday), although I could be wrong about the issue or its timing. It also seems to only occur for some command substitutions and not others, I initially suspected it might be related to outputting to /dev/null, but wasn't going to dive too deep. It always failed immediately after the command substitution was initiated.

---

My code

I had code similar to yours (reduced version below), tried manipulating it multiple ways, but each use of command substitution yielded the same failure message:

Cleaning up file based variables          00:01
ERROR: Job failed: exit code 1

Attempts I've made include the following:

- folders=$(find .[^.]* * -type d -maxdepth 0 -exec echo {} \; 2>/dev/null)
- >
  while read folder; do
      echo "$folder"
  done <<< "$folders"

And ...

- >
  while read folder; do
      echo "$folder"
  done <<< $(find .[^.]* * -type d -maxdepth 0 -exec echo {} \; 2>/dev/null)

Both those versions succeeded on my local machine, but failed in GitLab (I might have typos in above - please don't scrutinize, it's reduced version of my actual program).

---

How I fixed it

Rather than using command substitution $(...), I instead opted for process substitution <(...) and it seems to be working without issue.

- >
  while read folder; do
      echo "$folder"
  done < <(find .[^.]* * -type d -maxdepth 0 -exec echo {} \; 2>/dev/null)

I would try to substitute the same in your code if possible:

- >
  while read dir; do
      # the rest goes here
  done < <(git log -m -1 --name-only -r --pretty="format:" "$CI_COMMIT_SHA")

The issue might also be the line inside the if statement (the echo), you can replace that with the following:

read SERVICE < <(echo "$dir")

---

Again, not exactly sure this will fix the issue for you as I'm still unsure what the cause is, but it resolved my issue. Best of luck.

Answer 2

The error seemed to vanish for me once I removed the script from .gitlab-ci.yml file to another scipt.sh file and called the script.sh file in the gitlab yaml.

Answer 3

I had this error when tried to use protected CI/CD variable.