Gitlab: There has been a runner system failure

Question

I use Gitlab for doing Continuous Integration and Development and all of a sudden I get this error message "There has been a runner system failure, please try again"

There's no real error message or error code.

I've tried restarting the gitlab runner, using gitlab-runner restart, I've done a reboot of the server its running on but I keep getting this error message on Gitlab whenever I push a code change.

Answer 1

After a couple of hours, I realized the issue is that the server that Gitlab Runner is running on has no space left.

I logged into the server in question. Looked at the Gitlab log file using the following command: journalctl -u gitlab-runner

And it showed me the following logs:

May 21 08:20:41 gitlab-runner[18936]: Checking for jobs... received                       job=178911 repo_url=https://.......git runner=f540b942
May 21 08:20:41 gitlab-runner-01 gitlab-runner[18936]: WARNING: Failed to process runner                   builds=0 error=open /tmp/trace543210445: no space left on device executor=docker runner=f540b942

To fix this issue I ran docker conatiner prune which clears out stopped containers.

Alternatively you could use docker system prune which would remove all unused objects.

See https://linuxize.com/post/how-to-remove-docker-images-containers-volumes-and-networks/ for more information about those docker commands.

Afterwards, I no longer got the error on Gitlab when pushing changes.