How gitlab runner concurrency works?

Question

Q1: Whats the difference between

concurrent = 3  [[runners]]   ..   executor = "shell" 

and

concurrent = 3  [[runners]]   ...   executor = "shell"  [[runners]]   ...   executor = "shell"  [[runners]]   ...   executor = "shell" 

Q2: Does it makes sense, to...

have 3 executors (workers) of same type on a single runner with global concurrent = 3? Or can single executor with global concurrent = 3 do multiple jobs in parallel safely?

Q3: How they're related...

runners.limit with runners.request_concurrency and concurrent

Thanks

Answer 1

Gitlab's documentation on runners describes them as:

(...) isolated (virtual) machines that pick up jobs through the coordinator API of GitLab CI

Therefore, each runner is an isolated process responsible for picking up requests for job executions and for dealing with them according to pre-defined configurations. As an isolated process, each runner have the capability of creating 'sub-processes' (also called machines) in order to run jobs.

When you define in your config.toml a [[runner]] section, you're configuring a runner and setting how it should deal with job execution requests. In your questions, you mentioned two of those "how to deal with job execution request"' settings:

1. limit: "Limit how many jobs can be handled concurrently". In other words, how many 'sub-processes' can be created by a runner in order to execute jobs simultaneously;
2. request_concurrency: "Limit number of concurrent requests for new jobs from GitLab". In other words, how many job execution requests can a runner take from GitLab CI job queue simultaneously.

Also, there are some settings that apply to a machine globally. In your question you mentioned one of them:

1. concurrent: "Limit how many jobs globally can be run concurrently. This is the most upper limit of number of jobs using all defined runners". In other words, it limits the maximum amount of 'sub-processes' that can run jobs simultaneously.

Thus, keeping in mind the difference between a runner its sub-processes and also the difference between specific runner settings and global machine settings:

Q1:

The difference is that in your 1st example you have one runner and in your 2nd example you have three runners. It's worth mentioning that in both examples your machine would only allow running 3 jobs simultaneously.

Q2:

Not only a single runner can run multiple jobs concurrently safely but also is possible to control how many jobs you want it to handle (using the aforementioned limit setting).

Also, there is no problem to have similar runners running in the same machine. How you're going to define your runner's configurations is up to you and your infrastructure capabilities.

Also, please notice that an executor only defines how to run your job. It isn't the only thing that defines a runner and it isn't a synonymous for "worker". The ones working are your runners and their sub-processes.

Q3:

To summarize: You can define one or many workers at the same machine. Each one is an isolated process. A runner's limit is how many sub-processes of a runner process can be created to run jobs concurrently. A runner's request_concurrency is how many requests can a runner handle from the Gitlab CI job queue. Finally, setting a value to concurrent will limit how many jobs can be executed at your machine at the same time in the one or more runners running in the machine.

References

For better understanding, I really recommend you read about Autoscaling algorithm and parameters.

Finally, I think you might find this question on how to run runners in parallel on the same server useful.