How to create concourse loop for repetitive resources and jobs?

Question

Right now I have a concourse pipeline script that can update dockerhub images from a git repo. I created this based on the following tutorial. I have several docker containers in this git repo and I would like to be able to iterate through them to repeat the docker-image resources and the image-update jobs for each different container to make my script cleaner and more readable. Here is my current script:

---
resources:
- name: resource-docker
  type: git
  source:
    uri: https://github.com/$MYUSER/$MYREPO.git
    branch: master

# docker-image resources
- name: first-container
  type: docker-image
  source:
    repository: $MYUSER/first-container

- name: second-container
  type: docker-image
  source:
    repository: $MYUSER/second-container

jobs:
# image-update jobs
- name: first-container-image-update
  public: true
  serial: true
  plan:
  - get: resource-docker
  - put: first-container
    params:
      build: resource-docker/first-container

- name: second-container-image-update
  public: true
  serial: true
  plan:
  - get: resource-docker
  - put: second-container
    params:
      build: resource-docker/second-container

How can I change this so that I only have to create one docker-image resource and the image-update job?

Answer 1

You can't, as that's not really how Concourse works.

Concourse is entirely declarative, idempotent, repeatable and reproducible. So the idea of having configuration be procedural is not really something the tool supports. The configuration should be set once, stored under version control, and then be immutable.

You could automate the generation of the pipeline YAML file in the first place, or write a new resource that reports each Docker image as a new 'version' to go through a single pipeline - this is a similar approach to how the GitHub Pull Requests resource works.