GitHub pipeline/CI to generate files and push them back to the repository

Question

I maintain a public repository on GitHub where changes are only made to a single YAML file. I'm looking for a solution to process that file on every push and generate files based on it. Essentially, a pipeline or CI should parse the file and create many different markdown files. These files (or more specifically, the changes to these files) should then be pushed back to the repository.

Requirements:

• Manual changes to the YAML file and automatic changes to the markdown files should both be pushed to the master branch.
• The version history should be kept (e.g. forced push might not work).
• There is an arbitrary number of files that are generated.

There are Travis providers for GitHub Pages and GitHub Releases. But both have limitations that make them unsuitable for my requirements.

Using what tool/CI/pipeline can I achieve that on GitHub? I would prefer a service over a self-hosted CI.

Answer 1

Assuming that you already have the program/script to parse the YAML file and to generate the Markdown files, I can give you some insights on how I would do this from Jenkins CI. While I draw my experience from running my own instance, there are also hosted options such as CloudBees that you can explore.

1. Create a new Jenkins Freestyle project.
2. Under the Source Code Management section, configure your GitHub project coordinates.
3. Under Build Triggers section, activate the 'Build when a change is pushed to GitHub' option. That would launch the CI job at the moment you push a new version of the YAML file into the repository.
4. Under the build section, add an Execute shell build step.
5. In the shell step, launch the program or script that processes the YAML file/generates the .md files. End the script by adding the git add ., git commit -m "message", git pull and git push commands (assumes git is in the path).
6. Enable the new job to make it active in Jenkins.