What are the differences between GitHub Actions and other CI tools like Jenkins?

Question

GitHub announced an upcoming feature, GitHub Actions.

I'm positive on the benefits of CI tools like Jenkins for automatic building or testing, which GitHub Actions is aimed to be used for in the future.

Having a repository on GitHub and using an external CI tool has the huge benefit of allowing to move the repository to another Git repository platform (or even local) without rewriting of the whole CI process. With GitHub Actions, you're more or less tied to the GitHub ecosystem.

I assume the integration of GitHub's Actions will be more fluent in the native environment, but are there any other advantages or disadvantages besides that?

Answer 1

I've been working with GitHub actions full time for a couple of months now.

It's still early days (June 2019), but here's my list:

Advantages:

1. GitHub actions are just consecutive docker runs. Very easy to reason about and debug. Reproducing the build environment for container-based Travis is possible, but more difficult. On GitHub actions it's just a docker build docker run away.
2. The individual actions in a workflow are isolated by default. You can use a completely different computing environment for, say, compilation and testing. Travis CI (and I think other "traditional" CI) would run all "stages" (~ actions) in the same computing environment. Again, GitHub actions are much easier to reason about and debug.
3. The main.workflow spec (a subset of the HCL and really just a directed acyclic graph) is open source. The whole thing is a pretty thin wrapper around Docker anyway, so platform lock-in is arguably minimal.
4. There are already open-source reimplementations of GitHub actions, such as act for local testing.
5. You have ready access to the GitHub API with (somewhat limited) authentication out of the box.
6. There might be a vibrant community (marketplace?) where people can share actions. For example, I'm reusing deploy actions build by different people in different ecosystems.
7. A directed acyclic graph (DAG) and the visual editor for main.workflows is perhaps a good way to model CI/CD in particular and workflows in general. Takes some getting used to, but generalises well.
8. GitHub actions can do a whole lot more than just CI! You've got basically the whole API at your fingertips as inputs and outputs.

Disadvantages:

GitHub actions (still?) has sometimes surprisingly foundational limitations at this point (june 2019).

1. No native caching. You get image and layer caching (it's complicated), but nothing else. For build artefacts, you have to roll your own cache (via AWS, Azure, etc. ...), which can be a lot of work. (You can see a hacky setup here.
2. Surprisingly, no support for pull requests from forks. It's again a bit complicated, and understandable from a security standpoint, but it's currently not possible to run actions a) against the secrets of the receiving repo of a fork PR (base), and/or b) against the would-be merge result of a fork PR (that's what travis does). For a workflow that involves forks, that makes GitHub actions largely unuseable as CI/CD tool.
3. Single platform, it's just whatever you can run inside docker, so some Linux distro. That seems unlikely to change, but might be an acceptable limitation. You can always add an action to call other cross-platfrom CI/CD services.
4. The documentation is still pretty sparse. Not much in the way of best practices or scaffolding.
5. The quality and breadth of published GitHub actions (at least on the marketplace) is still pretty low / limited. We'll see whether that takes off.
6. No great way to unit-test actions. (I hacked something together, but I'm not too sure about it).