Make Cordova builds in CI/CD environment more efficient

Question

I would like to speed up the build process of a hybrid app with lot of feature branches. We have our docker container to create a cordova android and ios builds. Building a cordova app from scratch can be really slow, it has to download various third party libs, you need gradle dependencies for android, cocoapods update for ios, etc. Sometimes, also due docker-mac io performance penalties, it can take 30-40 mins to complete all native builds. Which is a lot.

Basically, our native part of the app is really static, we add or change a plugin really rarely, so I don't really see why should I rebuild an ipa, apk or any native platform anytime when we've only changed some javascript code.

So I am planning to optimise the build process a bit and here are my two ideas and I would like to hear some comments, pros or cons about those.

1. I could create a separate 'native app' build which could create an ipa and apk but without any content in www directory.

   The other build process should only grab these native artifacts, put the client code into www folder and resign the app. This way the build on feature branch only limited by the speed of our client code bundler and doesn't even need a lot of dependencies for native builds(e.g. in case of a 'clean' build agent)

2. Instead of build the build agent's build directory, the cordova related code can be moved to a shared directory, add the platform there and every build just link the www folder there, then I can start with cordova prepare.

Problems with the first one:

• Additional complexity
• How to know when to trigger a native rebuild(Store the whole cordova setup in separate repo?)
• Is resign viable for every platform?

Problems with the second one:

• Locking/caching issue(files from branchA left there when build branchB)
• Helps only the same agent if I don't use a network drive for sharing(but I really don't want to)

I didn't really find any existing methodology or utility which could help me.

I know I could do android builds not on a mac, I could bring down build of native part to 5 minutes or lower, but that is still 5 minutes more than 0. And that is only for one platform.

Here is an example what I mean by having a separate native app and client code:

create cordova app

cordova platform add android
cordova plugin add x.y.z
cordova build android
#export platforms/android/app/build/outputs/apk/debug/app-debug.apk as an artifact

bundle client code with native app

#grab app-debug.apk artifact from a previous native build(which is rarely run)
a
unzip app-debug.apk originalContent
webpack
cp app-entrypoint.js originalContent/assets/
pushd originalContent
zip -r ../app-modified.apk ./
popd
zipalign -c 4 app-modified.apk
apksigner sign --ks keystore --ks-key-alias=alias app-modified.apk
#export app-modified.apk as an artifact

Answer 1

I don't have a lot of experience with Cordova builds, so I can't speak specifically to you needs, but of the options you listed #1 seems like it's adding too much complexity and #2 sounds like it adds risk/uncertainty. I'd personally avoid both.

All is not lost, though, as I have a few suggestions. Since I don't know which runner system your using (I primarily use GitLab or Drone.io), I'll just speak to my prior use-cases and assume you're using something like a private GitLab Runner.

#1 Add Caching

This is the best/fasted way to speed up your builds.

Building a cordova app from scratch can be really slow, it has to download various third party libs, you need gradle dependencies for android, cocoapods update for ios, etc.

As you noted, downloading 3rd party dependencies is one of the slowest things a runner can do, and often for complex builds (such as android/ios/cordova/etc) this leads to a lot of data being downloaded every run. In this case, a runner-cache becomes critical so that it only needs to download the build assets every once in awhile (like, weekly vs every build).

Some runners (like GitLab and Bitbucket) provide this mechanism out-of-the-box while others (Drone) rely on plugins or hand-rolling:

• https://docs.gitlab.com/ee/ci/caching/
• http://plugins.drone.io/meltwater/drone-cache/

#2 Making Your Build Environment Ephemeral

While caching your build artifacts is an excellent first step, you may find that setting up the build environment is also tedious. You mentioned running a docker container as part of the build, but then a question arises if you're building that container for every run, or if you're installing things inside that container for every run.

Particularly for my Android CI builds, I found it easiest to create my own private container, and put it in a private registry for use as runner containers. This allowed for all of the JDK and Android SDK configuration to happen once, and allowed the runner to focus on just app build and not setup. Bonus is that I also had the docker container perform an initial app build during the build process so that I had all of the gradle packages installed already too.

#3 Parallel Builds

I'm not sure if you're using this yet, but using parallel builds can cut total pipeline runs in half (it won't help with single jobs though).

In GitLab this is accomplished by properly setting up 'stages', and then having multiple jobs setup per stage. For a GitLab example:

stages:
 - Build

"Build iOS":
  stage: Build
  script:
     - ...

"Build Android":
  stage: Build
  script:
     - ...

Or with Drone via the 'depends_on' keyword: https://discourse.drone.io/t/how-to-setup-parallel-pipeline-steps-1-0/3251

#4 IO Improvements

Sometimes, also due docker-mac io performance penalties

I'm assuming you're specifically referring to the mounting of local folders into a docker container? If so, don't mount volumes for the build, instead either perform a file-copy into the Docker container before a build or instruct the container to checkout the code directly from your repo prior to a build.

I primarily run Windows, but also use WSL and Docker, so the disk IO penalty is even worse for me, so I'm often picking/choosing where the files are located before I run a build (depending on what OS is doing the building).