Stopping a started background service (phantomjs) in gitlab-ci

Question

I'm starting phantomjs with specific arguments as part of my job.

This is running on a custom gitlab/gitlab-ci server, I'm currently not using containers, I guess that would simplify that.

I'm starting phantomjs like this:

- "timeout 300 phantomjs --ssl-protocol=any --ignore-ssl-errors=true vendor/jcalderonzumba/gastonjs/src/Client/main.js 8510 1024 768 2>&1 >> /tmp/gastonjs.log &"

Then I'm running my behat tests, and then I'm stopping that process again:

- "pkill -f 'src/Client/main.js' || true"

The problem is when a behat test fails, then it doesn't execute the pkill and the test-run is stuck waiting on phantomjs to finish. I already added the timeout 300 but that means I'm still currently waiting 2min or so after a fail and it will eventually stop it while test are still running when they get slow enough.

I haven't found a way to run some kind of post-run/cleanup command that also runs in case of fails.

Is there a better way to do this? Can I start phantomjs in a way that gitlab-ci doesn't care that it is still running? nohup maybe?

Answer 1

TL;DR; - spawn the process in a new thread with & but then you have to make sure the process is killed in successfull and failure builds.

i use this (with comments):

'E2E tests':
  before_script:
    - yarn install --force >/dev/null
    # if there is already an instance running kill it - this is ok in my case - as this is not run very often
    - /bin/bash -c '/usr/bin/killall -q lite-server; exit 0'
    - export DOCKERHOST=$(ifconfig | grep -E "([0-9]{1,3}\\.){3}[0-9]{1,3}" | grep -v 127.0.0.1 | awk '{ print $2 }' | cut -f2 -d ':' | head -n1)
    - export E2E_BASE_URL="http://$DOCKERHOST:8000/#."
    # start the lite-server in a new process
    - lite-server -c bs-config.js >/dev/null &
  script:
    # run the tests
    - node_modules/.bin/protractor ./protractor.conf.js --seleniumAddress="http://localhost:4444/wd/hub" --baseUrl="http://$DOCKERHOST:8000" --browser chrome
    # on a successfull run - kill lite server
    - killall lite-server >/dev/null
  after_script:
    # when a test fails - try to kill it in the after_script. this looks rather complicated, but it makes sure your builds dont fail when the tests succeedes and the lite-server is already killed. to have a successfull build we ensure a non-error return code (exit 0)
    - /bin/bash -c '/usr/bin/killall -q lite-server; exit 0'
  stage: test
  dependencies:
    - Build
  tags:
    - selenium

https://gist.github.com/rufinus/9ee8f04fc1f9248eeb0c73ad5360a006#file-gitlab-ci-yml-L7

Answer 2

As hinted, basically my problem wasn't that I couldn't kill the process, it's that running my test script and it failing stopped at that point, resulting in a deadlock.

I was already doing something quite similar to the example from @Rufinus, but it just didn't work for me. There could be a few different things, like different way of running tests or so or starting it in before_script, which is not an option for me.

I did find a way to make it work for me, which was to prevent my test runner from stopping the execution of further tasks. I managed to do that with a "set +e" and then storing the exit code (something I tried to do before but it didn't work).

This is the relevant part from my job:

# Set option to prevent gitlab from stopping if behat fails.
- set +e
- "phantomjs --ssl-protocol=any --ignore-ssl-errors=true vendor/jcalderonzumba/gastonjs/src/Client/main.js 8510 1024 768 2>&1 >> /dev/null &"
# Store the exit code.
- "./vendor/bin/behat -f progress --stop-on-failure; export TEST_BEHAT=${PIPESTATUS[0]}"
- "pkill -f 'src/Client/main.js' || true"
# Exit the build
- if [ $TEST_BEHAT -eq 0 ]; then exit 0; else exit 1; fi