How to merge multiple simplecov directories inside CircleCI when running parallel_test?

Question

I have a rails application that runs parallel_test with rspec inside circleci

Looking around on Internet I added up this to the very beginnging of my spec_helper.rb file:

if ENV['COVERAGE']
  require 'simplecov'
  # on circleci change the output dir to the artifacts
  if ENV['CIRCLE_ARTIFACTS']
    dir = File.join("..", "..", "..", ENV['CIRCLE_ARTIFACTS'], "coverage")
    SimpleCov.coverage_dir(dir)
    SimpleCov.merge_timeout 3600
    SimpleCov.command_name "rspec_#{Process.pid.to_s}#{ENV['TEST_ENV_NUMBER']}"
  end
  SimpleCov.start 'rails'
end

The problem is that as a result I get different folders one for each circleci instance:

What am I doing wrong ?

Answer 1

I work at CircleCI. Unfortunately this isn't going to work - we don't collect the artifacts directories from different builds until after all builds have finished running, so tools that try to merge them together during the build won't work. We have talked about adding capabilities to do this, but it's not currently on our feature roadmap, sorry!

Answer 2

For anyone who still looking for a solution to this problem, there is a new possibility: using ssh between containers to manually sync and merge reports, see the docs. This is, however, not a turnkey solution, you will have to write the necessary scripts yourself.

Otherwise, you can also use an external coverage service (we use coveralls codecov) together with CircleCI's notification webhook.

Edit

You can add the webhook like this in your circle.yml (thanks Ian): notify: webhooks: - url: https://coveralls.io/webhook?repo_token=(your repo token)