How do a generate vscode TypeScript extension coverage report

Question

It seems that coverage report with coveralls is not possible for VSCode extension built with TypeScript.

Currently, I am adding test cases to our project https://github.com/PicGo/vs-picgo/pull/42, I have found several ways to report coverages, but none of them work for me.

Using custom TestRunner

The official documentation mentions little about custom test runners, but I found a post here. It works when I use F5 to launch an Extension Test, but does not work when I run npm run test in the console (Got no coverage output at all).

I have also tried to understand the custom runner (source code) in the blog post, but I found I have nothing to do because I do not know why it works.

Using nyc

nyc with mocha is very powerful, but we cannot take advantage of it. When I run nyc ./node_modules/vscode/bin/test, I will got 0% coverage:

I have searched the issue page of nyc, lots of the same 0% coverage problems about TS projects exist, but none of them are the same with our environment. The main difference is that they are using mocha for testing, not like VSCode's ./node_modules/vscode/bin/test script, it will create a new process to run the test js files. I don't know how to deal with this.

I searched all the issues (mocha, tyc, istanbul, vscode, etc...), and there are few (I did not find any 😭 ) vscode TypeScripts are using coverage report for me to copy from. So my question is: how do I get the coverage report for my VSCode TS extension?

Answer 1

I have struggled with this myself for some time until I got it working properly. There were three main challenges in getting it working:

1. Proper setup of the nyc instance
2. Preventing race conditions on startup
3. Capturing nyc output and displaying it in the debug console

You can find my working test runner here. I'm also sharing additional insights on my blog.

Answer 2

I got everything working with Mocha, NYC, and VSCode!

You can see my solution to this in https://github.com/jedwards1211/vscode-extension-skeleton.

Basically, I use Babel to transpile my .ts code with @babel/preset-typescript and babel-plugin-istanbul before running the tests. This allows me to skip the convoluted extra steps of instrumenting the tsc output and using remap-istanbul.

Then in the test runner, I use the (not really documented) NYC API to write the coverage to disk after tests finish.

Finally, in my package scripts, I run nyc report after the test command finishes.

UPDATE: you need to delete the .nyc_output folder before each test run too.

src/test/index.js

import NYC from 'nyc'

export async function run(): Promise<void> {
  const nyc = new NYC()
  await nyc.createTempDirectory()
  // Create the mocha test
  const mocha = new Mocha({
    ui: 'tdd',
  })
  mocha.useColors(true)

  const testsRoot = path.resolve(__dirname, '..')

  const files: Array<string> = await new Promise((resolve, reject) =>
    glob(
      '**/**.test.js',
      {
        cwd: testsRoot,
      },
      (err, files) => {
        if (err) reject(err)
        else resolve(files)
      }
    )
  )

  // Add files to the test suite
  files.forEach(f => mocha.addFile(path.resolve(testsRoot, f)))

  const failures: number = await new Promise(resolve => mocha.run(resolve))
  await nyc.writeCoverageFile()

  if (failures > 0) {
    throw new Error(`${failures} tests failed.`)
  }
}