Real-time code coverage analysis

09 Apr 2015

I recently sat down to write a node service and happened upon a very helpful workflow when trying to write tests that I wanted to share.

When writing tests, one of the biggest jobs is to ensure you have a reasonable degree of coverage on the various branches in your code. This means that if you have an if statement, you want to check the case where the if is true and the case where the if is false. This gets a little more complicated if you have multiple statements in your if statement like if (foo || bar) {, but the idea is similar.

Determining whether or not you're successfully covering a branch of logic isn't as straight forward as it seems. If you follow TDD, you ensure the line of code is covered by writing the test first, ensuring it fails, then writing the line that makes it pass. I often follow FDD or (fear-driven development). I take the code that I'm most afraid of and I test it until I'm not afraid anymore. Kent Beck put it best when he said "Tests are the Programmer's stone, transmuting fear into boredom". So the question remains, how do you know if the tests you're writing are covering the lines you think they are. The answer is live-updating coverage analysis.

There are 3 big pieces to get working: coverage reports, real-time report updates, and displaying the updates in the browser.

The Report

With coverage analysis, you can get an HTML report like this:

Figure 1: HTML report of javascript source code

This simple facade for an API has some bare tests on it to ensure it operates more-or-less as I expect. You can see the orange and red at the bottom which shows that those lines aren't covered by tests. This is generated through a command line instrumentation step that happens before my tests are run. In node, I do that as a npm script which looks like:

"coverage": "istanbul cover -x '**/*_test.js' _mocha -- `find . -name '*_test.js' | grep -v 'node_modules'`",

This says "run istanbul over my non-test code (excluding node_modules), then invoke mocha (our test runner) with the filenames that end in _test.js". This command generates an HTML file in coverage/lcov-report/index.html. Updating the Report We want this new report to stay fresh as we change our tests and application code. Using a file watcher, we can have it run coverage for us. I like to use npm-watch, which hooks into the npm script we had before. You can set it up by doing npm install --save-dev npm-watch. You then add a "watch" stanza to your package.json which is a mapping of npm script to run to the file globs to watch.

{
  "name": "my-package",
  # ...lots of package.json stuff here...
  "dependencies": {
    "hapi": "^8.4.0",
    "joi": "^6.0.8",
    "request": "^2.54.0"
  },
  "watch": {
    "coverage": "src/**/*.js"
  }
}

This will run npm run coverage any time a .js file in the src directory changes. This keeps our coverage HTML up to date.

Updating the browser

So we have this html file that's constantly updating, but the browser stays static. Using a nifty application called browser-sync, we can have that file automatically refresh. Browser-sync operates by putting a server in front of the simple index.html file. I run it from yet-another-npm-script like this:

"coverage-watch": "browser-sync start –files='coverage/lcov-report/index.html' –server –startPath coverage/lcov-report",

This starts up a server at http://localhost:3000/ which proxies back to the index file. It also injects some javascript into the page which will reload the content any time the backing file changes. As a nice bonus, it does it in a way that doesn't scroll it to the top, so the window stays put right where you have it.

The Result

The result of this multi-process setup is that you can be editing code in your favorite editor and you can see the code coverage report updating in the window next to it.

Figure 2: Update code and coverage information auto-refreshes