Duplicating Node.js `assert` assertions in spec `expect` [closed]

Question

As opposed to non-asserted code with custom developer-friendly checks,

class Some {
  constructor(arg) {
    if (Array.isArray(arg) && arg[0] === 'foo')
      this.foobar = arg.concat('bar').join('');
    else
      console.error('Bad Some constructor arg');
  }
}

currently tested code is heavily packed with Node assert assertions with reasonably meaningful message arguments:

class Some {
  constructor(arg) {
    assert.deepEqual(arg, ['foo'], 'Some constructor arg');
    this.foobar = arg.concat('bar').join('');
  }
}

The assertion is there to

• keep the code flat and readable
• provide meaningful feedback on incorrect usage with call stack
• prevent function execution and don't propagate the error further
• throw an error and leave error handling to caller

Current spec may look like that:

it('...', () => {
  let some = new Some(['foo']);
  expect(some).to...

And it will pass - desirable usage is asserted in spec, undesirable usage it asserted in tested code.

To overlap code assertions in part it may be even

it('...', () => {
  const { AssertionError } = require('assert');
  let some = new Some(['foo']);
  expect(some).to...
  expect(() => new Some(['bar']).to.throw(AssertionError);

So we basically assume here that half of testing job was already done in the code itself with assert and skip the details (to.not.throw and matching AssertionError messages).

The example above uses Mocha+Chai, but the same thing applies to Jasmine.

• Should app assertions be treated as any other code lines and doubled with spec assertions (to throw, to not throw, AssertionError message matching), what are the consequences of taking a shortcut?

• Can test coverage tools (Istanbul) take into account assert assertions in app code in addition to expect?

• May test runners be confused by the fact that it was app, not spec assertion that threw an error?

Some examples of successful open-source JS projects that prove or refute 'assert assert assertions' point in practice could be also helpful.

Answer 1

Should app assertions be treated as any other code lines and doubled with spec assertions (to throw, to not throw, AssertionError message matching), what are the consequences of taking a shortcut?

I wouldn't recommend that. That is like testing a test: although technically it is possible, typically you don't do it because it's not worth the cost.

Can test coverage tools (Istanbul) take into account assert assertions in app code in addition to expect?

What is the behavior you'd like to get? With unit test this is quite clear what to meassure: you run the test and monitor which code gets visited. But what would you like to do with assert (in the code)? If the assert statement is reached by some unit test, well, the code obviously is test-covered. If it is not - should some code around the assert statement count as covered by test? How much of code would you like to count as covered? It all gets really nasty and I don't believe that some proper solution exist for this.

Moreover the whole covered-by-tests metrics is at least smelly - the mere fact there exist test that reaches some code tells you so little about its correctness!

May test runners be confused by the fact that it was app, not spec assertion that threw an error?

This was (IMO correctly) answered by William. Or is there anything missing? :)

Answer 2

Should app assertions be treated as any other code lines and doubled with spec assertions (to throw, to not throw, AssertionError message matching), what are the consequences of taking a shortcut?

App assertions are there to notify a developer of incorrect use of a certain piece of code, ideally they will never happen in production. If it does happen, then they serve as a tool to identity what went wrong tough, in the same way standard Error's do. But ideally, they are a first line of defense during development, and they should only happen then. ( hence this is one reason why in some languages you can disable assertions for run time all together )

If you write some class that uses assertions to make sure input parameters are validated or the program isn't being used inconsistently, then for sure it makes a lot of sense to put that logic under unit tests as well. If you're gonna change the assertions at some point, you wanna be able to test that you don't break other people's code.

Can test coverage tools (Istanbul) take into account assert assertions in app code in addition to expect?

Yes. If you set up a unit test that will trigger an assertion and then capture it in chai, then that code path will show up in your coverage report. This is no different then any other code path's unless I'm mis understanding your question.

May test runners be confused by the fact that it was app, not spec assertion that threw an error?

When you write Assertions using the assert module, it's gonna be throwing an AssertionError. This is the AssertionError Class from the assert module. When chai throws an error it will also throw an AssertionError, but this will be an entirely different class coming from the chai module, they only share the name. As such there shouldn't be any confusing as to where assertions are originating from.
Usually you aren't capturing assertions in test code at all trough try / catch constructs so this really shouldn't be a problem.

Usually assertions test much less complex states then unit test. But where assertions end and unit tests should start is indeed not formally defined. More reading:

https://softwareengineering.stackexchange.com/questions/18288/are-asserts-or-unit-tests-more-important