Asserting a Flow type error in unit tests

Question

We use Flow for static type-checking in JavaScript. Flow types can get complex, and we've had issues where we thought we had good static type guards against malformed objects, but Flow annotation issues meant that type checking didn't actually catch problems.

To prevent this problem, I'd like to write type-based "unit tests" that can statically assert our assumptions about type protections. This is easy if I want to assert the "valid" case, using type assertions:

type User = {|
    name: string
|};

const user = {name: "Bob"};
(user: User);

This will pass if user is a valid User, and fail type check if it's not.

However, in order to assert the actual protections type checking gives us, I need to be able to assert Flow errors. For example, let's say I wanted to make sure this wasn't valid:

const user = {name: "Bob", age: 40};

I can make the test pass with

// $ExpectError
const user = {name: "Bob", age: 40};

but I can't make it fail with

// $ExpectError
const user = {name: "Bob"};

Obviously this example is pretty trivial, but more complex types (e.g. with generics) could benefit from this type of testing.

Options I've considered:

• I could potentially run flow as an external process on specific files and then assert that it returned an error, but this would require one file per test case
• I could use flow-runtime to access the types at runtime and use them in unit tests, but this doesn't offer static assertions

Is there any way to statically assert that a Flow assertion should fail, and throw an error if it doesn't?

Answer 1

To assert that certain lines of code in your unit tests should raise Flow errors, just write a Flow error suppression comment above those lines, then run flow check --max-warnings 0 on your entire unit tests folder (or your entire codebase) and assert that the command has a 0 exit status (no errors or warnings). You don’t need to run the commands on each unit test file individually.

As the documentation for suppress_comment says, Flow will raise a warning if you write a suppression comment such as // $FlowExpectError above a line that does not actually trigger a Flow error.

Assuming you defined your suppress_comment to understand $FlowExpectError as a suppression comment, this is what your unit tests would look like:

// this type definition could be imported from another file
type User = {|
    name: string
|};

// Check that Flow doesn’t raise an error for correct usages
const user = {name: "Bob"};

// Check that Flow does raise an error for incorrect usages
// $FlowExpectError
const user = {name: "Bob", age: 40};

If you run flow check --max-warnings 0 on a folder containing this file and the command exits successfully, then your tests for your User type passed.