Jest Custom Matcher in Typescript

Posted by The Real One on

I was running along happily, having setup a new TS project using fp-ts and the Jest testing framework with this nifty script. I started validating some test results for a function that returned an Option<bigint> (see: fp-ts/Option).

// Names have been changed to protect the guilty.
it('multiplies ints together, returning bigint', () => {
  expect(fooMultipliesBigInt('1000', '55')).toEqual(O.some(BigInt(55000)));

  expect(fooMultipliesBigInt('121', '11')).toEqual(O.some(BigInt(1330)));
});

Everything ran fine as long as the tests passed. Then I ran into a rather calamitous failure. Note, this was run with jest --watchAll and the timer just started ticking and never stopped. Basically Jest simply crashed and hung, probably awaiting a promise resolution that would never come. 

(node:39088) UnhandledPromiseRejectionWarning: TypeError: Do not know how to serialize a BigInt
    at stringify (<anonymous>)
    at writeChannelMessage (internal/child_process/serialization.js:117:20)
    at process.target._send (internal/child_process.js:805:17)
    at process.target.send (internal/child_process.js:703:19)
    at reportSuccess (C:\Users\sdhms\git\skunky-as-all-get-out\node_modules\jest-worker\build\workers\processChild.js:67:11)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:39088) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:39088) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

RUNS  src/schema/Currency.test.ts

Test Suites: 0 of 1 total
Tests:       0 total
Snapshots:   0 total
Time:        154 s

Ah! The ever pervasive BigInt serialization problem in JS and TS, where someone’s code assumes JSON could be used for serialization of Javascript data, no problem, and all would be well. After all that is the stated purpose of JSON, right?

Along comes something new, something really useful and necessary which clashes with something even more useful and important like JSON and VOILA! we have a catastrophic failure of two intrinsic, endemic and necessary features, one an evolved extension and the other a beloved and tried and true feature. Such is the pain of change. Unfortunately this particular issue cannot be fixed easily in the context of the community and the language, because the creators of JSON made it impossible to do so. Sadly, this quandary is a great example of failing to build for extensibility: JSON can only be parsed as such and it cannot tolerate metadata nor new forms. It is also a topic for another day.

Now back to the problem. If we are willing to “patch” our JSON serialization in a necessarily limited context, we can work around this for the purpose of testing. Any fix to the JSON serialization is “one-way” as we cannot also fix the JSON parser to deserialize the data effectively, as it is considered a string. But this is OK for these purposes, if we can shim JSON for BigInt serialization our problem should go away. Yes?

So, as I’ve done in the past with Mocha tests on other projects, I added this to the start of my failing test file (it could also probably be added to setup scripts):

// @ts-ignore
BigInt.prototype.toJSON = function() { return this.toString() };

Then I killed the test process that was running and started it over, expecting all to be well. And what did I get? No difference:

(node:39088) UnhandledPromiseRejectionWarning: TypeError: Do not know how to serialize a BigInt
    at stringify (<anonymous>)
    at writeChannelMessage (internal/child_process/serialization.js:117:20)
    at process.target._send (internal/child_process.js:805:17)
    at process.target.send (internal/child_process.js:703:19)
    at reportSuccess (C:\Users\colt-pearson\git\project-foo\node_modules\jest-worker\build\workers\processChild.js:67:11)
(Use `node --trace-warnings ...` to show where the warning was created)
BLAH BLAH BLAH BLAH BLAH

Yikes! I even put some debug in after the shim to verify it worked:

// Needed to keep serialized bigint from crashing
// @ts-ignore
BigInt.prototype.toJSON = function() { return this.toString() };

console.log(JSON.stringify(BigInt(100)));

The shim functioned just fine, note the console log stuff after the error:

(node:39664) UnhandledPromiseRejectionWarning: TypeError: Do not know how to serialize a BigInt
    at stringify (<anonymous>)
... some stuff deleted because, you know, space and time ...
(node:39664) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
  console.log
    "100"

At this point, one might assume that the conjecture about JSON being used to serialize BigInt was not correct, however unlikely, and that some other serialization was being used and it was not JSON but still didn’t work with BigInt.

One might also hypothesize that the JSON parser was being used (note the error message is identical to a JSON parser error), but that it was in a different namespace or code module and thus not accessible. This conjecture, not being refuted in any way, seems more likely.

Having encountered this road block, one might eventually, after some wrangling and some research, come up with a nice solution to work through the problem. This imagined soul on their Hero’s Journey might just leverage the JSON patch and the feature of Jest that allows for custom matchers and come to this:

function toEqualOption<A>(
  this: jest.MatcherContext,
  received: O.Option<A>,
  expected: O.Option<A>
): jest.CustomMatcherResult {
  const failMessage = () => `${diff(expected, received)}`;
  return {
    message: failMessage,
    pass: this.equals(JSON.stringify(received), JSON.stringify(expected)),
  }
}
expect.extend({
  toEqualOption,
});

// Needed to keep the matcher from failing to be typed properly
declare global {
  namespace jest {
    interface Matchers<R> {
      toEqualOption<A>(expected: O.Option<A>): R;
    }
  }
}

describe('fooey test for demo', () => {
  it('multiplies ints together, returning bigint', () => {
    expect(fooMultipliesBigInt('1000', '55')).toEqualOption(O.some(BigInt(55000)));
    expect(fooMultipliesBigInt('121', '11')).toEqualOption(O.some(BigInt(1330)));
  });
});

Which, by the way, works beautifully:

    - Expected
    + Received

      Object {
        "_tag": "Some",
    -   "value": 1330n,
    +   "value": 1331n,
      }

There is some cool stuff in that matcher code, and I hear tell that jest-matcher-utils has even more stuff to help make the message even more like stock Jest matcher messaging. The types are clean, this can be used effectively to access the Jest context info, and the final test and messaging is effective and accurate (if not completely like stock code).

And now all is well. The testing work can continue, and that brave fellow who crafted the solution can post a blog entry showing how it all worked out for the best. But wait! Not so fast…

Having run into this problem yesterday, I stopped for the night while developing the custom matcher. I had it working but it was poorly typed and I wanted to get it to a healthier place before calling it done. So tonight when I started the blog post, I returned the original matcher into place to generate the error so I could copy it in here (since the error was the same before and after, one copy would suffice, right?). I returned the failing test to using .toError() 

it('multiplies ints together, returning bigint', () => {
  expect(fooMultipliesBigInt('1000', '55')).toEqualOption(O.some(BigInt(55000)));

  expect(fooMultipliesBigInt('121', '11')).toEqual(O.some(BigInt(1330)));
});

and you can guess what I saw:

    expect(received).toEqual(expected) // deep equality

    - Expected  - 1
    + Received  + 1

      Object {
        "_tag": "Some",
    -   "value": 1234567890000000000057n,
    +   "value": 1234567890000000000058n,
      }

Huh?!? Well, hard to refute the evidence itself – clearly the .toError() construct, which failed horribly after adding the shim was now working fine with the shim in place. So what did I do in the meantime that might have affected the tests? What is going on with this JSON parser that it both did and didn’t get the shim? I honestly don’t know. Jest must be doing some code caching behind the scenes, which makes sense because the first run of Jest is much slower than subsequent runs.

What is especially interesting – the thing that still puzzles me – is that the sequence of development during the crafting of this blog is EXACTLY as stated in real time. That shim started working just fine with toEqual() immediately after it worked just fine with toEqualOption().

So now that I’m down the rabbit hole, yet again, time to eat some cake and drink some tonic until I get everything just to the right size and have myself a solution:

  1. Rerun the test task. The test suite runs correctly, with the test failing politely.
  2. Remove the shim. The test fails catastrophically, hanging indefinitely.
  3. Add the shim back. The test fails catastrophically, again!
  4. Change .toEqual() to .toEqualOption(). The test suite runs correctly with the custom message, and once again all is well.
  5. Change .toEqualOption() to .toEqual(). The test suite runs correctly, with the stock message, and something is not quite right with the universe.
  6. Remove the shim again. The tests ran normally… ALMOST! This time the .toEqualOption() test that should have passed failed with a thrown error, not an unresolved promise, stating TypeError: Do not know how to serialize a BigInt, but the .toEqual() test failed with the stock message, not a serialization error. Once again… something not quite right here.
  7. Shim in – 1 test fails
  8. Shim out – 2 tests fail (like step 6). So how do I get back to the failure we saw in steps 2 and 3?
  9. Quit out of the Jest watch task and restart it. Same results as step 8.
  10. Quit out of the Jest watch task, run npm install, restart task. Same results as step 8
  11. Quit out of the Jest watch task, clear node_modules directory, run npm install and restart task. Same results as step 8.
  12. Change the .toEqualOption() that passes with the shim but currently fails to .toEqual(). Test fails catastrophically.
  13. Change it back to .toEqualOption(). Test fails catastrophically.
  14. Add the shim back in. Test *still* fails catastrophically.
  15. Change the .toEqual() that fails back to .toEqualOption(). Test runs normally with custom failure message.
  16. Change the .toEqualOption() that fails back to .toEqual(). Test fails catastrophically. So now we are in the same state as #5 but with a different result.
  17. Change the .toEqual() that fails back to .toEqualOption(). Test runs normally with custom failure message.
  18. Change the .toEqualOption() that fails back to .toEqual(). Test runs normally with stock failure message.
  19. Remove the shim. 2 test failures, custom fails without serializer, stock matcher still running with the shim.
  20. Stop the task, restart it. Test remains unchanged, same as 19.
  21. Stop the task, remove /tmp/jest, restart task. Test remains unchanged, same as 19.
  22. Stop the task, restart it. Test fails catastrophically.
  23. “Well, I’ve had enough nonsense! I’m going home.”

Clearly Jest is lifting the custom matcher runtime context into some sort of cache used by the stock matcher context, but only upon customer matcher failure (invoking the message() function in the custom matcher results). If the test passes, only the pass() call is being made, which YES does use the shim. So step 21 above *should* have demonstrated this caching, but it appeared to refute it.

Since we can’t go back to yesterday. Let’s try this.

  1. Start with NO SHIM and comment out the custom matcher, use .toEqual() stock matcher for all assertions.
  2. Run the test task. It fails catastrophically, as one would expect.
  3. Add the shim. Still fails catastrophically, as we have demonstrated early on.
  4. Remove /tmp/jest. Test runs normally.
  5. Make an inconsequential change with the watcher running, causing the tests to rerun. Test fails catastrophically. Eureka!

OK, so definitely a caching issue. It turns out that Jest is scraping dependencies, raking the file system, and REUSING cached dependencies. So essentially on the first run of a watcher task the JSON module used by the custom matcher is the SAME as the one used by the stock matcher, but in subsequent runs the cached JSON is used.

While this conjecture isn’t entirely the whole picture, and it does not and cannot, explain all the funnies we’ve seen, it is sufficient enough to tell us a sobering reality. We cannot trust the Jest cache. Another reality to accept, the JSON shim won’t work when using bigint with Jest. This is sad and must needs a remedy.

And we’ll leave that for another day. I’ve had enough nonsense. I’m going home.

Leave a comment