Speed up unit test suite performance

[mourner]

While trying to figure out why yarn run test-unit on the CI takes 6 minutes (which seems unreasonable), I discovered that there's a lot of overhead because we're running a new node process for every unit file. The ESM overhead for every file is typically 2–5 seconds (even if the tests inside run in 20ms), and there are 96 of them — each basically doing a custom build of Mapbox GL (depending on what's imported), that's executed by the VM after being built. The code is highly coupled, so for many files, those builds end up being pretty big.

Generally a 6-minute run isn't a problem because we can call individual test files when developing locally, and it's in parallel to 7-minute render tests, but reducing this time back to 2 minutes will free up Circle jobs, benefitting everyone.

I'm not sure what approach to take here though:

• Tried iterating over all files and requiring them in the same context, but apparently some tests (and imports like jsdom) cause problems for other (unrelated) tests.
• We could manually merge batches of test files together, but that will be quite a cumbersome undertaking and we'll have to find a compromise between maintainability and test suite performance.
• I wonder if using --experimental-modules instead of esm would solve this — it could. Although switching away from esm will also be cumbersome (involving renaming all files to .mjs, and potentially changing the way we use CommonJS modules because of stricter interop).

Ideas? Should we bother?

cc @jfirebaugh

[anandthakker]

involving renaming all files to .mjs

@mourner FWIW, I think there's a way we could add a custom Node loader hook to treat .js files as modules.

[asheemmamoowala]

This has been reduced back down to <4m with the removal of ESM and switch to experimental module loaders. Closing this for now.

[mourner]

Worth noting that there's still a huge room for improvement — since we still run a separate node process for each unit test file, it runs all the loading and transforming repeatedly over and over again. If we could fix all unit tests to run in a single process, I believe the unit test time could be reduced to 1-2 minutes.