Global failure: Trying to do a test run with zero assertions fails

[jayjayjpg]

Tell us about your runtime:

• QUnit version: 2.7.1
• What environment are you running QUnit in? (e.g., browser, Node): Both happening in browser and Node env. Browser: Chrome Version 77.0.3865.56, Node v10.5
• How are you running QUnit? (e.g., script, testem, Grunt): Ember QUnit / testem, Node v10.5.0

What are you trying to do?

I try to run a test set, e.g. a filtered test set using the --filter of the QUnit CLI, that contains zero assertions. Each test case that contains zero assertions is marked as such using assert.expect(0).

Why are you trying to do this?

Sometimes I want to write tests which use another assertion library, making the use of QUnit's assert obsolete. This might make filtered test runs using --filter or --module impossible for certain test sets.

Code that reproduces the problem:

Please check out the minimal reproduction of the issue here.

When in the reproduction repository, only running

qunit

will run both test files in the test directory. You should see that the test run passes:

---

To reproduce the error, run

qunit --filter="fail"

You should see that one test file was run, but that the test run fails with not ok global failure:

---

If you have any relevant configuration information, please include that here:

What did you expect to happen?

I expect a test run that contains zero assertions to pass without a global failure error as long as I declare these tests with the assert.expect(0) method as assertionless.

What actually happened?

When trying to run a test and the test run happens to contain in total zero assertions, the run will fail with a global failure error. The number of tests run will be displayed as passed and one additional failing assertion is displayed in the test output.

[Krinkle]

Thanks for reporting this. When a test suite contains actual tests, but uses assert.expect(0) that should still be counted as a test indeed.

[jayjayjpg]

@Krinkle That's awesome, thank you for reproducing this. If I can help with contributing, feel free to let me know!

[Krinkle]

@jayjayjpg Patch is welcome! What it comes down to is that we currently determine "No tests run" based on assertions.

Look around for stats, to find the three relevent spots for this issue:

• core.js: Defines the running count of assertions.
• test.js: Adds assertions length to the running count.
• processing-queue.js: Checks the count after all is said and done, to decide whether to throw.

I would recommend adding a count of tests alongside it, and use that for this purpose instead.

[jayjayjpg]

@Krinkle Thank you for the details! I just provided a first PR over at #1449 - feel free to let me know if this requires any further changes to be merged and I can iterate on it