Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner: fix delete test file cause dependency file change not rerun the tests #53533

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jakecastelli
Copy link
Contributor

@jakecastelli jakecastelli commented Jun 21, 2024

There is an edge case left from #53114

Reproduce step:

// create dependency index.js
export const a = 1;
export const b = 2;
export const c = 3;
// create test-a.mjs
import { a } from "./index.mjs";

import { test } from "node:test";
import assert from "node:assert";

test("test One", () => {
  assert.equal(a, 1);
});
// create test-b.mjs
import { a, b } from "./index.mjs";

import { test } from "node:test";
import assert from "node:assert";

test("test Two", () => {
  assert.equal(a, 1);
  assert.equal(b, 2);
});
// create test-c.mjs
import { a, b, c } from "./index.mjs";

import { test } from "node:test";
import assert from "node:assert";

test("test Three", () => {
  assert.equal(a, 1);
  assert.equal(b, 2);
  assert.equal(c, 3);
});
node --watch --test

# now delete test-a.mjs
# then make any change to index.mjs e.g. change const a = 1 to const a = 2

You would observe that test-b.mjs and test-c.mjs are not being rerun. The expected behaviour should be test-b.mjs, test-c.mjs being rerun and failed (since assert.strictEqual(a, 1) is not correct anymore).

Change explain:

When a watched test file is being deleted, the test runner will rerun the test(s) and because the test file is no longer there but the dependencyOwners wouldn't be able to know it, so it failed in silence. Since a test file is being deleted, we don't need to rerun anything so we can just safely return.

notes:

The test is quite identical to test/parallel/test-runner-watch-mode.mjs but with a more complicated setup, I am thinking to spend some time to see how I can refactor the test so we can write more complicated test cases in the future, I am committed and would like to do a separate PR for it.

Ref: #53114

When a watched test file is being deleted then the referenced dependency
file(s) will be updated incorrect when `unfilterFilesOwnedBy` method is
called, which will cause tests not being rerun when its referenced
dependency changed. To prevent this case, we can simply `return` when we
detect a watched test file being deleted.
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner
@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Jun 21, 2024
@jakecastelli jakecastelli changed the title test_runner: fix delete test file cause dependency file not watched Jun 23, 2024
lib/internal/test_runner/runner.js Outdated Show resolved Hide resolved
test/parallel/test-runner-watch-mode-complex.mjs Outdated Show resolved Hide resolved
@@ -0,0 +1,100 @@
// Flags: --expose-internals
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used only for util.createDeferredPromise?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it only used for util.createDeferredPromise

test/parallel/test-runner-watch-mode-complex.mjs Outdated Show resolved Hide resolved
test/parallel/test-runner-watch-mode-complex.mjs Outdated Show resolved Hide resolved
test/parallel/test-runner-watch-mode-complex.mjs Outdated Show resolved Hide resolved
jakecastelli and others added 2 commits June 28, 2024 22:12
Co-authored-by: Chemi Atlow <chemi@atlow.co.il>
@jakecastelli jakecastelli force-pushed the fix-test-runner-watch-dependency-update branch from d79b8d7 to e270e58 Compare June 28, 2024 12:42
@atlowChemi atlowChemi added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 28, 2024
@atlowChemi atlowChemi removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 29, 2024
@jakecastelli
Copy link
Contributor Author

maybe we can get CI started again? 👀

@atlowChemi atlowChemi added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 7, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 7, 2024
let currentRun = '';
const runs = [];

child.stdout.on('data', (data) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could just be .toArray()on child.stdout which would also save the util.createDeferredPromise and would make it easier to listen to stdout

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(just make sure to not await it before you trigger the watch)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I get a little bit of help here? I tried to use child.stdout.toArray() - but I cannot seem to be able to control the test runs, I cannot manage to get 2 runs (get result for initial run 3/3 pass, delete a test file modify another test to get the result of second run 2/2 pass) successfully, I can only get result for 1 run

I must be doing something incorrectly.

Would you mind explaining not await to what?

(just make sure to not await it before you trigger the watch)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.
5 participants