Allow multiple trees as input to Filter #175

SparshithNR · 2019-08-23T00:44:12Z

Using fs-merger
1. Allow filter developers to reduce mergeTree or funneling entire array of trees to one tree and then pass it to filter
2. Allows input array of trees as input.
input/output :
1. Provides good ergonomics and avoids appending the this.outputPath
Test:
1. Added additional tests to make sure that Filter can take more than one tree

Example:

class Awk extends Filter {
  constructor(inputNode, search, replace, options) {
    options = options || {};
    super(inputNode, {
      annotation: options.annotation
    });
    this.search = search;
    this.replace = replace;
    this.extensions = ['txt'];
    this.targetExtension = 'txt';
  }
  processString(content, relativePath) {
    return content.replace(this.search, this.replace);
  };
}
module.exports = new Awk(['fixture','fixture_2'], 'test', 'real');

Since we are now allowing users to pass two paths/nodes. We avoid a uncessary merge/funnel.
Before to achive the above result we had to below sinppet.

let fullTree = mergeTree(['fixture','fixture_2'];
module.exports = new Awk(fullTree, 'test', 'real');

chriseppstein

I didn't review everything, but I have enough questions to pause at this point.

index.js

lib/dependencies.js

index.js

test/test.js

index.js

package.json

Using fs-merger 1. Allow filter developers to reduce mergeTree or funneling entire array of trees to one tree and then pass it to filter 2. Allows input array of trees as input. input/output : 1. Provides good ergonomics and avoids appending the this.outputPath Test: 1. Added additional tests to make sure that Filter can take more than one tree

stefanpenner · 2019-12-12T19:02:41Z

index.js

@@ -104,8 +101,19 @@ module.exports = class Filter extends Plugin {
    return !!result;
  }

-  constructor(inputTree, options) {
-    super([inputTree], {
+  get fsMerger() {


as implemented fsMerger is still part of the public API for subclasses, this seems sorta strange given the above weakmap usage.

index.js

SparshithNR · 2019-12-12T21:50:39Z

Input output facade is addressed in #187.
We will address allowing multiple trees as input in this PR.

stefanpenner · 2019-12-16T20:01:16Z

index.js

    try {
      let result;
-      let srcPath = srcDir + '/' + relativePath;
+      let srcDir = this.inputPaths[0]; // keeping this line to maintain the signature of the fn.


Using this.inputPaths[0] here is breaking encapsulation, post this.input and this.output migration it should be possible to use only the new APIs, what features are missing or different approaches are required to make this statement true.

stefanpenner · 2019-12-16T20:01:26Z

index.js

        let forceInvalidation = invalidated.includes(relativePath);

        this._logger.debug('[operation:%s] %s', operation, relativePath);

        switch (operation) {
          case 'mkdir': {
            instrumentation.mkdir++;
-            return fs.mkdirSync(outputPath);
+            return this.output.mkdirSync(outputPath, { recursive: true });


@SparshithNR ^^

stefanpenner · 2019-12-16T20:01:41Z

index.js


    if (this.dependencyInvalidation && !this.dependencies) {
-      this.dependencies = this.processor.initialDependencies(srcDir);
+      this.dependencies = this.processor.initialDependencies(this.inputPaths);


stefanpenner · 2019-12-16T20:02:40Z

index.js

@@ -121,6 +129,10 @@ module.exports = class Filter extends Plugin {
      loggerName += ' > [' + annotation + ']';
    }

+    FSMERGER.set(this, {


Currently I am worried, that this plugin does not use this.input, did we make a mistake with it? By not using this.input we are making it hard for future optimizations, as ensuring all plugins go through this.input this.output opens many doors when it comes to future optimizations.

We may have to make changes to the fs-merger library to somehow match inputPaths and input to broccoli-persistent-filter where we expect input can be broccoli-node string or an object with prefix/getDestinationPath function.

SparshithNR · 2020-05-12T17:38:09Z

Closing this as #188 is addressing this

SparshithNR force-pushed the in-out-changes branch from 34be033 to 37f31d9 Compare August 23, 2019 20:30

SparshithNR closed this Aug 28, 2019

SparshithNR force-pushed the in-out-changes branch from 9f820b2 to 7ba9626 Compare August 28, 2019 18:13

SparshithNR reopened this Aug 28, 2019

SparshithNR force-pushed the in-out-changes branch from ca98c0d to b760b22 Compare August 28, 2019 18:17

SparshithNR changed the title ~~[WIP] Using fs-merger module to facilitate array of trees as input to persistent filter~~ Aug 28, 2019

chriseppstein suggested changes Sep 11, 2019

View reviewed changes

index.js Outdated Show resolved Hide resolved

index.js Outdated Show resolved Hide resolved

index.js Outdated Show resolved Hide resolved

index.js Outdated Show resolved Hide resolved

index.js Outdated Show resolved Hide resolved

lib/dependencies.js Outdated Show resolved Hide resolved

SparshithNR force-pushed the in-out-changes branch from d084c3e to 9e01bda Compare December 6, 2019 22:40

SparshithNR changed the title ~~Using fs-merger module to facilitate array of trees as input to persistent filter~~ Dec 6, 2019

stefanpenner requested changes Dec 9, 2019

View reviewed changes

SparshithNR force-pushed the in-out-changes branch from 9e01bda to bbe5f19 Compare December 10, 2019 00:23

SparshithNR added 2 commits December 9, 2019 16:29

Remove unwated constructor

90f7ede

use input.lstatSync

53b5ffa

stefanpenner reviewed Dec 12, 2019

View reviewed changes

SparshithNR changed the title ~~Use inout/output feature from broccoli-plugin~~ Dec 12, 2019

stefanpenner requested changes Dec 16, 2019

View reviewed changes

SparshithNR closed this May 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow multiple trees as input to Filter #175

Allow multiple trees as input to Filter #175

SparshithNR commented Aug 23, 2019 •

edited

Loading

chriseppstein left a comment

stefanpenner Dec 12, 2019 •

edited

Loading

SparshithNR commented Dec 12, 2019

stefanpenner Dec 16, 2019

stefanpenner Dec 16, 2019

stefanpenner Dec 16, 2019

stefanpenner Dec 16, 2019

SparshithNR Dec 17, 2019

SparshithNR commented May 12, 2020

Allow multiple trees as input to Filter #175

Allow multiple trees as input to Filter #175

Conversation

SparshithNR commented Aug 23, 2019 • edited Loading

chriseppstein left a comment

Choose a reason for hiding this comment

stefanpenner Dec 12, 2019 • edited Loading

Choose a reason for hiding this comment

SparshithNR commented Dec 12, 2019

stefanpenner Dec 16, 2019

Choose a reason for hiding this comment

stefanpenner Dec 16, 2019

Choose a reason for hiding this comment

stefanpenner Dec 16, 2019

Choose a reason for hiding this comment

stefanpenner Dec 16, 2019

Choose a reason for hiding this comment

SparshithNR Dec 17, 2019

Choose a reason for hiding this comment

SparshithNR commented May 12, 2020

SparshithNR commented Aug 23, 2019 •

edited

Loading

stefanpenner Dec 12, 2019 •

edited

Loading