---
This PR is an example of how to update fixture to add sprout annotations.
Running with `yarn sprout --verbose --filter` shows the fixture outputs.
```
PASS array-access-assignment
ok [[2],[[2],[],[3]]]
PASS array-expression-spread
ok [0,1,2,3,null,4,5,6,"z"]
PASS array-map-frozen-array
ok [[],[]]
PASS array-map-mutable-array-mutating-lambda
ok [[],[]]
PASS array-pattern-params
ok [{"a":"val1"},{"b":"val2"}]
PASS array-properties
ok {"a":[[1,2],2,"hello"],"x":3,"y":"[[ function params=1 ]]","z":"[[ function
params=1 ]]"}
PASS array-property-call
ok {"a":[1,2,"hello",42],"x":4,"y":1}
PASS assignment-expression-computed
ok [7]
PASS assignment-expression-nested-path
ok {"b":{"c":6}}
PASS capturing-func-mutate-2
ok {"a":2}
PASS capturing-function-alias-computed-load-2
ok "val2"
PASS capturing-function-alias-computed-load-4
ok "val2"
```
---
Rename `SproutOnlyFilterTodoRemove.ts` to `SproutTodoFilter.ts`. I've tried to
group the skipped fixtures by difficulty and add comments about what need to be
done, also happy to change the structure of this.
From this point, sprout will run on all new fixtures by default. If the fixture
forgets to export a `FIXTURE_ENTRYPOINT`, sprout will fail with this error.
<img width="853" alt="Screenshot 2023-08-15 at 5 59 32 PM"
src="https://github.com/facebook/react-forget/assets/34200447/0f80d650-1dc1-4df4-9710-e49acbb424b0">
See #1961 for an example of how to annotate existing skipped fixtures or new
ones. The `README.md` is also updated with a guide.
---
```
yarn sprout --verbose
```
Verbose mode prints out all outputs of tests. The test output is a status (`ok`
or `exception`), a returned or thrown value, and a set of console logs.
Currently as of #1960 , this is the output:
```sh
$ yarn workspace babel-plugin-react-forget run build && node
../sprout/dist/main.js --verbose
$ rimraf dist && tsc
PASS alias-nested-member-path
ok {"y":{"z":[]}}
PASS assignment-variations-complex-lvalue
ok {"y":{"z":4}}
PASS assignment-variations
ok 1
PASS chained-assignment-expressions
ok {"z":null}
PASS computed-call-evaluation-order
ok {"f":"[[ function params=0 ]]"} [
"A",
"B",
"arg",
"original"
]
PASS const-propagation-into-function-expression-primitive
ok 42 [
"42"
]
PASS constant-propagation-for
ok 0
PASS constant-propagation-while
ok 0
PASS constant-propagation
ok -6 [
"foo"
]
PASS controlled-input
ok <input value="0">
PASS do-while-continue
ok [1.5,1,0.5]
PASS do-while-simple
ok [6,4,2]
PASS expression-with-assignment
ok 5
PASS for-of-break
ok []
PASS for-of-conditional-break
ok []
PASS for-of-continue
ok [0.5,1,1.5]
PASS for-of-destructure
ok [0,2,4]
PASS for-of-simple
ok [0,2,4]
PASS function-declaration-reassign
ok {}
PASS function-declaration-redeclare
ok "[[ function params=0 ]]"
PASS lambda-reassign-primitive
ok 41
PASS lambda-reassign-shadowed-primitive
ok {}
PASS property-call-evaluation-order
ok {"f":"[[ function params=0 ]]"} [
"A",
"arg",
"original"
]
PASS reactive-scope-grouping
ok {"y":[{}]}
PASS sequentially-constant-progagatable-if-test-conditions
ok "ok"
PASS simple-function-1
ok "[[ function params=1 ]]"
PASS ssa-complex-multiple-if
ok
PASS ssa-complex-single-if
ok
PASS ssa-for
ok 11
PASS ssa-if-else
ok
PASS ssa-objectexpression-phi
ok {"x":1,"y":3}
PASS ssa-property-call
ok {"x":[[]]}
PASS ssa-property
ok {"x":[]}
PASS ssa-return
ok 2
PASS ssa-simple-phi
ok
PASS ssa-simple
ok
PASS ssa-single-if
ok
PASS ssa-switch
ok
PASS ssa-throw
exception undefined
PASS ssa-while
ok 10
PASS type-field-load
ok 1
PASS type-test-field-store
ok {}
PASS type-test-primitive
ok 2
PASS update-expression-constant-propagation
ok {"a":0,"b":0,"c":2,"d":2,"e":0}
44 Tests, 44 Passed, 0 Failed
✨ Done in 9.27s.
```
---
Not sure why, but snap kept silently crashing when I used fs/promises to write
to many file handles. I tried a `.catch(...)`, but couldn't figure it out. This
diff changes snap to use sync fs apis to avoid crashes, but I'd love to get
feedback if someone knows how to debug this.
This PR moves the phi evaluation to a separate function.
Most importantly, it inverts the default case to _not_ constant propagate unless
we have explicit validation of the phi operands.
---
## Sprout 🌱 Overview
**(Overview copied from
[sprout/README.md](https://github.com/facebook/react-forget/blob/0468ddf8bbee54a9394317a667ea4ed96bb3bef3/forget/packages/sprout/README.md))**
React Forget test framework that executes compiler fixtures.
Currently, Sprout runs each fixture with a known set of inputs and annotations.
We hope to add fuzzing capabilities to Sprout, synthesizing sets of program
inputs based on type and/or effect annotations.
Sprout is currently WIP and only executes files listed in
`src/SproutOnlyFilterTodoRemove.ts`.
### Milestones:
- [✅] Render fixtures with React runtime / `testing-library/react`.
- [ ] Make Sprout CLI -runnable and report results in process exit code.
After this point:
- Sprout can be enabled by default and added to the Github Actions pipeline.
- `SproutOnlyFilterTodoRemove` can be renamed to `SproutSkipFilter`.
- All new tests should provide a `FIXTURE_ENTRYPOINT`.
- [ ] Annotate `FIXTURE_ENTRYPOINT` (fn entrypoint and params) for rest of
fixtures.
- [ ] Edit rest of fixtures to use shared functions or define their own helpers.
- [ ] *(optional)* Store Sprout output as snapshot files. i.e. each fixture
could have a `fixture.js`, `fixture.snap.md`, and `fixture.sprout.md`.
### Constraints
Each fixture test executed by Sprout needs to export a `FIXTURE_ENTRYPOINT`, a
single function and parameter set with the following type signature.
```js
type FIXTURE_ENTRYPOINT<T> = {
// function to be invoked
fn: ((...params: Array<T>) => any),
// params to pass to fn
params: Array<T>,
// true if fn should be rendered as a React Component
// i.e. returns jsx or primitives
isComponent?: boolean,
}
```
Example:
```js
// test.js
function MyComponent(props) {
return <div>{props.a + props.b}</div>;
}
export const FIXTURE_ENTRYPOINT = {
fn: MyComponent,
params: [{a: "hello ", b: "world"}],
isComponent: true,
};
```
---
## Implementation Details
- jest-worker test orchestrator (similar to Snap 🫰).
I chose to write a test runner instead of directly using Jest for flexibility
and speed.
- Sprout 🌱 currently runs much more code per fixture (2 babel transform
pipelines + 2 `exec(...)` than snap, so all scaling concerns apply.
- Sprout may need more customization in the future (e.g. fuzzing component
inputs, caching artifacts)
- We probably want to add snapshot files for Sprout, which is much easier with a
custom runner.
This is also one of the main reasons we wrote Snap. Jest consolidates all
external snapshots (i.e. non-inline snapshots) from a test into [a single
file](https://github.com/facebook/react-forget/blob/d0a006ffa9842ee9db6b23ada94622af901eb07f/forget/src/__tests__/__snapshots__/compiler-test.ts.snap).
This was painful mainly for rebasing changes, but also for small papercuts (e.g.
needing to run `yarn test -u` twice when deleting a fixture)
- Currently does not save output to snapshot file, but we can easily add this
later (i.e. each test would have a `.snap.md` and `.sprout.md` file)
- Supports filter mode (same testfilter.txt file as snap)
- Currently does not support watch mode. I expect that sprout's primary use will
be catching bugs in the PR / Github Actions phase. Can be changed if we need to
iterate on sprout output while developing.
- All tests are run with `react-test-renderer` to access the React Runtime,
which is needed for calls to `useMemoCache` (and other potential hooks).
- react-test-renderer required a mocked DOM, so I ~~used js-dom~~ did a terrible
js-dom hack to add document, window, and other browser globals to the
jest-worker globals, then exec the test code in the jest worker global.
I can clean this up later by bundling library code (e.g. react,
react-test-renderer) to not use `require(...)`, then calling `exec` on all "test
client code" in the js-dom mock global (instead of the real jest worker one)
- Tests marked `isComponent` need to return valid jsx or primitives. Values
returned by all other tests are converted to a string primitive via
JSON.stringify.
```sh
# install new dependencies to node_modules
$ yarn
$ cd forget/packages/babel-plugin-react-forget
$ yarn sprout:build && yarn sprout
PASS alias-nested-member-path
PASS assignment-variations-complex-lvalue
PASS assignment-variations
PASS chained-assignment-expressions
PASS computed-call-evaluation-order
PASS const-propagation-into-function-expression-primitive
PASS constant-propagation-for
PASS constant-propagation-while
PASS constant-propagation
...
✨ Done in 3.91s.
```
Nice find from @mofeiz, we weren't adding declarations for function params in
LeaveSSA. This caused us to hit an invariant for update expressions that updated
params which assumed that all named variables had a declaration. This is why
it's helpful to add invariants, it helps you find places where you violate them
:-)
Adds basic support for hoisting semantics: * Resolution of variable references
is _always_ deferred in case the correct binding hasn't been seet yet due to
hoisting. * Var and function declarations bubble to the appropriate scope
There are lots of subtleties that aren't implemented yet but these rules cover a
lot.
This is a precursor to adding support for hoisting in semantic analysis.
Previously when we encountered an unknown reference we immediately reported an
error. But hoisted variables may be referenced before they're defined, so we
don't know for sure when we see an unknown variable if its actually unbound or
not.
This PR adds the first part of hoistingn support: rather than immediately report
an error when encountering an unbound variable we store it in a list of
unresolved references on the current scope. As we close each scope we recheck
and see if the variable can now be resolved. If yes we record that, otherwise we
bubble up the unresolved reference to the parent scope (and try again there).
The next PR(s) will handle hoisting of `var` and other syntax to the apropriate
nearest scope boundary (function/module).
When updating the data model for operands from instruction indices to
identifiers, I forgot to rewrite terminal operands.Doing so required a refactor
to use the new BlockRewriter helper.
Adds a `Destructure` instruction closely following the design of the TS-based
compiler. The main change is fairly small: the TS compiler doesn't support rest
spreads that are not identifiers, such as the `...y[]` in `const [x, ...[y]] =
z`. I added support for this in the Rust compiler.
`compileProgram` was getting complex, so this extracts some of the logic into
smaller functions. Additionally, the `try` block now only wraps the `compileFn`
generator from Pipeline, which means not accidentally catching other non-Forget
errors
Initial data types for `forget_reactive_ir`, which is the Rust analogue of
`ReactiveFunction` in the TS compiler. I'm renaming here for clarity, though
naming suggestions are very welcome!
Aligns the data model for `Instruction` closer to JS: * Adds an `lvalue:
IdentifierOperand` property (Identifier + Effect) * Changes rvalues from being a
reference to the definining instruction by instruction offset (InstrIx) to be
a variable reference (IdentifierOperand)
There are pros and cons to the previous offset-based approach. It's definitely
convenient to be able to jump directly to the instruction that defined an
operand value. However: * Many passes need to track things like basic
reassignments, which mean they need to build up mappings of IdentifierId to
some data. When all operands are IdentifierOperands, this can be a single
mapping. * It aligns closer to JS, which makes porting a bit easier.
But most of all: porting the `ReactiveFunction` data type — which is in tree
form - is non-trivial with the offset-based approach. As we map HIR to the tree
form, we'd have to remap every operand offset. Using identifiers for operands
simplifies this.
We can always revisit this design choice later.
After the previous PR we no longer depend on SWC for the critical path of
development/testing. Notably this unblocks adding support for the rest of the
language: the new `forget_hermes_parser` crate automatically converts Hermes
Parser's AST into our `forget_estree` format via codegen. Achieving full
language support with SWC would have required manually defining the remaining
conversions for the rest of the language.
Long-term we'll need to revisit how to integrate into SWC, and more generally
into setups build atop SWC such as Next.js's Turbopack-based build
configuration. But i'll delete for now since we don't depend on it for
iteration, it's slow to build, and requires us to opt-in to nightly Rust.
Replaces the use of SWC parser in the Forget fixture tests with Hermes Parser.
This includes aligning on a single type to represent JS Values and numbers
(combining semi-duplicated code from forget_hir and forget_estree) and adding
support for lowering babel-style
NumericLiteral/BooleanLiteral/StringLiteral/NullLiteral node types (since Hermes
Parser produces a mix of estree/babel node types)
Updates HIR builder to rely on the new semantic analysis instead of assuming
that the ast nodes will already have binding info attached. That was a stopgap
until we had our own name resolution :-)
Once this lands we can remove SWC and switch everything to forget_hermes_parser,
and also remove the non-spec Identifier.binding field (which stored the
temporary name resolution data).
Adds new instructions to accurately model UpdateExpression semantics, since
`x++` is un-intuitively not the same as `x = x + 1`. There are a few different
ways to model the combination of prefix/postfix and increment/decrement:
* One instruction for all combinations of prefix/postfix and
increment/decrement, eg 'UpdateExpression'
* Instructions for Increment/Decrement, each with a property to distinguish
prefix/postfix
* Instructions for Prefix/Postfix, each with aproperty to distinguish
increment/decrement.
I chose the latter, `PrefixUpdate` and `PostfixUpdate`, because it keeps the
number of new instructions minimal while keeping separate instructions for the
most important distinction: whether the result of the instruction is the value
before applying the operation or after. I'm open to suggestions about this
though.
A few quick notes:
* Constant propagation is supported but only for numbers (we don't support
bigint yet anyway)
* LeaveSSA needs to know about these instructions since their presence requires
making the original variable declaration Let, not Const.
* EnterSSA mapped lvalues before rvalues, which is out of order but didn't
previously matter. I just had to flip the order and everything worked.
Lauren Tan
Mofei Zhang
Sathya Gunasekaran
Joe Savona