snapshotting
As demo'd on our sync. This is meant as a replacement for `yarn test` just for
fixtures. On my machine, a test run from a steady state of Jest watch mode takes
6 seconds. With this script, it takes ~800ms.
Workflow: make edits in `packages/snap/`, then `yarn build` in that directory to
build the test runner.
To run tests, `node packages/snap/dist/runner.js` from the main forget/
directory to run tests. Pass `--watch` for watch mode, `--update` to update
snapshots. Note that this _only_ updates .expect.md files, it does not update
Jest's `__snapshots__` directory (this is intentional, our use of Jest snapshots
is a hack that this script is meant to replace).
When running in watch mode, ctrl-c or q will quit, 'u' will update snapshots,
and any other key will re-run tests. Tests will re-run on changes to the source
code (after an incremental TS rebuild) and on changes to the fixtures.
Main things that are missing:
* Don't run tests if TS compilation had errors (the errors should be logged to
console already, but we need to wire that up so that we abort tests if there are
errors)
* Actually delete stale files in update mode (there is some commented-out code
to double-check and then uncomment)
* Improve diff view: just print the diffed segments, not the whole file diff.
See the API at https://github.com/facebook/jest/tree/main/packages/jest-diff
(right now we're using `diff()` but should probably use one of the lower-level
functions)
* Properly parse/validate args with `yargs`, add a help option, etc
* In watch mode, when tests finish print a list of commands so users know what
they can do (like Jest does)
* Support skipping tests and selecting a single test to focus. Suggestions:
* Name files with `skip.` to skip
* Allow passing a test name to the script to run only tests matching that
pattern, eg `node runner.js -p <pattern>`
* In watch mode, support typing 'p' to prompt the user for a pattern. After
that, support typing 'a' to clear the pattern and run all tests.
Fix for bug demonstrated in #1506. When we add variables as output of their
defining scope, we need to propagate this information upwards to all parent
scopes which are not current active.
Minimal(ish) repro of a bug we saw internally, where an output of a nested
reactive scope is defined at the wrong block scope, and so later references to
that value are invalid.
The simplified structure is:
```
scope0 inputs=[] outputs=[] {
scope1 inputs=[] outputs=[t0] {
t0 = ...
}
}
t0
```
Note that `t0` correctly appears as an output of the inner scope1, but not as an
output of the outer scope0. We need to propagate outputs upward as necessary to
ensure they are available at the right block scope: in this case, that would add
`t0` as an output of scope0.
An earlier version of PropagateScopeDependencies did this but it looks like it
got lost along the way (not a big deal)
This aligns the playground configuration with our internal compiler
configuration to make it easier to repro compilation issues on playground. There
is a bug that doesn't repro right now and i suspect it's because of different
hooks being configured.
Test plan:
Before: playground output has no obvious bugs, but is different than internal
compilation output where the bug occurs
After: playground output matches internal compilation output w the bug
Making ReturnTerminal.value non-nullable broke our optimization to elide final
value-less return statements. We now check if a return value is explicitly
`undefined` and elide the value in this case, which then also propagates to
allow removing the final `return` statement of a function if the value is
missing.
Updates our Babel plugin to add an import to React if we succesfully compiled a
function and cached one or more values. For now this logic is entirely in the
Babel plugin itself, but long term we should move this into codegen and teach
Forget to start the pipeline by lowering the whole Program node to an HIR (which
would also allow us to understand imports and other module scope values being
used). Right now it's not possible to add in codegen (without some ugly code) as
a file containing multiple components would result in duplicate imports being
generated
Previously useMemo inlining created a new StoreLocal assignment (not
reassignment!) instruction for every return value. This breaks when the return
is inside a block (like an if-block) as the scope is tied to the block.
For example: ``` let x = useMemo(() => { if (...) { return { ... }; } })
``` would become: ``` if (...) { const temp = { ... }; } const x = temp; ```
This PR instead changes the inlining to declare a temporary in the function
prologue and then reassign values to it when replacing return statements.
``` let x = useMemo(() => { if (...) { return { ... }; } }) ```
becomes
``` let temp; if (...) { temp = { ... }; } const x = temp; ```
This is a simplified version of #1454. The goal of this PR is to inline the
contents of `useMemo()` callbacks, rather than just immediately invoke the
lambda. Turning useMemo() into an IIFE works, but it means that we can't
optimize within the lambda block. Our investigations showed that there's a lot
of room to optimize at a finer granularity than manually written useMemo calls.
For example, one product instance had a useMemo that created a list of child JSX
elements. Most of those elements only relied on a single variable (`a`), but a
few relied on a second variable (`b). Thus _all_ elements were invalidated
whenever `b` changed. If Forget retains the original lambda, we have no choice
but to keep that (coarse) granularity for memoization. When we inline, we can
optimize to make e.g. individual JSX elements depend on their precise
dependencies.
The rough idea is:
* Keep track of all function expressions
* When we find a useMemo, lookup its function expression, and add its CFG to the
main function (the previous PR ensures that BlockIds won't collide)
* Replace any return statements with a StoreLocal to save the result and a Goto
to the code following the useMemo call.
* Then we run the usual set of passes to patch the HIR back up again.
Example:
```javascript
// Before
function Component(props) {
const x = useMemo(() => {
if (props.cond) {
return null;
}
return foo(props.x);
}, [props.x]);
return x + props.y;
}
// Intended - **before** memoization
function Component(props) {
let x;
if (props.cond) {
x = null;
} else {
x = foo(props.x);
}
return x + props.y;
}
```
This PR ensures (via a static assertion function) that all terminal variants
have a SourceLocation, and adds locations to the variants which didn't have it
before. This also adds a static assertion that terminals have an InstructionId,
though we already relied on that so it was checked via usage.
This is a prerequisite to inlining `useMemo()` lambdas so that we can better
optimize them. Nested functions are evaluated with a fresh HIRBuilder, which
means that they currently have their own `bindings` object for mapping
identifier instances to IdentifierIds. This means that identifier ids in a
closure are _always_ different that those outside the closure, even when they
refer to the same identifier:
```
function Component(props) {
props; // becomes e.g. props$1
const onClick = () => {
props // becomes e.g. props$2
};
}
```
For useMemo inlining this is problematic because we've lost the association that
these identifiers actually refer to the same thing. This PR changes that,
sharing the name resolution data structure between the top-level function and
any nested function expressions.
To unblock internal experimentation, for now let's just skip over compiling any
file that contains one or more disables of React's eslint rules, and log that.
This is a little coarse in the sense that we could skip over just functions that
contain the comments, but Babel doesn't provide an easy way to traverse comments
afaict so this is the simplest solution. I did check our internal repo and noted
that there was only one disable of exhaustive-hooks in that entire directory in
one file, so this should be fine.
Notably we are not throwing any errors if we detect these violations as we don't
want to fail the build, we just want to skip them for now.
While running the latest Forget build on www I noticed that a lot of the
bailouts were special-cases where we used interpolation in the error `reason`
string to provide more context for debugging. This is a pretty cool result,
because it means that we actually support nearly all the common syntax (at least
based on a sample of the codebase). But it makes our tools for aggregating
errors break down a bit.
This PR adds a new, nullable `description` property to CompilerErrorDetail, and
manually updates to ensure that we always pass a static `reason` and only use
interpolation in the `description`. This will allow our aggregation tools to
group by the reason.
NOTE: See background in #1476.
Updates BuildHIR to use the new LabelTerminal for LabeledStatements, and adds
support for HIR->ReactiveFunction transformation and codegen. Note that we
sometimes produce an extraneous block wrapper if it turns out the label wasn't
necessary, that seems...fine?
Adds a new `LabelTerminal` which will be used to represent LabeledStatements
that contain a statement other than a loop. What we do for these cases is
basically break the containing block in two, with a goto after the inner
statement to the fallthrough. This allows us to model the label, and any `break`
to it, in the HIR. However this fails in codegen because we can't find the
fallthrough branch — we need a high level terminal that knows about this
structure.
Hence LabelTerminal. Now, instead of just a continuation block and a goto, we
have a structured terminal. The LabelTerminal expresses the block for the
labeled statement and the continuation, and we can use this to put it back
together when constructing a ReactiveFunction. Note that this PR is just the
scaffolding for LabelTerminal, the next PR is the interesting bits.
Adds a script to automate adding/updating the copyright header to all
appropriate files. For now i've excluded fixture inputs, just because it would
impact fixture outputs too, but we can add them in a later PR if we want.
This PR ensures that we use a single id space for the `BlockId`s in both
top-level functions as well as any nested FunctionExpressions (note, we already
do this for `IdentifierId`). This will make it easier for follow-ups to merge
the CFG of nested functions (ie useMemo bodies) with the parent without block id
collisions.
Fixes `<fbt>`. This required a bunk of yak shaving to work through several
issues:
* First, there was a bug in codegen for JsxNamedspacedName. I added handling for
it for identifiers, but JsxNamespacedName gets converted to a Primitive. The
output looked correct because Babel happily creates invalid Jsx identifiers!
* Next, I needed to add locations to JSX nodes. It took me a while to pinpoint
which specific node needed the location, so I ended up just adding locations to
all the parts of a Jsx element.
* That uncovered the fact that FBT was expecting the `<fbt:param>`'s `name`
attribute value to be a StringLiteral, not a StringLiteral wrapped in a
JsxExpressionContainer. So now we special-case JsxAttribute and emit raw
StringLiteral (either is allowed per the spec)
And with that, voila, `<fbt>` works.
Adds the two FBT (https://facebook.github.io/fbt/) plugins to our test setup so
that we can verify Forget plays well with FBT. Unfortunately FBT's plugins are a
bit finicky, and things that are technically allowed per the JSX spec (such as
wrapping string attribute values in a JsxExpressionContainer) aren't supported
by FBT's plugin. This PR is just to add the fbt plugins and highlight some cases
that fail; these are fixed in later PRs in the stack.
For example, the `fbt-params.js` fixture fails on this PR:
Input
```error.fbt-params.js
import fbt from "fbt";
function Component(props) {
return (
<fbt desc={"Dialog to show to user"}>
Hello <fbt:param name="user name">{props.name}</fbt:param>
</fbt>
);
}
```
Output
```
React Forget › __tests__/fixtures/compiler › fixtures › fbt-params
Expected fixture 'fbt-params' to succeed but it failed with error:
/Users/joesavona/github/react-forget/forget/fbt-params: fbt: unsupported babel
node: MemberExpression
---
props.name
---
```
See fixes later in the stack.
Adds support for `for` statements with an empty or unreachable update
expression. In both cases, reversePostorderBlocks() will remove the
empty/unreachable update block, leaving the ForTerminal.update pointing to a
non-existent block. We explicitly rewrite this (much like we null out
unreachable fallthroughs after shrink). When transforming to ReactiveFunction,
we emit the update block as null if it was the same as the test block.
Fix for the previous issue, suggested by @gsathya: when we run
InferReferenceEffects on the outer function we check each closure to see if it
actually captured any mutable values. If it didn't, we can mark the closure as
readonly and memoize it independently.
Repro of a closure that we currently treat as readonly because it captures a
possibly-mutable value, but which we later realize is not mutable. Specifically,
when we check `exit()` we think `dispatch()` is mutable and therefore consider
it captured, which means we can't independently memoize `exit`.
Updates `PruneNonEscapingScopes` to consider hook arguments as potentially
escaping. This is because hook inputs are "owned" by React — for example,
closures passed to `useEffect`, or a value that is passed to a custom hook and
which then becomes a memoized input.
Similar to what we did for `<fbt>` jsx elements, this PR ensures that `fbt()`
calls have their operands memoized in the same scope to honor the limited
contract for what's allowed as an argument of an fbt() call expression.
There are some internal restrictions in Metro that only allow us to specify one
gating module as an injected dependency. To allow multiple projects, this PR
updates the Babel plugin to take a gating options config specifiying a project
name. The project name is used as a suffix for the generated import; for
example:
```js
const options = {
// ...
gating: {
module: "ReactForgetFeatureFlag",
importSpecifierName: "isForgetEnabled_Secret",
};
// generates
import {isForgetEnabled_Secret} from "ReactForgetFeatureFlag"; // a module that
exports multiple flags
// ...
```