This PR updates the conditions for which variables are promoted to context
variables.
The previous rule was to promote any variable where the variable was reassigned
in some function expression other than the function that declared the variable.
Notably, this meant that we did not use context variables for variables which
were captured in a function expression, but reassigned _outside_ a function
expression.
The new rule is more consistent: we promote any variable which is a) reassigned
somewhere and b) referenced in some function expression outside of their
declaring function. The implementation builds two sets of identifiers, one for
each criteria, then takes the union of these two sets.
## Motivation
The motivation for this change is to unblock additional validations and
optimizations of function expressions. It's currently difficult to translate
metadata that we infer about identifiers outside of a function expression into
metadata about the identifiers within a function expression — for example to
infer types within function expression bodies based on type information outside,
propagate constants into functions, infer reference effects, etc.
After this change, the only free variables inside function expressions will be
variables that are effectively `const` - never reassigned anywhere. Thus it will
be safe to renumber those identifiers to match the outer context (during
EnterSSA), making it trivial to map metadata from outside the function into the
function.
This change also more closely models the runtime representation — any variable
referenced in a function, and reassigned somewhere, would have to be compiled
(ie in a JS engine) to use a context variable.
The goal of this PR is to improve ValidateNoRefAccessInRender to find function
expressions which a) access refs and b) may be called during render. Currently
we always allow ref access in any function expression, but that's obviously
optimistic.
For the approach, the observation is that we already have a system that tells us
whether a function may get called — mutable range inference. So long as we
consider a function "mutable", we'll infer a range for it, but the problem is
that we don't currently view functions which depend on refs to be mutable. So
here I'm doing ~~sort of~~ a hack to force function deps on refs to be treated
as Effect.Capture. This is enough for the function to be considered mutable, for
a mutable range to be assigned, and for us to detect that during ref validation.
I don't love the hack, i'm open to other ideas!
---
- [patch] Find context variables within FunctionDeclarations (previously
missing)
- [todo] Need to fix non-allocating values / variables being DCE'd
One approach is to label everything referenced by a lambda as context variables.
I couldn't come up with other examples that break without this change, so I
wonder if something lighter / hackier works just as well. (My only hesitation is
that we may end up losing out on potential optimizations for everything aliased
to these variables).
---
(wip, waiting for feedback on workplace post)
- remove calls to `isInROMode`, as we want to log all mutations after 'freezing'
a value (both within and outside of render cycle)
- add `source` parameter -- this is the function name of the parent component /
hook
- Gating checks for debugging / profiling can be moved to within the
instrumentation or makeReadOnly functions. This simplifies codegen and reduces
code bloat.
- Warn if importing conflicting identifiers
- Aggregate imports from the same source
---
Emit calls to makeReadOnly for memoized values.
```js
function MyComponent() {
let x;
if (c_0) {
x = // ... (recompute x)
$[0] = __DEV__ ? makeReadOnly(x, "MyComponent") : x;
} else {
x = $[0]
}
}
```
- import source / specifier should be configurable, as
- we'll likely want to add gk gating to `makeReadOnly` itself to reduce codesize
bloat
- each Forget project needs different logging and filter configurations
- codegen function name as an argument for easier debugging
- only freeze memoized outputs
This is an attempt to scaffold yarn workspaces into our repo with as minimal as
possible changes to our current setup and directory structure. Essentially this
PR moves current `src` into `packages/babel-plugin-react-forget` as a first step
to keep everything working. Later on when we're ready we can split out a
`react-compiler` package that is decoupled from Babel, but it's too early for
that now
Not to get ahead of myself (sorry i had to), but i think this is the last order
of evaluation bug. At least it's the last one we know of[1]. Per the previous
PR, the issue is that constant propagation can copy the last value of a sequence
expression to where the sequence is used, leaving the original sequence
expression out of order after other instructions are moved around. We fix that
here by explicitly skipping constant propagation for the last value of a
sequence block.
[1] There are some places where we _would_ have evaluation order bugs if we
allowed arbitrary expressions, but we explicitly limit the expressions we allow
in those places. For the curious: switch test case values and destructuring
default values.
Changes the lowering for sequence expressions to use the new terminal. When
converting to a ReactiveFunction, we convert these terminals into
ReactiveSequenceValues, which nests the instructions and preserves order of
evaluation in the output.
The only catch is constant propagation — constant propagation breaks
order-of-evaluation because it can effectively copy the final value of a
sequence elsewhere, leaving the original sequence in the wrong place. I'll
address that in a follow-up.
I realized that we can use our value block system to fix _most_ of the remaining
order-of-evaluation issues we had with sequence expressions. This PR adds a new
SequenceTerminal to HIR; there is already a ReactiveFunction equivalent
(ReactiveSequenceValue) that the next PR will convert this terminal into.
Handles three more cases:
* Template literals
* deletion (property/computed)
* type casts
Only the latter has an observable impact, though i added tests for deletion just
in case and found a bug. For type casts, they're reasonably common internally
for fixmes, so this PR will help to ensure we don't drop type information just
because of a cast.
Renames some error fixtures for clarity, "error.invalid-*" are fixtures that are
expected to fail for invalid input, where other "error.*" fixtures are basically
todos. While i was here i clarified the error messages for invalid useMemo
callbacks, and changes the error severity from Invariant to InvalidInput.
Fixes one more category of bug. For assignment expressions, we validating
against redeclaring a global variable when the assignment target was an
identifier, but not when the global was reassigned via destructuring. This PR
adds a `lowerIdentifierForAssignment()` helper and uses it for assignment of all
identifier variants, including destructuring.
I reviewed the test cases we have marked as bugs ("_bug.*") and realized that
several of them are already fixed — woohoo! Then one wasn't fixed _yet_: our
type inference loses track of refs if you stash them inside an object/array. But
that's why I added the ValidateNoRefAccessInRender pass, which i've updated to
detect and reject these invalid cases. There are now only a few bugs left (more
fixes coming).
These are still quite Babel specific, but the interface is slightly more
generic: CompilerEntrypoint takes a non-Babel specific CompilerPass as an
argument instead of passing Babel's PluginPass directly.
In the future we can consider lowering the whole Program into HIR but that
involves a significant lift in our representation and a small amount of new
syntax to support (eg import statements), so this PR is the extent of this stack
for now
Updates PruneNonReactiveDependencies to treat setState functions as
non-reactive, since we know they have a stable identity. This is based on type
inference and our recently added definitions for useState and its return type,
so it's conservative and will only work when our inference can prove that the
scope dependency has the SetState type.
Note that this approach is simple and has limitations, notably the fact that the
setState is non-reactive doesn't propagate. But it's simple, trivially correct,
and already improves codegen somewhat, so i figured it's worth landing for now.
## Test Plan
Tested on internal app
Adds validation to reject freezing mutable lambdas, since lambdas cannot _be_
frozen, they're either frozen or not. Example invalid code:
```javascript
function Component(props) {
const x = {value: ""};
const onChange = (e) => {
// MUTATION!!!!!
x.value = e.target.value;
setX(x);
};
return <input value={x.value} onChange={onChange} />;
}
```
Note that there is a separate issue in which we are not detecting lambdas that
would definitely modify immutable values. We may need to distinguish
ConditionallyCapture (captures if the value is mutable, otherwise readonly) from
Capture (definitely mutates) in order to make that case work. But already this
validation helps prevent some invalid code.
Adds a `removeAllMemoization` flag that runs the entire compiler pipeline but
strips out all memoization. The intent is to be able to compare (in limited
use-cases) the performance of an existing app with all memoization removed, vs
the performance with manual memoization, vs the performance with Forget enabled.
In terms of how this works: we already strip out useMemo/useCallback since
Forget is more accurate. The new option adds an extra pass that strips out all
reactive scopes. Collectively this leaves ~zero memoization within components
(this does leave React.memo, but close enough).
---
Remove jest fixture tests in favor of snap runner. Main reasons:
- maintaining feature flags and compatible behavior required syncing all changes
to 3 files (`generateTestsFromFixtures`, `compiler-test`, and `compiler-worker`)
- jest snapshot test file causes rebase conflicts on most rebases
- speed 🙌
$ time yarn test compiler-test
(the extra test here is `has a consistent extension for input fixtures`)
```
Test Suites: 1 passed, 1 total
Tests: 37 skipped, 480 passed, 517 total
Snapshots: 479 passed, 479 total
Time: 27.668 s
Ran all test suites matching /compiler-test/i.
✨ Done in 43.18s.
yarn test compiler-test 57.05s user 3.85s system 139% cpu 43.546 total
```
$ time yarn snap
```
478 Tests, 478 Passed, 0 Failed
✨ Done in 13.12s.
yarn snap 53.96s user 9.35s system 468% cpu 13.518 total
```
Jest and snap should have the same set of features:
- report test failures via exit status (used by Git Actions)
- watch mode
- breakpoints + `debugger` statements
- note that `--sync` is not required for this
- skip `todo.` prefixed fixtures
- fixtures in nested directories e.g. `rules-of-hooks/testname.js`
- filter mode (via editing `testfilter.txt`)
- filter + debug mode
(1) edit `testfilter.txt` to filter out all but one test
(2) add `@debug` pragma to the first line of the test
testfilter.txt
```js
// @only
testfixture_basename1
testfixture_basename2
```
Turns out I just forgot to forward `process.env` when forking 😅
`debugger` statements and breakpoints should work with both `--sync` and
`--no-sync` (default) modes
This is the example we discussed in our design sync.
```javascript
function Component(props) {
const [x, setX] = useState({ value: "" });
const onChange = (e) => {
// INVALID! should use copy-on-write and pass the new value
x.value = e.target.value;
setX(x);
};
return <input value={x.value} onChange={onChange} />;
}
```
Here `onChange` is a mutable lambda, and it should be invalid to pass a mutable
lambda where a frozen value is expected. This is because unlike other value
types, you cannot freeze a lambda — the only choice is to not call it at all.
Note that there is a harder case to catch:
```js
function Component(props) {
const [x, setX] = useState({ value: "" });
const onChange = (e) => {
// INVALID! should use copy-on-write and pass the new value
x.value = e.target.value;
setX(x);
};
const x = constructAValueThatMaybeAliasesItsInput(onChange);
return <input value={x.value} onChange={x.maybeGetTheLambdaBack()} />;
}
```
This case demonstrates how mutable lambdas can be captured and then accessed
later — the analysis to catch this case is more sophisticated bc it involves
inferring that `x` aliases a mutable lambda. But we also can't be sure that `x`
does alias the lambda, so disallowing this code could prevent a lot of valid
code from compiling. My hypothesis is that we should start with at least
validating the example at the top, while allowing the second case for now.
We can now type `Array.prototype.{map,filter}`:
* The callee is ConditionallyMutable because, although the array itself is not
modified, its items flow into the lambda and may be modified there.
* The argument is ConditionallyMutable because it accepts both mutable and
immutable lambdas. Mutate would disallow immutable lambdas (wrong), while Read
would be incorrect for mutable lambdas since calling them triggers mutation.
Adds test cases to ensure we're correctly inferring mutative builtin operations
— property store, computed property store, property deletion, and computed
property deletion — as definite mutation and that we're rejecting inputs where
these operations are used on immutable/frozen values.
Adds back `Effect.Mutate`, and changes so that `Effect.ConditionallyMutate`
never rejects frozen/immutable values, while `Effect.Mutate` _always_ rejects
frozen/immutable values.
We currently use `Effect.Mutate` both for places that _may_ mutate (ie untyped
function calls) and for places that have known mutation (typed function calls,
or operations like `delete x.y`). We then use a separate mechanism to decide
whether to reject the input, with some call paths checking the effect and others
not.
This stack refactors this logic in InferReferenceEffects per our discussion, so
that `Effect.ConditionallyMutate` is for "may or may not mutate" either because
we're not 100% sure (untyped function) or because the mutation depends on the
operand (ie, a callback arg that will be invoked and thus will mutate if the
lambda is mutable, not mutate if the lambda is immutable). Later diffs add back
`Effect.Mutate` as "definitely 100% mutating".
Joe Savona
Lauren Tan
Mofei Zhang
Sathya Gunasekaran