Cleans up the public exports for the package itself:
* `parseFunctions()` was only used in playground, so this moves the definition
there. I had to update playground's dependencies to ensure the babel version
matched.
* Flattens away the `HIR` const in the export, and exports just 4 functions all
at the top-level: `run()`, `compile()`, `printHIR()`, and
`printReactiveFunction()`.
This will make it very easy to split the core compiler into a separate package
from the babel plugin, though i'm not sure it's worth doing that (yet).
Changes playground to use the modified `run()` function of the compiler, polling
the generator and building up a Map of tabs automatically based on the passes
that the compiler runs. This means tabs are always derived from the current
state of the compiler and we can never forget to add a pass.
<img width="1497" alt="Screen Shot 2023-01-10 at 11 06 03 AM"
src="https://user-images.githubusercontent.com/6425824/211639442-da421f73-e19e-4b63-9f33-0ce5a68cceb7.png">
Note the inclusion of some recently added passes that weren't added to the
playground — which was my fault but only bc i intended to ship this PR soon :-)
Thanks old architecture! We learned a lot about what does and doesn't work via
this POC, but it's time to move forward w the ~~new~~ current architecture.
It's pretty tedious to keep the playground in sync w `Pipeline` — we need some
abstraction so we can write the sequence of passes once and reuse it (while
inspecting intermediate states).
This is mostly unused and there's just not enough benefit for now. We can re-add
this if folks start using this site on mobile. Removing now to simplify the
codebase.
There are a bunch of ways we can go about converting from the input HIR into a
final form that has the preamble inserted and memoized blocks of code wrapped
with change detection and caching. This is just one way, it might not be the
ideal way. In any case, this pass converts HIRFunction -> ReactiveFunction. The
latter is a recursive (tree-shaped) data structure that attempts to represent
blocks each of composed of scopes or instructions, where scopes are themselves
composed of blocks etc. The idea is a) this makes it easy to visualize the
structure and check that the scopes and their dependencies are correct and b)
this is a really nice form for adding the memoization code. We can convert from
a ReactiveFunction back to an HIRFunction, wrapping each scope in the
appropriate if checks and caching.
## Example
Consider the following example, which has 2 main scopes: an outer one for `x`
and an inner one in the consequent for `y`:
```javascript
function foo(a, b, c) {
const x = [];
if (a) {
const y = [];
y.push(b);
x.push(<div>{y}</div>);
} else {
x.push(c);
}
return x;
}
```
## Output
The new builder constructs a ReactiveFunction for this example along the lines
of the following (note that inputs are always empty bc we don't collect those
yet):
```
{
scope @0 [1:11] inputs=[] {
[1] Const mutate x$11_@0[1:11] = Array []
[2] if (read a$8) {
scope @1 [3:5] inputs=[] {
[3] Const mutate y$12_@1[3:5] = Array []
[4] Call mutate y$12_@1.push(read b$9)
}
scope @2 [5:6] inputs=[] {
[5] Const mutate $13_@2 = "div"
}
scope @3 [6:7] inputs=[] {
[6] Const mutate $14_@3 = JSX <read $13_@2>{freeze y$12_@1}</read $13_@2>
}
[7] Call mutate x$11_@0.push(read $14_@3)
} else {
[9] Call mutate x$11_@0.push(read c$10)
}
}
[10] return x$11;
}
```
This shows the hierarchy: there's an outer scope, `@0` to compute `x` (the first
scope), then within the if consequent there's another scope, `@1`, to compute
`y`. We have some technically extraneous scopes to compute the JSX element; that
can be cleaned up with a bit more refinement.
With this structure — and the inputs and outputs of each scope filled in — we
can convert to code in a straightforward manner. Each scope turns into a block
along the lines of the following:
(note here we use strings to index the cache, in reality these would be ints)
```javascript
// one change variable pet input:
let c_a = a !== $['a'];
...
// one variable for each output:
let x;
...
// if (changed) { recompute } else { use-cache }
if (c_a || ... ) {
x = ...;
// one assignment per output
$['x'] = x;
// update cache per input
$['a'] = a;
...
} else {
// one assignment per output
x = $['x'];
...
}
```
Reorders LeaveSSA so that it runs before we begin evaluating reactive scopes.
Note that reactive scopes must span the full construction of each variable — for
variables with a phi, this must span the declaration and all assignments of the
phi operands. And that's exactly what the new LeaveSSA does! LeaveSSA removes
phi nodes and ensure that all versions of a variable which flow into a phi have
been assigned a single canonical identifier (with an appropriate mutable range).
This PR includes this and some related changes:
* Reorders the pass
* Changes hir-test to print the final HIR, eg just prior to codegen
* Teaches LeaveSSA to update the mutable range of the canonical identifiers it
assigns, based on the min/max of the variables assigned.
The `automaticLayout` config option for Monaco causes it to remeasure itself,
and the parent container's height being longer than the screen causes it to
constantly grow infinitely. You can observe this bug by going to the playground,
expanding any tab, and watch the scrollbar grow as the editor quickly grows to
ridiculous heights and your laptop starts glowing red hot and̶͙̕ H̴͉͘e comes
t̵͙́o ̴̜̿de̷̼̚s̷̻̍ec̶̮͒rate all knowled̵̥̆ge
We might need to revisit the tabs vs. options split, but for now this just adds
a checkbox toggle that outputs codegenned JS instead of HIR in the HIR tab. Open
to ideas to organize this in the future...
Instead of hacking into the babel plugin passes, this now takes a new approach
for the HIR tab:
- The different passes are exported from the babel plugin (for simplicity)
- The tab actually runs the compiler steps based on local config in the tab.
- Re-purposed the CompilerFlags component to configure what passes to run. This
is currently mostly causing different errors, but could be useful going forward
as a direction.
This diff adds styling to the compiler options editor. The floating input/output
toggle button on small screens now spans the bottom of the screen, so that it
doesn't block the compiler options.
Height overflows when adjusting screen size are complicated by Monaco Editor and
will be addressed in a later diff.
Test plan:
Start Playground and see the latest look of the compiler options editor beneath
the output section.
## Proper Detection of Out-of-order Functions
The no-use-before-define rule from ESLint has a strange behavior in which it
treats variables differently than functions:
```javascript
function foo() {
return bar(X);
}
const X = null;
function bar(x) {}
```
By default, `bar(x)` has two errors: one because X is used before defined, and
once because `bar` is used before defined. The rule has an option `{variables:
false}` which only enables validation when the variable is from the same "scope"
as the reference, the net result of which is it means it doesn't report spurious
errors such as X being undefined. There is _also_ a `{functions: false}` option,
but for some reason that doesn't work the same way, it just turns off all
validation of references that came from functions. So enabling that option would
suppress the (spurious) error on invoking `bar()` above, but causes the rule to
miss invalid code such as:
```
function foo() {
return bar();
function bar() {}
}
```
This PR adds a fork of the rule that makes `{functions: false}` behave similarly
to `{variables: false}`, which should help avoid some of the spurious errors i
saw internally. The rule is exported from Forget itself, which will make it
easier to consume internally, in tests, and in the playground.
## Targeting the validation to Forget functions
Even with the above, there are still some false positives coming from code such
as:
```javascript
const x = foo();
function foo() {}
```
This PR changes codegen to ensure that the output of a function _always_ has the
body starting with 'use forget'. The ESLint rule then only looks at function
declarations/expressions whose body starts with that expression. The new unit
test confirms that the validation finds invalid reorderings even on functions
that weren't explicitly tagged as 'use forget'.
ESLint's default parser doesn't support any non-standard syntax, which includes
JSX. So when I added the ESLint validation step to the playground, it meant that
valid examples containing JSX still reported "invalid output". I tried to use an
alternative parser, but I couldn't figure out the right webpack incantations to
make `@babel/eslint-parser` or `hermes-eslint` work. I even tried recreating
some of their code to avoid problematic imports, no dice.
Instead this PR:
* No longer uses the `postCodegenValidator` step, and runs the validation on the
output after compilation completes. This is better anyway since we can see the
output *and* the error messages
* Shows rule violations as an "Invalid output" comment
* Shows parser errors as a note (mostly to indicate that the validation step
couldn't run, there could still be no-use-before-define violations that weren't
found)
Invalid example:
<img width="1502" alt="Screen Shot 2022-10-21 at 9 50 23 AM"
src="https://user-images.githubusercontent.com/6425824/197249007-1ec244a0-6dfe-4ec6-a0d0-60302efd86bd.png">
Sample example but with some JSX:
<img width="1500" alt="Screen Shot 2022-10-21 at 9 50 39 AM"
src="https://user-images.githubusercontent.com/6425824/197249030-e68ba968-4101-47c7-a148-f548f84f375c.png">
#686 added an option to validate generated code after transformation and adds an
ESLint-based validator function to transform-test. Unfortunately it isn't super
easy to wire up ESLint for use in a browser: traditionally the ESLint project
specifically did _not_ support browser builds, but they recently have relaxed
this because they added a browser playground on their website. There isn't
official support, but the [playground
repo](https://github.com/eslint/playground/blob/f3b1f78cc1c06dadfe7bb50c6c0f913c0d23670d/webpack.config.js)
has a webpack config that, when combined with requiring a specific file, allows
making things work in a browser.
I tried using this directly in our playground app but Next's default webpack
config doesn't work. So I created a separate package, playground-validator,
which exports a webpack-built version of `eslint.Linter`. Then the playground
can consume that, and everything works:
## Test Plan 👀
Confirmed that a known problematic example displays the validation message in
playground (both locally and on the preview deployment):
<img width="1500" alt="Screen Shot 2022-10-20 at 12 22 59 PM"
src="https://user-images.githubusercontent.com/6425824/197041265-966ffda2-a3d0-450e-8fc4-fd1a7ca06e1a.png">
Some prior [microbenchmarking](https://jsbench.me/7ol98ws520/1) showed that a
for loop outperformed `fill` (which is about ~60% slower). This is the same
approach we use in the latest useMemoCache PR
This is a new module that holds:
- the `useMemoCache` stub (hopefully to be deleted next week)
- various helpers that can be imported by the compiler, e.g. the dispatcher
guard `$startLazy`
- skipped the implementation of `makeReadOnly` for now as there's already
multiple copies and I wanted to avoid typescript in this file for now to make
the build easier (i.e. no build)
I didn't make this an option as it's unclear we'll really need this. We can
always add an option later, I think.
This is the name that's available on facebook.com at the moment.