Commit Graph
1003 Commits
Author SHA1 Message Date
Joe Savona 0c1193ecd7 [rust] estree codegen for improved serialization
This is meant to replace the initial `estree` crate with a version that is 
generated from a JSON description of ESTree. The idea is to make it easy to 
experiment with slightly different representations to balance ergonomic usage of 
the data at runtime with serialization compatible with ESTree spec. The JSON 
schema looks like this (somewhat abbreviated): 

``` 

{ 

// Objects are struct types that don't have a `type` and can't appear as an enum 
variant 

objects: { 

Position: { 

line: {type: "NonZeroU32"}, 

column: {type: "u32"} 

}, 

... 

}, 

// Nodes are struct types with a `type` and which can appear as enum variants 
(statements, expressions, patterns, etc) 

nodes: { 

ArrayExpression: { 

elements: { 

type: "Expression", 

plural: true, 

nullable_item: true, 

} 

} 

... 

}, 

// Categories of nodes with multiple variants, represented as enums. 

// Can be recursive, eg ForInit can be VariableDeclaration or Expression, 

// where Expression is also an enum 

enums: { 

Expression: [ 

"ArrayExpression", 

... 

] 

}, 

// Simple enums which have a corresponding string value. Used primarily for 
operators (binary/unary/logical/etc) 

// but also for things like variable declaration kind (var/const/let) 

operators: { 

BinaryOperator: { 

Plus: "+", 

Instanceof: "instanceof", 

} 

} 

} 

``` 

The core estree files are now generated using Cargo's build script mechanism. 
Right now i only defined the types and fields from ES5, so i'll have to flush 
out the rest of the modern JS spec and extensions like JSX, TypeScript, and 
Flow. But already the for-statement example works, showing that this approach 
can handle complex cases such as unions of types or other unions (ForStatement 
initializer is tricky bc it can be a VariableDeclaration or an Expression - that 
works now!). 

This is still WIP a bit - now that the ESTree definition is more precise i can 
go back and clean up some other code (have to, because the swc -> estree 
conversion needs some tweaks now).
2023-07-08 22:59:56 +09:00
Joe Savona a32a2baa73 [rust] Start of error handling
Until now i've freely used `panic!`, `unwrap()`, and friends for "error 
handling". This PR switches to consistently returning `Result` within the HIR 
builder, using a structured error representation that exploits helpers from 
`thiserror` and `miette` crates. Miette has a super graphical formatter for 
diagnostics as you can see in the screenshot (also see the 
[repo](https://docs.rs/miette/5.9.0/miette/index.html)). 

This is just a first pass and we'll need to flush out the error handing story 
more. Two obvious directions to go next: 

* Make HIR construction error-tolerant, so that it can find as many errors as 
possible at once rather than failing on the first error. We did this in Relay 
Compiler as well, and we can likely borrow some of its helpers. 

* Decouple from `miette`. It's very nice but less flexible than I'd like. We can 
define our own more generic diagnostic type that contains structured data, then 
have a generic conversion mechanism into a miette type so we can use their 
display logic. 

<img width="789" alt="Screenshot 2023-07-06 at 3 55 38 PM" 
src="https://github.com/facebook/react-forget/assets/6425824/e1f1ed4b-5188-4af5-9af4-8f6c5c345023">
2023-07-08 22:59:55 +09:00
Joe Savona 8176f25719 [rust] ForStatement support
Implements support for `ForStatement` from swc -> estree -> hir, flushing out 
more of the HIR representation and porting pieces from HIRBuilder as necessary.
2023-07-08 22:59:54 +09:00
Lauren Tan 58a8d0bdc7 [ez] Update gitignore for Rust 2023-07-06 16:25:18 -04:00
Sathya Gunasekaran 853d0b1b69 [fuzzer] Make fuzzer work with Forget
Remove v8 specific bits. It's still not very useful as we barely have any tests 
that can be used as input corpus.
2023-07-06 11:44:14 +05:30
Sathya Gunasekaran 31ec959e9b [fuzzer] Initial import of v8's fuzzer
Copied from  
https://chromium.googlesource.com/v8/v8/+/master/tools/clusterfuzz/js_fuzzer/
2023-07-06 11:44:14 +05:30
Joe Savona 1bf00e6b6e [rust] lint 2023-07-06 09:24:53 +09:00
Joe Savona 357e9de19b [rust] IfStatement support
Lowering for IfStatements, which includes porting some of the HIRBuilder helpers 
like enter() and reserve().
2023-07-06 09:24:52 +09:00
Joe Savona c59ac387ae [rust] Assignment expressions 2023-07-06 09:24:51 +09:00
Joe Savona d99da01698 [rust] basic VariableDeclaration support 2023-07-06 09:24:50 +09:00
Joe Savona 2eb7c897f3 [rust][ez] cleanup 2023-07-06 09:24:49 +09:00
Joe Savona f6957e9a32 [rust] Partial handling of identifiers
Fundamentally this PR is about lowering identifiers during construction of HIR. 
For now i'm punting on context variables and assuming all variables are either 
global, module-scoped, or locals. The implementation involves a few pieces: 

* `estree::Identifer` gets extended with optional binding information. The idea 
is that _some_ name resolution mechanism will populate this. Eventually our own, 
but we can also borrow data from another source... 

* `estree-swc` now configures SWC's (possibly broken?) name resolution mechanism 
and sets the above binding data when translating identifiers from swc into our 
estree format. 

* hir `Builder` tracks identifiers based on `(name, BindingId)` pairs, and 
assigns a unique `hir::Identifier` instance for each pair. `Identifiers` are 
clone-able (shared). 

* Tangential: i updated the printer to handle more instruction variants, 
including the now-ported LoadGlobal instr. 

I don't love this but it's a start. Long-term we definitely should have our own 
name resolution mechanism which we run on the estree prior to lowering to HIR.
2023-07-06 09:24:48 +09:00
Joe Savona 08e51d3fd7 [rust] Pretty-printer for HIR
Implements a pretty-printer for the HIR and switches the fixture tests to use 
this instead of the debug format. It's much more readable now! 

Note that not all types are properly printed, I only implemented the 
instructions and terminals used in the example. For others we fall back to the 
Debug impl so we at least print something.
2023-07-06 09:24:47 +09:00
Joe Savona d3b9948b5e [rust] Fixture tests for parsing and lowering
Adds a new `fixtures` crate intended for running end-to-end tests of the 
compiler. As we expand the compiler this will eventually match our JS fixture 
setup, where we have .js files as input and produce memoized JS output. 

For now, this does the following: 

* Parses with SWC (omg this was painful to setup) 

* Runs SWC's name resolution, which annotates the SWC ast in-place 

* Convert the SWC ast into our `estree` representation 

* Convert `estree` into `hir` for each top-level function declaration in the 
input program 

* Print the Rust `Debug` view of the resulting HIR 

As a next step i'll add a pretty-printer for the HIR to roughly match what we 
have in JS.
2023-07-06 09:24:46 +09:00
Joe Savona 1d2e7ee747 [rust] Use Rc<RefCell<>> for shared identifiers
Multiple Place instances can share a reference to a given Identifier in our JS 
implementation. For simplicity of the initial port I’m using Rc (for sharing) 
and RefCell (for runtime-checked mutability). This is the standard pattern for 
shared mutable references in Rust when you don’t need multi-threaded support. We 
don’t need HIR to be accessible by multiple threads so this is fine, if we do 
multiple threads it will be to parallelize compilation of separate functions. 

There are other idioms w less runtime overhead, such as Place holding an index 
into a separate vec of identifiers, but that would make the port much less 
straightforward.
2023-07-06 09:24:45 +09:00
Joe Savona b1c8cb358b [rust] Start of lowering (BuildHIR)
Starts to port BuildHIR, in the Rust case this means the ESTree -> HIR 
conversion. This necessitated flushing out the Builder struct a bit more. Mostly 
the logic translates over very directly, and if anything it's cleaner because of 
the lack of noise dealing with TypeScript unsoundness for Babel typedefs and 
switch statements.
2023-07-06 09:24:44 +09:00
Joe Savona 2ac69bf7d9 [rust] Passes from HIRBuilder
This is a start to porting HIRBuilder, with a largely complete implementation of 
`build()`. Notably this includes all the passes which build() calls, and the 
helper functions those passes call in turn: 

```rust 

reverse_postorder_blocks(&mut hir); 

remove_unreachable_for_updates(&mut hir); 

remove_unreachable_fallthroughs(&mut hir); 

remove_unreachable_do_while_statements(&mut hir); 

mark_instruction_ids(&mut hir)?; 

mark_predecessors(&mut hir); 

``` 

This was pretty straightforward. I ran into one borrow checker issue with (iirc) 
`remove_unreachable_for_updates` where i needed to simultaneously hold a mutable 
reference into the HIR (to write the ForTerminal) and also get an immutable 
reference to check if the update block is reachable. The challenge is this: 

```rust 

for block in hir.blocks.values_mut() { 

^^^^^^^^^^^^^^^^^^^^^^^ hir borrowed mutably here 

if let TerminalValue::ForTerminal(terminal) = &mut block.terminal.value { 

if let Some(update) = terminal.update { 

if !hir.blocks.contains(&update) { 

^^^^^^^^^^ borrowed immutably here 

terminal.update = None; 

^^^^^^^^ mutable borrow still active here (and also bc of the loop) 

} 

} 

} 

} 

``` 

I quickly worked around this as we do in the other passes here by first building 
a set of the block ids contained in the function (so that the 
`hir.blocks.contains()` call becomes a call to the copied set of block ids). 

An alternative would be to add a helper function for mutable iteration which 
takes the desired value out of the data structure so that you can safely mutate 
it and reference the rest of the HIR. Usage might look like this: 

```rust 

hir.blocks.each_mut(|mut block, hir| { 

if let TerminalValue::ForTerminal(terminal) = &mut block.terminal.value { 

if let Some(update) = terminal.update { 

if !hir.blocks.contains(&update) { 

terminal.update = None; 

} 

} 

} 

}) 

``` 

The lambda would receive the current `block` as a mutable reference, and for the 
duration of the call `hir.blocks[block.id]` would be set to a sentinel value 
that would crash if accessed. Meanwhile, `hir` would be a readonly reference to 
the HIR, allowing the lambda to otherwise lookup information on the HIR but not 
mutate it. This seems...kinda fine? But also not immediately necessary as 
there's an easy and efficient-enough-workaround for the cases i've encountered 
so far.
2023-07-06 09:24:43 +09:00
Joe Savona 6c3792e8d5 [rust] Initial translation of HIR and related types
This is an initial translation of HIR (minus the ReactiveFunction bits). It's 
mostly a straightforward translation. A few differences: 

* Instead of Effect having an Unknown variant, we type `Place.effect: 
Option<Effect>`. Maybe i'll revert that but it seems right. 

* Terminal is divided into `Terminal { id: InstructionId, value: TerminalValue 
}` and `enum TerminalValue { ... }`, so that we can reference the terminal's id 
without caring about which kind of terminal it is. 

* Each instruction value variant gets a named struct, whereas in JS we had lots 
of anonymous InstructionValue variants. 

* Rust generators are still an unstable nightly feature and likely to change 
syntax enough that it seems safer not to rely on them. So 
`eachTerminalSuccessor()` returns a `Vec<BlockId>`. If the perf of this is bad 
enough we can switch to something like SmallVec (a popular crate) which stores a 
few elements inline on the stack to optimize for small vecs (which our case 
should be). 

Perhaps more importantly, what's _not different_: I kept the existing approach 
of BasicBlocks having a Vec of owned instruction instances, with 
InstructionValue variant operands as Places rather than indices into a shared 
instruction array. If this works out it will make porting straightforward. 

However there's one aspect of this that is incorrect: right now each `Place` 
owns its `Identifier`, whereas in JS the Identifier instances are shared mutable 
references. I'll definitely need to refactor the data structures to allow 
sharing Identifiers in some form, at which point we may want to change other 
things too.
2023-07-06 09:24:40 +09:00
Joe Savona 4c0122f4cf [rust] Small improvements plus snapshot testing for estree 2023-07-06 09:24:39 +09:00
Joe Savona 2b168ad673 [rust] SWC->ESTree conversion
Start of converting swc's representation into our estree format.
2023-07-06 09:24:38 +09:00
Joe Savona 4230eb63d8 [rust] ESTree representation with basic serialization support
Start of an `estree` crate for representing ESTree with serialization to/from 
ESTree JSON. To get the serialization to work quickly I took some shortcuts and 
uses a slightly less precise modeling. Longer-term we can write some custom 
serialization code and get more precise types (eg avoid the `ExpressionLike` 
enum in favor of proper Expression and ExpressionOrFoo types).
2023-07-06 09:24:37 +09:00
Joe Savona 292f07e19f [rust] SWC usage example
Basic example of parsing some code with SWC and then running name resolution.
2023-07-06 09:24:36 +09:00
Sathya Gunasekaran d10885ae0e [snap] Blow up if testfilter is not found in filter mode
No more failing silently when testfilter file is not found.
2023-07-05 18:53:18 +05:30
Sathya Gunasekaran 4a8b36b8b1 [hir] Check for globals while lowering UpdateExpression
lowerIdentifierForAssignment does extra checks such as checking if globals are 
lvalues. 

UpdateExpression lowers into an assignment and previously we missed out on such 
checks.
2023-07-05 20:21:09 +05:30
Sathya Gunasekaran 8eca031508 [test] Add failing test for mutating global 2023-07-05 20:21:05 +05:30
Lauren Tan 6e6ecfc4d9 [eslint-plugin] Remove babel/traverse and babel/parser
These are unused since we switched over to hermes-parser
2023-07-03 12:25:37 -04:00
Lauren Tan 2a882d798b [eslint-plugin][ez] Remove unused scripts 2023-07-03 12:25:35 -04:00
Lauren Tan c6b437b402 [eslint-plugin] Remove fbt dependencies
I incorrectly included these as deps, we were only using these to verify 
codegen. It seems fine to leave in but when I imported it internally vscode 
would error, so just remove it
2023-07-03 12:25:33 -04:00
Sathya Gunasekaran cff435ca3d [plugin] Remove duplicate export statement 2023-07-03 15:51:05 +05:30
Sathya Gunasekaran 64637ac84c [playground] Refactor pragma parsing into separate function 2023-06-23 14:56:00 +05:30
Sathya Gunasekaran 3883eb9507 [playground] Make all options configurable 2023-06-23 14:55:59 +05:30
Lauren Tan 0892e305bc [eslint-plugin] Clean up error message
Omits some unncessary details from the error message surfaced in eslint
2023-06-28 11:43:56 -04:00
Lauren Tan f850d3b265 Improve remaining InvalidReact error messages 2023-06-28 11:43:56 -04:00
Lauren Tan dda59d093a Add InvalidJS error severity 2023-06-28 11:43:55 -04:00
Lauren Tan 3fb8937391 Add first example of error suggestion 2023-06-27 14:01:40 -04:00
Lauren Tan 448ed2b67b Refactor CompilerErrorDetails
- Made most static methods on CompilerError take a single options object   as an 
argument. With the exception of invariant which takes a condition and an options 
object. - Added a new `suggestions` field on CompilerErrorDetail, which we'll   
use to provide eslint suggestions - Updated eslint-plugin-react-forget to handle 
suggestions
2023-06-27 14:01:39 -04:00
Lauren Tan c3d6789857 Reclassify existing errors
Most of these errors were incorrectly using InvalidInput as a catchall for 
rejecting code. I went through each one and manually updated them to be more 
accurate
2023-06-27 14:01:39 -04:00
Lauren Tan 6c4318e5c2 Add InvalidReact and InvalidConfig error severities 2023-06-27 14:01:38 -04:00
Sathya Gunasekaran e762d5730b [test] Update stale test expectations 2023-06-23 14:51:38 +05:30
Sathya Gunasekaran 0897cc8a0f Add failing test for DropMemoCalls failing to DCE dep array
Found this while running Forget on the React tests. 

This isn't a high priority because the ESLint plugin would've caught this. But 
it'd be nice if either our validation rules caught this or if our compiler did 
correctly eliminate the dep array.
2023-06-20 14:58:36 +01:00
Sathya Gunasekaran 2ca614cb4f [playground] Sync playground flags to Store experiment flags 2023-06-23 14:36:56 +05:30
Sathya Gunasekaran 7caa18c4b9 [playground] Parse pragma in playground 2023-06-23 14:36:55 +05:30
Lauren Tan dfb9e09984 [eslint-plugin][ez] Plugin options are const 2023-06-22 10:18:43 -04:00
Lauren Tan 9c033a481d [eslint-plugin] Only report on InvalidInput errors
Invariants aren't actionable by users, so we can omit reporting them in eslint 
as errors (while continuing to throw in the compiler itself)
2023-06-22 10:18:43 -04:00
Lauren Tan e7e67f335a Use CompilerError.invariant consistently
This PR updates all current usages of invariant to CompilerError.invariant along 
with any available locs
2023-06-22 10:18:42 -04:00
Lauren Tan e9faa5b90d SourceLocation can be null 2023-06-22 10:18:42 -04:00
Lauren Tan 3f2b346ba0 Make CompilerError.invariant assert the condition
This allows us to mimic the `invariant` api, which means you can just assert 
that something holds true after execution proceeds to the next line without 
throwing
2023-06-22 10:18:41 -04:00
Lauren Tan 6c425f6b36 Remove codeframe/nodepath options from CompilerError
These were effectively unused since we almost always passed in null when 
creating errors, so remove them and instead pass in a `loc` explicitly. The 
downside is that we no longer will see Babel codeframes in our test fixtures, 
which doesn't seem like a huge loss
2023-06-22 10:18:41 -04:00
Joe Savona ba75c96077 Fix mutable range validation in PrintHIR
Fixes the mutable range validation in PrintHIR: when we checked the identifier 
scope's range we failed to allow end=start=0 cases which are also valid. 

While I was here, I created a separate assertion pass to check all ranges, that 
way we aren't relying on the printer to validate them. The pass is on for 
playground and tests, but disabled by default so it can't break internal apps.
2023-06-21 16:08:44 -07:00
Joe Savona fa913b423e Distinguish internal invariant passes from user code validation
Distinguishes between two types of "validation" passes: 

- Passes which assert the validity of the HIR. These remain in HIR/ but are 
renamed "assertFoo". 

- Passes which validate that the user code is correct. These move to 
Validation/.
2023-06-20 15:16:57 -07:00