This eliminates a significant number of dependencies, eliminating all
npm audit issues, speeding up `npm ci` by 20%, and overall making the
build faster (faster startup, direct code is faster than streams, etc)
and clearer to understand.
I'm finding it much easier to make build changes for the module
transform with this; I can more clearly indicate task dependencies and
prevent running tasks that don't need to be run.
Given we're changing our build process entirely (new deps, new steps),
it seems like this is a good time to change things up.
When bundled, it's very likely that the function "require" will actually
exist at runtime, so we can't use this to determine if we are running in
Node. Consolidate that logic and use other things to check instead.
This is still not perfectly accurate, but I don't want to change this
_too_ much, lest someone downstream depend on our inconsistent logic.
There are yet other places this commit does not fix; search for "typeof
process" for more examples.
This adds a "small" d.ts bundler script. This script is very basic,
using Node printing to produce its output. Generally speaking, this is
inadvisable as it completely disregards name shadowing, globals, etc.
However, in our case, we don't care about the globals, and we can opt to
restructure our codebase in order to avoid conflict, which we largely
had to do anyway when we were namespaces and everything was in scope.
This configures the existing build tasks to use esbuild by default. If
using the plain files is desired, passing `--bundle=false` will build
using plain files and still produce a runnable system.
This is only a basic build; a more efficient build is provided later
when gulp is replaced by hereby.
If these are regular comments, then they won't appear in our d.ts files.
But, now we are relying on an external d.ts bundler to produce our final
merged, so they need to be present in the "input" d.ts files, meaning
they have to be JSDoc comments.
These comments only work today because all of our builds load their TS
files from scratch, so they see the actual source files and their
non-JSDoc comments.
The comments also need to be attached to a declaration, not floating,
otherwise they won't be used by api-extractor, so move them if needed.
This step converts as many explicit accesses as possible in favor of direct imports from the modules in which things were declared. This restores the code (as much as possible) back to how it looked originally before the explicitify step, e.g. instead of "ts.Node" and "ts.Symbol", we have just "Node" and "Symbol".
This step converts each file into an exported module by hoisting the namespace bodies into the global scope and transferring internal markers down onto declarations as needed.
The namespaces are reconstructed as "barrel"-style modules, which are identical to the old namespace objects in structure. These reconstructed namespaces are then imported in the newly module-ified files, making existing expressions like "ts." valid.
* Remove unnecessary parameter
* Dont store name unnecessarily in the watchers
* Polled watches and not files
* Use fs events as default watching
* Some refactoring
* Make single per directory native watchers now that we are using it as default
* Rename
* Comment
* Convert some of the watchEnvironment tests to baselines for updating later
* Add tests for inode watching by making fsWatch part of system function that tests presence before creating fs watch
* Refactor for simpler tests
* Accept map of file content or file or symlink or folder
* Add test when rename event occurs when file has already reappeared
* On rename event for the file, replace file watcher irrespective of file presence
* Fix regex
* Ensure that when doing inode watching watchers is replaces only on disappearance or appearance
* Some logging for debugging further
* Revert "Some logging for debugging further"
This reverts commit dd2164ac83.
* Add test when rename event occurs on mac with ~ appended to file name
* If the relativeFileName ends with tilde, remove it from the event
* Some logging for debugging further
* Revert "Some logging for debugging further"
This reverts commit e1ba8a8d5f.
* Add documentation and fail safe the event firing
* Use fixed time for vfs so baselining is consistent
* Baseline buildinfos
* Write new file text in baseline even if the file wasnt read on the shadow
* Remove unnecessary debugger statement
* Make sure that incremental correctness is checked with correct writeFile so we know buildInfo was written
Also baseline these so its easy to verify the changes
* More baselines for the tsbuildinfo
* If we are writing dts file and have used file text as version, we can update the signature when doing actual emit
* Make WriteFileCallback Api ready for future
* Assert that there is only single source file when emitting d.ts file
* Add test
* Renames
* More refactoring
* If we are updating dts of any of the file and it affects global scope, everything needs update in signature and dts emit
Fixes#42769
* Stacktrace optimization for getModified time in anticipation of using it more than fileExists wherever possible
* Baseline getModifiedTime, setModifiedTime, fileExits and directoryExits for experiment
* Remove unnecessary write file finger print code since its not used at all
* Use modified time instead of file existence check
* Remove unnecessary getModifiedTime
* No need to check for file existence before reading the d.ts file
* Do project reference errors before doing input/output file checks
* Dont call getModifiedTimes if dts change
* Passdown modified time if queried
* Use modified time passed through the file watching in tsbuild
* Handle force build as separate upto date status
* uptodate status worker to read buildinfo and use it to determine upto date ness
* No need to update output timestamps if buildinfo will determine uptodateness
* Store change file set instead of hasPendingChange to be able to reuse the information
* Add test that shows input file is not present
* No need to check input time stamp before buildinfo
* Keep buildinfos for lifetime of the solution builder and project
* Store modified time along with text of buildinfo
* Non composite projects dont need to track declaration change time
* Pass through buildInfo so we dont have to parse it back
* Save dts change time in buildinfo itself
* Store dts time for --out in the buildInfo
* Store hash of text in the bundle info so it can be verified before manipulating text for fast updates during prepend
This helps when text changes during incremental build toggling and we determine we can just manipulate text
* Since buildinfo is cached no need to maintain version check state
* Store output time stamps for non incremental builds
* Revert "Baseline getModifiedTime, setModifiedTime, fileExits and directoryExits for experiment"
This reverts commit 7e65cd3315.
* Change verbose messages for upto date status
* Reconcile reusable builder state and builder state so there are not two different types that are almost similar looking
* Cleanup impliedFormat
* Cleanup
* Cleanup noEmit option
* BuildInfo options emit as a flag
* Factor out types for program written in buildinfo with and without bundle emit
* No need to store output file stamps if not in watch mode
* Cleanup
* Test for single watch per file
* Fix emit and error update baselines that were duplicate
* More refactoring
* Only copy emit state fields when backing up to restore if emit fails
* Instead of maintaining delta of changes, maintain old state for those changes
* Add test to verify build when input file does not change
* If version of the input file does not change, dont mark as out of date
* Disable lint warning as build fails without the assert
* Report aggregate statistics for solution as well as some solution perf numbers
* Options solutionDiagnostics instead so that its not too verbose when printing diagnostics
* When tsc --build --clean, only remove tsbuildinfo if its incremental build
* Revert "Options solutionDiagnostics instead so that its not too verbose when printing diagnostics"
This reverts commit 0cf9e30103.
* Revert "Report aggregate statistics for solution as well as some solution perf numbers"
This reverts commit 405d8e9197.
* Revert "When tsc --build --clean, only remove tsbuildinfo if its incremental build"
This reverts commit e4e6672609.
* Comments in the code
* Feedback
* If we are writing dts file and have used file text as version, we can update the signature when doing actual emit
* Make WriteFileCallback Api ready for future
* Assert that there is only single source file when emitting d.ts file
This removes a few places where global compiler bindings are mutated:
1. Make `sysLog` call an internal binding which is changed via
`setSysLog`.
2. Use `Object.assign` to change values *in* `objectAllocator` instead
of mutating the binding itself. (The type should verify that any
future uses of this will properly override all bindings.)
3. `localizedDiagnosticMessages` is not really needed as an exported
value, there's only one place that uses it to test whether it is set
or not. So drop the export and replace it with a new
`maybeSetLocalizedDiagnosticMessages` (internal) function.
* shorthand -> long for `factory` since the typeformer script doesn't
know how to handle it.
* Use setter to change `ts.sys` (similar to #35399).
* Fix `loggedIO` with empty namespaces to indicate dependency
(similar to 50603eda).
* Move `Map` / `Set` stuff from `core.ts` to `corePublic.ts` since the
types are there.
* Make getSourceOfProjectReferenceRedirect take a Path
* Add useCaseSensitiveFileNames to ModuleResolutionHost
...so that path comparisons can use it during module resolution.
* Re-enable realpathSync.native for case-insensitive file systems
* Don't use _fs.realpathSync.native on windows, a semi-revert of #41292 (#43348)
We're planning a real fix for TS 4.3, but port the workaround from 4.2
so the beta doesn't have this bug.
(cherry picked from commit e462dfa347)
* Un-reverse condition
Co-authored-by: Orta Therox <ortam@microsoft.com>
* Extra tests in preparation for lazy signature making sure the original intent of test is maintained
* Whenver we cant use state delay signature calculation and use source file version as signature
* Incremental correctness checks
* Retain old behavior in compile on save by disabling use of file version as signature in when state is not reused
* Test that forceConsistentCasingInFileNames does not apply to Windows drive roots
* Add file symlink baselines
* Add directory symlink baselines
* Update test to retain its meaning
Its purpose is (apparently) to demonstrate that
forceConsistenCasingInFileNames can interact badly with synthesized
react imports. Since the casing of the synthesized import has changed,
also modify the casing of the explicit reference to still conflict.
* Make VFSWithWatch.realpath use the path on disk
* Update VFS realpathSync to behave like realpathSync.native
* Use fs.realpathSync.native when available
In local measurements of an Office project, we saw initial project
loading get 5% faster on Windows and 13% faster on Linux. The only
identified behavioral change is that it restores the case used on disk,
whereas realpathSync retains the input lowercase.
* Rename SortedMap.getKeyAndValue to getEntry
Future versions of node will be able to return undefined, rather than
allocating and throwing an exception, when a file is not found.
See https://github.com/nodejs/node/pull/33716
* Parse excludeDirectories and excludeFiles
* Use watch factory in typings installer
* Some refactoring for watchFactory
* Create Noop watcher if file or directory being watched is excluded
* Baselines without using exclude watch options
* Baselines including exclude option
* Handle exclude options in the system watches
* Add test without exclude option for recursive directory watching
* Test baselines with exclude option
* Always set sysLog
* Test for exclude option in server
* Add exclude options in the config file and fix the test
* Fix host configuration for server
* Handle host configuration for watch options
* Fix sysLog time log so baselines can be clean
* Handle reloadProjects to reload the project from scratch
* Ensure that file updates are reflected
* Feedback
* Feedback
The exception thrown by Node.js's fs.statSync function contains a stack
trace that can be expensive to compute. Since this exception isn't used
by fileSystemEntryExists, we can safely set Error.stackTraceLimit to 0
without a change in behavior.
---
A significant performance improvement was noticed with this change while
profiling tsserver on packages within a proprietary monorepo.
Specifically, my team saw high self time percentages for Node.js's
uvException and handleErrorFromBinding internal functions. These
functions are executed within fs.statSync when it fails to find the
given path.
https://user-images.githubusercontent.com/906558/90183227-220cb800-dd81-11ea-8d61-f41f89481f46.png
fs.statSync: https://github.com/nodejs/node/blob/v14.4.0/lib/fs.js#L1030-L1037
handleErrorFromBinding: https://github.com/nodejs/node/blob/v14.4.0/lib/internal/fs/utils.js#L254-L269
uvException: https://github.com/nodejs/node/blob/v14.4.0/lib/internal/errors.js#L390-L443
## Measurements
After adding Error.stackTraceLimit = 0, we saw:
- For a large configured project with 12,565 files, tsserver reached the
projectLoadingFinish event 48.78% faster. (~46.786s vs ~31.447s)
- For a medium project with 7,064 files, tsserver was 25.75% faster.
(~20.897s vs ~16.618s)
- For a small project with 796 files, tsserver was only a negligible
3.00% faster. (~3.545s vs ~3.442)
Measurements were taken on macOS 10.15.6, Node.js 14.4.0, and a recent
master commit of TypeScript (610fa28d). The average of 3 runs before and
after this change were taken.
I would normally include .cpuprofile and isolate-*-*-*.log files, but
can't post them publicly in this case. If there's any other summaries
the TypeScript team would be curious about I can report them.
## fs.statSync Misses
Within our monorepo, the fs.statSync misses were mostly searches for
alternative file extensions of module imports.
- For node_modules imports, a lot of .ts/.tsx lookups failed until the
.d.ts file was found.
- Within projects with a lot of JSX files, .ts files were looked for
before finding the .tsx version.
- In the medium scale project mentioned above, a total of 38,515
non-existent files were queried during createProgram.
* Add full implemention of Map and Set to shims
* Update default Map interface
* Remove WeakMap/WeakSet
* Add tests for set shim
* Update most usages of Map<K, true> to Set
* PR Feedback
* Fix lint issues
* Change key in fsWatchCallback
* Simpler shim, more tests
* Fix typo in collection shim
* If there is no changes to folder structure when watching directories recursively, send the updates to fileNames only
Fixes#37994
* Ignore excluded directories from wild card watching
* Testcase showing that renaming file with non sync directory watcher displays correct error in the end
Testcase for #38684
Jake Bailey
Sheetal Nandi
Eli Barzilay
Andrew Branch
Andrew Casey
Ryan Cavanaugh
Song Gao
Oleksandr T
Wesley Wigham
Ron Buckton
Brandon Cheng
Anders Kaseorg
