Commit Graph
10 Commits
Author SHA1 Message Date
Rubén Norte c169250a36 Implement test.only in benchmarks (#49222)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/49222

Changelog: [internal]

This implements `test.only` in Fantom benchmarks, so we can focus on a specific case to speed up iteration.

Reviewed By: sammy-SC

Differential Revision: D69241220

fbshipit-source-id: 42b02fcb4d693988da4fa15a0c6bd7e90e473b9f
2025-02-06 05:50:19 -08:00
Rubén Norte 1902c3c4d5 Rename add as test in benchmarking API (#49223)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/49223

Changelog: [internal]

Renames `suite.add()` as `suite.test()` for symmetry with Jest.

We'll also allow `test.only` in a following change for quick iteration.

Reviewed By: rshest

Differential Revision: D69241221

fbshipit-source-id: d141f80dc0c8e51b419ce233bca68bf0755fd356
2025-02-06 05:50:19 -08:00
Rubén Norte 6554a99c0d Add option to enable test only mode in benchmarks (#49193)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/49193

Changelog: [internal]

Adds an option to force test-only mode in Fantom benchmarks.

Reviewed By: javache

Differential Revision: D69176982

fbshipit-source-id: 97b23604961eb4ee8747b278a8439d0b0075dc07
2025-02-05 04:29:19 -08:00
Rubén Norte 95160c1e6b Fix testOnly behavior in CI (#49192)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/49192

Changelog: [internal]

We refactored the public API of Fantom benchmarks in https://github.com/facebook/react-native/pull/49014 but that refactor broke test only mode, as we started overriding the options after setting them. This fixes that.

Reviewed By: javache

Differential Revision: D69176983

fbshipit-source-id: 9afc2d2f27fb2ee0aa452d4b02c28531acf40b8e
2025-02-05 04:29:19 -08:00
Rubén Norte 7e12060420 Move Fantom-related specs to src/private/testing/fantom/specs (#49101)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/49101

Changelog: [internal]

We can move this out of the deprecated directory. Also added a `.npmignore` entry so this won't be published to npm with the package.

Reviewed By: lenaic

Differential Revision: D68896208

fbshipit-source-id: ec85236aeeabdc9abcd870f0f4c1322eeb3cc659
2025-02-03 09:15:31 -08:00
Rubén Norte 2f27327f33 Rename src/private/specs as src/private/specs_DEPRECATED (#49068)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/49068

Changelog: [internal]

Renamed directory to better signal it's deprecated and added README.md to clarify intent.

Reviewed By: huntie

Differential Revision: D68895998

fbshipit-source-id: 5bc70d0782194db27c27cc89cc402d99f11aafa4
2025-02-03 09:15:31 -08:00
Rubén Norte 252294bc76 Improve naming for benchmark suite options (#49014)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/49014

Changelog: [internal]

Use better names for the Fantom benchmarking API than the ones defined in `tinybench`:
* `time` => `minDuration`
* `iterations` = `minIterations`
* `warmupTime` => `minWarmupDuration`
* `warmupIterations` => `minWarmupIterations`

Reviewed By: rshest

Differential Revision: D68710952

fbshipit-source-id: 05dc1145a72a50ea73de7ccbb08bb28d7975245f
2025-01-30 03:24:39 -08:00
Rubén Norte 8702ed5f37 Improve format of benchmark results (#48881)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/48881

Changelog: [internal]

Small improvement in the printed benchmark results.

Reviewed By: andrewdacenko

Differential Revision: D68552379

fbshipit-source-id: 39312a757a9de57e462c17a9f1455dfe8cab53f1
2025-01-23 10:26:47 -08:00
Rubén Norte ff7c550a86 Run benchmarks in test mode when not specifying verification functions in CI (#48451)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/48451

Changelog: [internal]

Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a `verify` method.

If a benchmark uses `verify`, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time.

Reviewed By: rshest

Differential Revision: D67637754

fbshipit-source-id: 33b78a9c809386cf2e040314b0427de6a53da3e3
2025-01-06 07:10:28 -08:00
Rubén Norte 3795f0fb66 Add API to run benchmarks in Fantom (#48452)
Summary:
Pull Request resolved: https://github.com/facebook/react-native/pull/48452

Changelog: [internal]

Implements a basic API to run benchmarks with Fantom (using `tinybench` under the hood):

```
import {benchmark} from 'react-native/fantom';

benchmark
  .suite('Suite name', {
    // options
  })
  .add(
    'Test name',
    () => {
      // code to benchmark
    },
    {
      beforeAll: () => {},
      beforeEach: () => {},
      afterEach: () => {},
      afterAll: () => {},
    },
  )
  .verify(results => {
    // check results and throw an error if the expectations fail
  });
```

Features:
* Print benchmark results in the console as a table.
* It opts into optimized builds automatically
* Verifies that optimized build is used (unless manually opting out of the check via `disableOptimizedBuildCheck`).
* Supports verification of results (making expectations and making the test fail if the benchmark doesn't meet some expectations).

Reviewed By: rshest

Differential Revision: D66926183

fbshipit-source-id: 61cfa7689ea7684eb870fbbc815b8d236a1871e6
2025-01-06 07:10:28 -08:00