Writes a stamp file, engine_stamp.json, containing interesting
information about this build:
```json
{
"build_date": "2025-06-27T16:35:13.279188",
"build_time_ms": 1751067313279,
"git_revision": "e68e105667e9e5cace56b0431781062b773e4831",
"git_revision_date": "2025-06-27 15:51:55 -0700",
"content_hash": "23542e452fc55890626fd1c5eb77c4788c19f983"
}
```
Later; I'll update to the flutter tool to download this artifact to
bin/cache so it can be used instead of git commands.
🚫 **BLOCKED** : Do not merge until
https://github.com/flutter/flutter/issues/161261 (additional test
coverage).
---
This PR eliminates (1) `scenario_app/android`, and all references to an
Android `scenario_app`, including documentation, CI configuration, build
rules, test runner, Android-side test application, and (2), to verify
and disambiguate the remains, renames the folder `ios_scenario_app`,
which is now accurate.
It also eliminates elements that were _only_ used in the Android-side
scenario_app, such as Firebase Test Lab uploading.
I would be open to doing this in phases if we thought it was better to
do so, but given its mostly a mechanical change (and by renaming the
directory, references can be checked merely by looking for
`/\bscenario_app/`, I believe this is safe to iterate on and eventually
merge after the holidays.
## Background
As of the merged mono-repo, there is no longer a requirement for the
engine to be testable as a standalone unit.
As an example,
[`%ENGINE%/testing/scenario_app`](3762f2e973/engine/src/flutter/testing/scenario_app),
which was intended to _emulate_ the Flutter framework (and some of
`flutter_tools` tooling), load the iOS and Android embedder, and run
various "scenarios" (which ran a combination of `dart:ui` code and
Android Java/iOS Obj-C) verifying golden-file screenshots.
Instead, it is now possible to write and run _real_ (full) Flutter apps
the same way that an end-user (or our own tests) would. One such example
is
[`dev/native_driver_test`](https://github.com/flutter/flutter/tree/master/dev/integration_tests/native_driver_test),
which is a full-fledged Flutter app, which uses standard tooling (i.e.
`flutter`), to test most of the same elements that previously were only
tested in the Android version of `scenario_app`.
Closes https://github.com/flutter/flutter/issues/157945.
```sh
% et cleanup --dry-run
Checking /Users/matanl/Developer/engine/src/out...
The following directories were accessed later than 2024-10-02:
android_debug
android_debug_arm64
android_debug_unopt
android_debug_unopt_arm64_ide
host_debug_unopt_arm64_ide
ios_debug
ios_debug_sim_unopt_arm64
ios_debug_unopt
ios_debug_unopt_arm64
license_script_output
wasm_release
web_tests
Run without --dry-run to reclaim 2.79gb.
```
```sh
flutter % et cleanup
Checking /Users/matanl/Developer/engine/src/out...
Deleted 12 output directories and reclaimed 2.79gb.
```
Closes https://github.com/flutter/flutter/issues/156909.
This PR adds (and implements) the `--gn-args` (extra command-line GN args) functionality by generalizing on the concept of "merged" GN args that @zanderso had special-cased for `--lto` and `--rbe`, and further testing it.
There is also a logical place for us to expand support of merged arguments at a future point in time.
Closes https://github.com/flutter/flutter/issues/156260.
Fixes the degenerate case where you specify an unrecognized target, and it falls back to rebuilding the entire engine as if you specified nothing. In addition, added recognition of `group`.
Closes https://github.com/flutter/flutter/issues/148444 (code de-duplicated).
Closes https://github.com/flutter/flutter/issues/150877 (`--build-strategy=local`).
Closes https://github.com/flutter/flutter/issues/150884 (`--build-strategy=remote`).
Replaces duplicate code across ~3 commands (query, test, build) with a class called `BuildPlan`, which encapsulates (a) concurrency, (b) build configuration, (c) RBE, (d) LTO, and (e) build strategy. I also moved all of the validation of the build plan into `build_plan_test`, which gives us better coverage at a lower cost (less things to think about in that suite of tests).
I know the diff looks scary, but 1K of the 1.4K is tests.
/cc @gaaclarke @flar
`build`, `test`, and `run` commands all support the `--exec-strategy`
flag ('auto', 'local', 'remote').
This flag controls the RBE task execution strategy (when RBE is
enabled).
Closes#150877
Does a few things:
- Refactors `run_command_test` significantly to reduce global fixtures
- Replaced stringly-typed things with enum-like objects
- Adds a lot stronger coverage for `run_command` to make future refactors safer
- Takes advantage of `package:test` having a workable matchers system and uses it
- Changes `return 1` into `throw FatalError(...)` where it makes sense in `run_command`
As a result of the refactoring work, I also fixed a bug: Closes https://github.com/flutter/flutter/issues/147646.
Android only for now but this avoids rebuilding a bunch of test targets that aren't necessary to do a `flutter run` with a custom engine.
I need to fill in some blanks for other platforms before landing.
Closes https://github.com/flutter/flutter/issues/147666.
This is a large change, I'd be happy to either review synchronously or
change commands one at a time, but I think this is overall the right
approach. I also didn't see any reason to reuse the `BuildRunner` code
for these commands, the flow is basically:
```mermaid
graph LR
A[et targetsOrPatterns] --> B[gn desc --format=json]
B --> C[existing code that runs ninja/workers]
```
Quick summary of changes:
- Introduced a [`Label` and `TargetPattern`][1] type to avoid awkward
string parsing/manipulation all over
- Replaced `gn_utils` (i.e. `targetsFromCommandLine` and friends) with
[an invocation of `gn desc`][2]
- Some tests were brittle in terms of expected output, I mostly left
them alone and wrote TODOs where applicable
Here is the fun part, results (and some manual integration tests):
```sh
# build, previously was 10-15s, is now ~3s
flutter % time et build -c host_debug_unopt_arm64 //flutter/impeller:impeller_unittests
[2024-05-14T19:05:55.163][macos/host_debug_unopt_arm64: GN]: OK
[2024-05-14T19:05:56.119][macos/host_debug_unopt_arm64: RBE startup]: Proxy started successfully.
[macos/host_debug_unopt_arm64: ninja] 0.0% (0/1) Regenerating ninja files[2024-05-14T19:05:57.309][macos/host_debug_unopt_arm64: ninja]: OK
[2024-05-14T19:05:57.573][macos/host_debug_unopt_arm64: RBE shutdown]: Actions completed: 0
$ENGINE/flutter/bin/et build -c host_debug_unopt_arm64 3.20s user 0.87s system 107% cpu 3.776 total
# test, also benefits from speedup, but I didn't time it
flutter % et test -c host_debug_unopt_arm64 //flutter/fml:fml_unittests
[2024-05-14T19:07:01.843][macos/host_debug_unopt_arm64: GN]: OK
[2024-05-14T19:07:02.707][macos/host_debug_unopt_arm64: RBE startup]: Proxy started successfully.
[2024-05-14T19:07:07.400][macos/host_debug_unopt_arm64: ninja]: 100.0% (3/3) LINK ./fml_unittests
[2024-05-14T19:07:07.404][macos/host_debug_unopt_arm64: ninja]: OK
[2024-05-14T19:07:07.748][macos/host_debug_unopt_arm64: RBE shutdown]: Actions completed: 1 (1 racing local)
OKAY: 7s.95ms //flutter/fml:fml_unittests
# query, also benefits from speedup
flutter % time et query targets -c host_debug_unopt_arm64
//flutter/display_list:display_list_benchmarks
//flutter/display_list:display_list_builder_benchmarks
# ... many targets omitted ...
$ENGINE/flutter/bin/et query targets -c host_debug_unopt_arm64 1.27s user 0.18s system 147% cpu 0.978 total
```
In other words, ~5x improvement on `et build` and `et query` is _much_
faster as well.
[1]:
https://github.com/flutter/engine/compare/main...matanlurey:engine:gn-label?expand=1#diff-ac008475ce9e209f4ecf6594b0cd48d12d86a1ffa0aa7d555375ff43b983eb2b
[2]:
https://github.com/flutter/engine/compare/main...matanlurey:engine:gn-label?expand=1#diff-ad116b8504dd0a500bf758a327c7052e5733a2b465d3295e35b74458d483276f
I started on a refactoring of
https://github.com/flutter/flutter/issues/147666, and was frustrated
with the JSON decoding of `gn desc --format=json`, so I used a similar
pattern from my micro-json library (https://pub.dev/packages/jsonut) and
encapsulated it as an extension type.
My favorite is the `JsonObject.map` function which replaces:
```dart
/// Construct a RunTarget from a JSON map.
factory RunTarget.fromJson(Map<String, Object> map) {
final List<String> errors = <String>[];
final String name = stringOfJson(map, _nameKey, errors)!;
final String id = stringOfJson(map, _idKey, errors)!;
final String targetPlatform =
stringOfJson(map, _targetPlatformKey, errors)!;
if (errors.isNotEmpty) {
throw FormatException('Failed to parse RunTarget: ${errors.join('\n')}');
}
return RunTarget._(name, id, targetPlatform);
}
```
... with:
```dart
/// Construct a RunTarget from a JSON map.
factory RunTarget.fromJson(Map<String, Object> map) {
return JsonObject(map).map((JsonObject json) => RunTarget._(
json.string(_nameKey),
json.string(_idKey),
json.string(_targetPlatformKey),
));
}
```
For illustrative purposes:
```sh
$ et build | grep '.*'
```
... should still get line-per-line status updates, but it does not without this patch.
It's hard to write tests because of global state, so I've declined to do so at the moment.
Closes https://github.com/flutter/flutter/issues/147903.
Previously, `Logger.test` was a leaky abstraction that stored a `@visibleForTesting` variable that was sometimes unused. It saved a few lines of code for tests, but doesn't seem necessary, so I swapped it out for a callback.
Closes https://github.com/flutter/flutter/issues/147894.
While doing this PR I realized we were basically passing `(bool verbose, Envrionment)` as a tuple around, so I just moved the concept _into_ `Environment` directly, and made the necessary code changes across the tool and tests.
To clarify, this does _not_ mimic the output of `ninja --verbose` _today_, because we also don't stream the output directly, and instead do terminal magic. Combined with a hypothetical fix for https://github.com/flutter/flutter/issues/147903, this would work exactly the same as before.
/cc @loic-sharma @johnmccutchan
https://github.com/flutter/engine/pull/52021 introduced a bug where `ci`
build could only be specified when also passing the `--verbose` flag.
This PR fixes the issue so that by default the `help` message will only
show `ci` builds when `--verbose` is passed to `help`, but the `ci`
builds can be specified even without the `--verbose` flag.
The `Command`s added to a `CommandRunner` don't automatically inherit
the `CommandRunner`'s usage line length limit. This PR does the
necessary plumbing. Help messages look nicer now.
This PR does two things. First, in many of the `ci` builds, LTO is
enabled by default. This is usually not what we want when doing local
builds, so this PR adds an `--lto` flag to the `build` command which is
disabled by default, causing `--no-lto` to be passed to GN. When `--lto`
is passed to the `build` command, `--lto` is passed to GN, which results
in the build using LTO.
Second, this PR eagerly parses the `--verbose` flag out of the command
line so that help messages can optionally show less information. In
particular, in this PR, `ci` and `web_test` builds are only displayed by
`help build` when `--verbose` is passed to the `help` command. There's
some extra text in the help message as well indicating that passing
`--verbose` to `help` will show more builds.
Addresses the notes I left on
https://github.com/flutter/engine/pull/51868. Mainly fixes an issue in
which the `build`, `query`, and `test` commands would fail if GN had not
yet been run for the requested configuration. Instead, in this PR, if
the build output directory doesn't exist yet, then it will run GN before
querying the targets.
Additionally, for the `build` command, if no targets are specified on
the command line, the PR uses the targets in the build config as the
default rather than all targets. `query` and `test` keep the same
behavior.
- s/TestTarget/BuildTarget.
- Use a more informative way of querying for build targets from gn
- Port existing code to use new interfaces
- Replace 'query tests' with 'query targets' and a --testonly flag
- Extend 'et build' with support for build target selectors.
- Extend 'et query targets' with support for build target selectors.
