1204 Commits

Author SHA1 Message Date
Zachary Anderson
0b18c484ac Explicitly list Android embedding dependency jars in GN (flutter/engine#51303)
Part of https://github.com/flutter/flutter/issues/144430.

From `gn help inputs`: "It may be tempting to write a script that
enumerates all files in a directory
  as inputs. Don't do this!"
2024-03-09 12:02:46 -08:00
Zachary Anderson
286169bb52 Don't rely on dart binary on PATH in run_test.py (flutter/engine#51302) 2024-03-08 23:49:32 +00:00
Chinmay Garde
4f8e474738 Update the instructions for updating licenses. (flutter/engine#51297)
* Adds markdown formatting to better highlight the important bits.
* Removes the bit about patch files not being valid because they are
copied from the GitHub UI (we use LUCI now).
* Add a shortcut for Mac to apply the patch from your pasteboard.
2024-03-08 15:18:01 -08:00
Dan Field
a4c787e072 Use io.flutter.Build.API_LEVELS rather than android.os.Build.VERSION_CODES (flutter/engine#51171)
Updates the linting script to ban the use of `VERSION_CODES`.

We currently have a mish-mash of using the integers, using `VERSION_CODES`, and even how we import the version codes. This makes it more confusing when doing things like #51070 - I think it is clearer to see `22` than `LOLLIPOP_MR1`.

I'd like to get LGTM (or at least no opinion) from all the requested reviewers here.
2024-03-08 19:04:20 +00:00
zijiehe@
3ad51979fd [Fuchsia] Providing FUCHSIA_SDK_PATH env var for fuchsia lsc (flutter/engine#51234)
This change updates test-scripts with https://crrev.com/c/5347216 so that it can override the sdk location from an env var. The override location can be used by update_product_bundles to download emulator images of an unreleased sdk from gcs buckets. The process is used by fuchsia lsc (https://ci.chromium.org/ui/p/turquoise/builders/global.ci/sdk-core-linux-flutter), now it's failing due to the wrong product_bundle location being used.

This change should have no impact to the existing builders since they do not use either gclient variables relevant. E.g. download_fuchsia_sdk and fuchsia_sdk_path.

Bug: http://b/328110079

FYI: @mbrase 

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
2024-03-08 17:21:05 +00:00
Chris Bracken
c1415300a1 Migrate vulkan_memory_allocator to flutter/third_party (flutter/engine#51275)
Migrates vulkan_memory_allocator from the buildroot's third_party directory to the engine's.

Issue: https://github.com/flutter/flutter/issues/144812
Part of: https://github.com/flutter/flutter/issues/67373

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
2024-03-08 17:17:16 +00:00
Jason Simmons
7da9c29b79 Update CI scripts to use either src/flutter/third_party/dart or src/third_party/dart (flutter/engine#51276)
See https://github.com/flutter/flutter/issues/143335
2024-03-08 17:00:17 +00:00
Martin Kustermann
899fc2a7e0 Add fuchsia_gn_sdk GN variable pointing to //flutter/tools/fuchsia/gn-sdk (flutter/engine#51287)
The Dart -> Flutter Engine autoroller seems to be failing atm due to
fuchsia build errors.

The Dart SDK CL in [0] is depending on a fuchsia gn sdk import. The
import path defaults to `//third_party/fuchsia/gn-sdk` in the Dart build
rules.

Though flutter seemingly has it in `//flutter/tools/fuchsia/gn-sdk`
(which was added to DEPS in c274921fa6034e5e133129967c0789ab8c7fc827)

=> This is an attempt to override the default & fix autoroller

[0] https://dart-review.googlesource.com/c/sdk/+/355283/14
2024-03-08 14:49:37 +01:00
Chris Bracken
0575633c34 Move ANGLE to flutter/third_party (flutter/engine#51270)
This moves Flutter out of the buildroot's third_party directory and into
//flutter/third_party and updates all BUILD and gni files.

Issue: https://github.com/flutter/flutter/issues/144786
Part of: https://github.com/flutter/flutter/issues/67373

## Pre-launch Checklist

- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] The odds of this passing on its first CI run is 0%, because
there's no way the licence script isn't going to have a fit over it, at
which point I promise on my honour that I will dutifully fix all the
things it complains about and re-run, probably more than once.
- [X] This PR fulfils the minimum [Canadian
Content](https://www.youtube.com/watch?v=h9NMnPm5kXc&list=PLJ7QPuvv91JuoPWrLhqX0zDFUoLdynMEk&index=3)
regulations mandated by the CRTC.
- [X] I signed the [CLA].
- [X] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2024-03-07 16:23:14 -08:00
Zachary Anderson
0fb043e6f6 [gn] Disable running some python scripts from GN (flutter/engine#51248)
For https://github.com/flutter/flutter/issues/144430
2024-03-07 22:18:12 +00:00
Chris Bracken
b9d7f23abd Migrate build_overrides from buildroot to engine (flutter/engine#51258)
Files under the build_overrides directory in the buildroot are typically hardcoded imports in third-party dependencies used for project-specific configuration. For example, ANGLE hardcodes an import of `//build_overrides/angle.gni` so that consumers of ANGLE can configure the build to their needs.

This adds a copy all existing src/build_overrides files at the equivalent `src/flutter/build_overrides` path in the engine. A followup patch will replace each existing file under `src/build_overrides` in the buildroot with a shim that just imports the files landed in this patch. This allows third-party dependencies to continue hardcoding the `//build_overrides/foo.gni` path, but provides a seamless path forward when we drop the buildroot.

Issue: https://github.com/flutter/flutter/issues/144790
Part of: https://github.com/flutter/flutter/issues/67373

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
2024-03-07 21:42:06 +00:00
Chris Bracken
0e6d56bc1c Move Abseil from src to flutter third_party dir (flutter/engine#51245)
In combination with:
* https://flutter-review.googlesource.com/c/third_party/abseil-cpp/+/55848
* https://github.com/flutter/buildroot/pull/831

this updates Flutter's references to Abseil from
//third_party/abseil-cpp to //flutter/third_party/abseil-cpp.

Issue: https://github.com/flutter/flutter/issues/144201
Part of: https://github.com/flutter/flutter/issues/67373

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
2024-03-07 01:19:17 +00:00
Loïc Sharma
8f1baa30bb [et] Allow users to update dependencies (flutter/engine#51177)
Allow users to update dependencies. Examples:

* `et fetch` Fetch dependencies

In the future, `et build` will update dependencies if it detects that they have changed.

Also:
* Updates the status in the README
* Adds instructions on how to run tests
* Fixes `et run`'s description
* Makes the `--verbose` flag global
2024-03-06 17:00:23 +00:00
Loïc Sharma
d6bd7667e9 Fix git hooks on Windows (flutter/engine#51203)
Windows does not pre-install python3. Previously Windows users would just use Python from depot tools, but this was broken by https://github.com/flutter/engine/pull/51156.
2024-03-05 22:54:42 +00:00
Zachary Anderson
408b0a8231 Fix typo in githook message (flutter/engine#51205) 2024-03-05 12:02:54 -08:00
John McCutchan
5a25c24c43 Add device selection to et run (flutter/engine#51184)
- `et run` now detects the target device automatically and builds the
related engine builds.
- `et run -- -d <device-id>` also works.
2024-03-05 10:49:05 -08:00
Zachary Anderson
6220641bde Adds githooks that reminds to run gclient sync -D (flutter/engine#51156)
This PR adds githooks for `post-checkout`, `post-merge`, `pre-rebase`
that remind to run `gclient sync -D`. This is probably going to print
the reminder too much. The `pre-rebase` hook runs before a `git pull
--rebase` that is actually going to update something, but the other
hooks may be needed to cover other workflows. The printed message will
also include the hook that it comes from, so we can remove the message
from hooks where it doesn't make sense.

<img width="670" alt="Screenshot 2024-03-04 at 18 36 15"
src="https://github.com/flutter/engine/assets/6343103/4d3e4661-035d-4ed6-8ed6-2a05b372bf65">
2024-03-05 10:23:34 -08:00
Zachary Anderson
8928d5284c Folds concurrent job calculation into tools/gn (flutter/engine#51193)
Folds the `get_concurrent_jobs.py` script into `tools/gn` and stops calling it with `exec_script()` during GN.

Part of https://github.com/flutter/flutter/issues/144430
2024-03-05 17:57:14 +00:00
Zachary Anderson
1d7737e0a0 Shift git version fetching to tools/gn (flutter/engine#51175)
Avoids GN invoking python scripts that run git commands to determine git
hashes.

Part of https://github.com/flutter/flutter/issues/144430
2024-03-04 18:14:04 -08:00
Dan Field
19296c0ffd Guard against API 22 (flutter/engine#51167)
This was introduced in #51070 - my grepping for `LOLLIPOP` was too aggressive, and apparently the linter didn't catch this. I think it's just a bug in the linter.

Fixes b/327717572, because Google internal tests caught this.
2024-03-04 18:40:19 +00:00
Zachary Anderson
49b2bad89d Run apple SDK finders as gclient hooks (flutter/engine#50957)
Currently, these scripts run on each invocation of `gn`, and can take
many seconds to run.

This PR shifts them to run as `gclient` hooks instead, so that `gn` will
be faster.

Needs https://github.com/flutter/buildroot/pull/825.
2024-02-29 18:14:31 -08:00
Matan Lurey
7b7d9a2f43 Add a basic e2e-test for skia_gold_client. (flutter/engine#51078)
Basically, I take blank images of red/blue/green squares, like this:

![solid_red_square](https://github.com/flutter/engine/assets/168174/b1de63e4-8c3c-43ff-9bcf-b067a7e9d920)

... and use Image Magick to annotate it with the current git hash:

```sh
$ drt ./testing/skia_gold_client/tool/generate.dart

Writing annotation "8069cb4ca1" on images in testing/skia_gold_client/tool/source_images and saving them in testing/skia_gold_client/tool/e2e_fixtures.
Writing to testing/skia_gold_client/tool/e2e_fixtures/solid_red_square.png
Writing to testing/skia_gold_client/tool/e2e_fixtures/solid_green_square.png
Writing to testing/skia_gold_client/tool/e2e_fixtures/solid_blue_square.png
Done: wrote 3 image.
```

![solid_red_square](https://github.com/flutter/engine/assets/168174/0ece31a8-9e64-44a2-ad06-78c04b02650c)

... then, I upload the digests very similar to how we do it in `scenario_app`.

---

The idea here is to have a way for me to know if Skia gold is working the way I/we expect, independent of a more complicated test suite with various race-y or flake-y conditions. We could also augment it with tests of "dimensions" properties.
2024-03-01 01:33:07 +00:00
John McCutchan
0be7a9c4f0 Add et run command (flutter/engine#51037)
The `run` command builds both the host and target engines and then invokes `flutter run` so that it runs the app using the custom engine builds.

It is expected that 'et run' be used inside the directory of a flutter application.

Command line flags passed after `--` will be forwarded to `flutter run`.

Some examples:

- `et run` Build the debug variant and runs the app in that mode.
- `et run -- --profile` Build the profile variant and runs the app in that mode.
- `et run -- --release` Build the release variant and runs the app in that mode.

Also:

- Start a local_engine.json builder definition (it is missing a lot)
- Tweak the test fixture.
- Add a new Matcher to litetest.
- Tweaked the build config linter to only care about duplicate build names within the same builder.
2024-03-01 01:09:04 +00:00
Dan Field
2e9e6107eb Reapply "Bump everything to Android 21" (#51056) (flutter/engine#51070)
This reverts commit 98485b3f82fec45a6a0ac932c3c20285a9a14e02.

Reverted in https://github.com/flutter/engine/pull/51056 because it
caused failures in the framework. No changes, but framework test has
been updated to use an actual robolectric implementation to avoid those
failures.

This should not land until
https://github.com/flutter/flutter/pull/144348 has landed in the
framework.

@gaaclarke @johnmccutchan fyi
2024-02-29 09:00:01 -08:00
Dan Field
98485b3f82 Revert "Bump everything to Android 21" (flutter/engine#51056)
Reverts flutter/engine#51032

Probably causing some of the failures in https://github.com/flutter/flutter/pull/144326

@johnmccutchan fyi
2024-02-28 19:15:04 +00:00
Dan Field
6efee79357 Bump everything to Android 21 (flutter/engine#51032)
Followed the linter, got rid of a bunch of version check/guards we don't need anymore.
2024-02-28 15:56:40 +00:00
John McCutchan
cdeb8166a2 Rename some classes in the engine_build_configs package (flutter/engine#51016)
- BuildConfig -> BuilderConfig
- GlobalBuild -> Build
2024-02-27 15:12:10 -08:00
Zachary Anderson
e7b97723ec Remove rewrapper prefix from compiler commands for clang-tidy (flutter/engine#51001)
This PR also enables RBE for builds for clang-tidy.
2024-02-27 14:57:51 -08:00
Jason Simmons
e017114099 Move vulkan-deps to //flutter/third_party/vulkan-deps (flutter/engine#51013)
See https://github.com/flutter/flutter/issues/144205
2024-02-27 21:59:00 +00:00
zijiehe@
c274921fa6 [Fuchsia] Use shared gn-sdk from chromium (flutter/engine#50855)
Fuchsia team is working on getting rid of non-versioned libraries and sysroot (see b/40935282), but flutter is using a fairly old version of in-house gn-sdk. It's hard to maintain it anymore.
So this change removes the in-house gn-sdk and replaces it with the gn-sdk from chromium.
This is a prerequisite to select the idk / sdk lib version according to the api-level.

This change contains three parts:
1) Replacing gn-sdk/ with gn-sdk/src/ since the chromium needs a README.chromium in gn-sdk/ folder.
2) Making the existing build rules work with the newer gn-sdk.
3) Drive-by replacing "gtest_runner.shard.cml" with "sys/testing/gtest_runner.shard.cml", it's in fuchsia sdk already.

Bug: [b/40935282](https://issues.chromium.org/issues/40935282?pli=1&authuser=0)

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
2024-02-27 17:30:05 +00:00
Zachary Anderson
827e920f48 Move protobuf to //flutter/third_party (flutter/engine#50997)
For https://github.com/flutter/flutter/issues/67373

Currently the files in the secondary buildroot for protobuf are checked
out from their own git repo. Flutter owns a mirror of that repo. To move
protobuf to `//flutter/third_party`, we can land a change in that repo,
and then revert it after we're done with the buildmoot. Alternately, in
this PR, I've just copied the files into the flutter/engine repo under
the secondary buildroot and made the changes.
2024-02-27 08:37:21 -08:00
Zachary Anderson
cae1c8dc3a Run engine unit tests on mac host_debug_unopt_arm64 (flutter/engine#50327)
Related https://github.com/flutter/flutter/issues/142868
2024-02-26 13:39:05 -08:00
Zachary Anderson
8a4feb869c [et] Improve the logger for the ninja build, adds a spinner (flutter/engine#50952)
For https://github.com/flutter/flutter/issues/132807

The spinner is mostly copied from the flutter_tool.
2024-02-26 09:31:15 -08:00
Zachary Anderson
9f1ed38ee0 Shift //third_party/icu to //flutter/third_party (flutter/engine#50924)
For https://github.com/flutter/flutter/issues/67373
2024-02-23 23:23:27 +00:00
John McCutchan
82dd6cd6e3 Add an initial 'build' command to engine_tool (flutter/engine#50681)
* Adds a 'build' command to `et`.
* Renamed `query builds` to `query builders` because that seemed to be
more precise(?)

Some extra requests during review:
1. I've left some questions I'd like answers to at the top of
`build_command.dart` I suspect @zanderso and @loic-sharma can give me
(some?) answers.
2. I suspect I'm holding the FakeProcessManager wrong or there is a
better way to write the tests in `build_command_test.dart`. Pointers to
good examples are appreciated.
2024-02-23 10:02:34 -08:00
Matan Lurey
b357d7a45e Remove/reduce unused or private methods and add tests to SkiaGoldClient (flutter/engine#50844)
@Piinks correctly pointed out we had to make a number of changes to the engine's `SkiaGoldClient` and we had no test coverage (https://github.com/flutter/engine/pull/50829, https://github.com/flutter/engine/pull/50826).

This corrects the problem.

I am not in _love_ with these tests (they could be less snapshot-y), but the mechanisms I need to write better tests is beyond what I can spend ~1-2 hours on (https://github.com/flutter/flutter/issues/133569).

/cc @dnfield @mdebbar
2024-02-22 00:16:57 +00:00
Zachary Anderson
e6d9dca1b7 Shift some deps to //flutter/third_party (flutter/engine#50830)
Part of https://github.com/flutter/flutter/issues/67373
2024-02-21 23:51:16 +00:00
Zachary Anderson
701be200cd Starts a .ci.yaml parser (flutter/engine#50783)
Towards https://github.com/flutter/flutter/issues/132807.

This is needed so the engine tool can map the names of CI builders
listed in `.ci.yaml` to the names of the configurations in the build
config json files in `ci/builders`.
2024-02-21 14:56:07 -08:00
Matan Lurey
05fedd7c11 Ignore EOF newline characters and added tests to dir_contents_diff tool (flutter/engine#50805)
No issue filed, but was [discussed on discord](https://discord.com/channels/608014603317936148/608021010377080866/1209568840644567091).
2024-02-21 19:05:26 +00:00
Zachary Anderson
f6629ffe5c Use 'et format' in CI. Check formatting of all files in CI (flutter/engine#50810)
This PR changes the format check on CI to use the command added in
https://github.com/flutter/engine/pull/50747.

Additionally, while making this change, I noticed that the CI check was
not checking the formatting of all files, and that as a result, files
were present in the repo with incorrect formatting. I have fixed the
formatting and fixed the check to always check all files.
2024-02-21 09:38:08 -08:00
Jason Simmons
fa635a5b27 Add pub.dev to the hosted package URIs checked by the pub_get_offline script (flutter/engine#50811)
pub get --offline may fetch packages that are in the local package cache but were previously downloaded from a hosted source.  The script needs to check for packages that came from either the old or new locations of the Dart hosted package repository.
2024-02-21 04:45:53 +00:00
Zachary Anderson
6ebbce07e7 Make global builds uniquely named per platform (flutter/engine#50764)
This is expected to help with
https://github.com/flutter/flutter/issues/132807
2024-02-20 19:41:40 -08:00
Zachary Anderson
6a936a530c [et] Adds a format command (flutter/engine#50747)
For https://github.com/flutter/flutter/issues/132807
2024-02-20 19:40:47 -08:00
Zachary Anderson
77cbd54c43 [et] Adds a .bat entrypoint for Windows (flutter/engine#50784)
For https://github.com/flutter/flutter/issues/132807
2024-02-20 18:48:18 +00:00
gaaclarke
8515af7ee9 Pulled out dir contents golden tool (flutter/engine#50703)
fixes https://github.com/flutter/flutter/issues/143459

This also starts using it as part of scenario_app.

Testing: Is part of the testing infrastructure.

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
2024-02-16 22:18:05 +00:00
Jason Simmons
ea6b80144d Add the web package to the set of Dart third_party packages in DEPS (flutter/engine#50714)
Dart is rolling the http package to a version that depends on the web package. (see https://dart.googlesource.com/http/+/d8b237d273f49bada2aa7feaa0e7795e2541fe83)
2024-02-16 06:54:49 +00:00
Zachary Anderson
80076b93fb [et] Adds a logger (flutter/engine#50693) 2024-02-16 00:22:54 +00:00
gaaclarke
fcbd40e8e4 Added tool to easily check golden diffs locally. (flutter/engine#50654)
This addresses the problem where people feel like they have to upload a PR to get diffs from Skia Gold.  This should work for most workflows.

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
2024-02-15 17:50:06 +00:00
Chinmay Garde
6c17d7bf9d Use a GN variable (dart_src) to reference the location of the Dart checkout. (flutter/engine#50624)
Towards https://github.com/flutter/flutter/issues/143335
2024-02-14 21:12:23 -08:00
Matan Lurey
53ab3dc4b4 Move boringssl to flutter/third_party. (flutter/engine#50601)
As ready as it's ever going to be.
2024-02-14 17:17:28 -08:00