Also use the name "gen_snapshot" when building for Android targets
instead of appending a target architecture to the filename.
This allows flutter_tools to find the universal gen_snapshot binary when
using a locally built engine. Previously the tools would search
directories like "clang_x64" and find build outputs that may not match
the host architecture.
Moving the universal gen_snapshot binary to a separate directory avoids
conflicts with Dart's gen_snapshot binary, which is placed at the root
of the target output directory.
As of the following patches, we now bundle FlutterMacOS.framework.dSYM as part of FlutterMacOS.xcframework. The dSYM is automatically copied into the release build products directory, and bundled in the .xcarchive produced by Xcode's *Product > Archive* feature which produces bundles for upload to the App Store.
* https://github.com/flutter/engine/pull/54696
* https://github.com/flutter/flutter/pull/153975
The .dSYM bundle is now available both in the uploaded .xcarchive and in the xcframework in Flutter's internal artifact cache. For developers with CI toolchains that do additional manual handling or local archiving of .dSYMs, the dSYMs no longer need to be downloaded from cloud storage as previously detailed in `docs/Crashes.md`, but can instead be copied up from the appropriate dSYM subdirectory in the framework cache:
* `flutter/bin/cache/artifacts/engine/darwin-x64-release/FlutterMacOS.xcframework`
Also adds documentation for crash symbolication on macOS.
Issue: https://github.com/flutter/flutter/issues/153879
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
For https://github.com/flutter/flutter/issues/153925.
This limits these builds to using a little under 60% of mac capacity. If
the queue times go up, we can trade intel and arm machines between the
prod and try pool.
This PR does a few things. Mainly it makes the builds in
mac_host_engine.json each build fewer targets to increase parallelism
for post-submit and release builds. To ensure that increasing
parallelism doesn't lead to capacity issues, this change also allows the
mac_host_engine.json builds to run on either intel or arm64 macOS hosts.
Finally, when building on an arm64 macOS host to target macOS, this PR
changes the `gn` script to ensure that the arm64 native clang toolchain
will be used.
To keep mac_host_engine.json focused on builds that produce artifacts,
this PR also moves tests from that file into the ill-named
mac_unopt.json. In a subsequent PR, I'll rename all the *_unopt.json
files to *_tests.json or something similar.
The artifacts produced by these builds are passing framework presubmit
checks in https://github.com/flutter/flutter/pull/152345.
In #53524, we started producing universal `gen_snapshot_arm64` and `gen_snapshot_x64` binaries. This migrates away from bundling `gen_snapshot` binaries with x64-only host architecture to universal binaries that bundle both x64 and arm64 host architectures.
Also adds a dependency on `flutter/lib/snapshot:create_macos_gen_snapshots` to the profile build, where it was previously missing. Presumably, `gen_snapshot` was being built as a side-effect of one of the other targets.
Issue: https://github.com/flutter/flutter/issues/101138
Issue: https://github.com/flutter/flutter/issues/69157
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This patch updates `create_macos_gen_snapshots.py` to take the path to
the input `gen_snapshot` binaries rather than just the path to the
buildroot, and hardcoding the binary names and buildroot-relative paths
internally.
This has a couple advantages:
* Those grepping the source for gen_snapshot to see where it's bundled
will immediately find it in the recipe JSON as a parameter to this
script.
* In future (the next patch I send out) we can pass different input
paths that point to gen_snapshot binaries that are universal binaries.
The current binaries are single-architecture x64 host binaries.
This is a refactoring with no semantic change, therefore no tests have
changed.
Background
----------
`create_macos_gen_snapshot.py` is used to:
* Generate a named copy of gen_snapshot for the specified target
architecture: e.g. gen_snapshot_arm64, gen_snapshot_x64
* Strip bitcode, if present. Today, bitcode is no longer enabled by
default.
* Create a zip archive containing the (two) gen_snapshot binaries and
bundle with an entitlements.txt file.
Issue: https://github.com/flutter/flutter/issues/151848
This is refactoring to support:
Issue: https://github.com/flutter/flutter/issues/101138
Issue: https://github.com/flutter/flutter/issues/69157
## Pre-launch Checklist
- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I've dealt with this script more than any one person should ever
have to deal with such a thing in one lifetime.
- [ ] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I signed the [CLA].
- [X] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
This target produced an unused zipfile named `gen_snapshot.zip` containing the `gen_snapshot_${target_arch}` binary for the current build's target architecture.
The macOS recipe JSON produces the real `gen_snapshot.zip` that gets uploaded to the cloud bucket and pulled down by the tool. See: 29a474fe3e/ci/builders/mac_host_engine.json (L555-L592)
The recipe did, however, rely on the `archive_gen_snapshot` rule to ensure that the underlying `gen_snapshot` targets were actually built. See, for example:
29a474fe3e/ci/builders/mac_host_engine.json (L45)
A few benefits:
* Eliminates an unnecessary zip operation.
* Eliminates confusion from grepping for gen_snapshot.zip and finding the wrong location in the source.
* Grepping for create_macos_gen_snapshots will turn up the usage in the JSON build file.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Part 1 of https://github.com/flutter/flutter/issues/145263
This PR updates the names of builds outside of `local_engine.json` to be
prefixed with the string `ci/` (or `ci\` on Windows). For better or
worse, the "name" field of a build is used to construct a path used as
the source directory of a copy operation (I think the CAS archive
step?). Because of that, changing the name of a build also requires
updating the build output directory of the ninja build.
This PR also adds tests to make sure the naming of these builds remains
consistent.
Creates and adds FlutterMacOS.xcframework to out/mac or out/host.
Creates and archives FlutterMacOS.xcframework when building mac_host_engine. Archives the xcframework in a new zipped folder at `darwin-x64/framework.zip`, `darwin-x64-profile/framework.zip`, `darwin-x64-release/framework.zip`.
The FlutterMacOS.framework is also still archived currently - I thought it'd be better to keep it archived so we don't have to worry about the tool breaking until we're ready to remove it.
Part of https://github.com/flutter/flutter/issues/126016.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
For consistency, I don't think we have any other tools with `-`'s.
I'll be honest - this seemed easier than teaching the header guard tool
how to handle `-`'s in directory names, but if you feel strongly about
it I can revert.
This PR creates a new test type for `run_tests.py` called `dart-host`.
The tests remaining under the `dart` type are tests that run in
`flutter_tester`. The `dart-host` tests run in the Dart CLI. This allows
`run_tests.py` to be simplified a bit, and while doing this I spotted a
couple of mistakes that were causing some tests not to run.
This PR also makes a couple of small style fixes to the build config
json files, converting tabs to spaces, and moving some "script" fields
before the "parameters" fields.
Reverts flutter/engine#45097
Many failures on CI like:
```
ld: warning: ignoring file ../../../../out/ios_debug_sim_arm64_extension_safe/libocmock_shared.dylib, building for iOS Simulator-x86_64 but attempting to link with file built for iOS Simulator-arm64
ld: warning: ignoring file ../../../../out/ios_debug_sim_arm64_extension_safe/libios_test_flutter.dylib, building for iOS Simulator-x86_64 but attempting to link with file built for iOS Simulator-arm64
```
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20Production%20Engine%20Drone/131188/overview
Not sure if there are also other tests failing in different ways.
