Reverts "fix: get content hash for master on local engine branches (#173114)" (#173145)

<!-- start_original_pr_link -->
Reverts: flutter/flutter#173114
<!-- end_original_pr_link -->
<!-- start_initiating_author -->
Initiated by: jtmcdole
<!-- end_initiating_author -->
<!-- start_revert_reason -->
Reason for reverting: there is still another problem with the merge
queue causing the content hash to be different:

git revision: e13dd5384cc0db576291c2925852879f1111f284
actual hash: 4b6f7b0f9849efaa59f515c8e95f3f27a6eb2ffb
hash in the queue? 9e5b2eef4ba79b15b4f80dbba812d199d262366f



<!-- end_revert_reason -->
<!-- start_original_pr_author -->
Original PR Author: jtmcdole
<!-- end_original_pr_author -->

<!-- start_reviewers -->
Reviewed By: {matanlurey, chingjun}
<!-- end_reviewers -->

<!-- start_revert_body -->
This change reverts the following previous change:
The content hash doesn't exist for local engine changes, except for on
CI. If we detect we're on a branch with committed or uncommitted changes
to engine files; use "master".

towards #171790

re-land attempt for #172792  with the following changes:

1. content_aware_hash.(ps1|sh) now consider multiple branches to choose
between HEAD and merge-base.
2. content_aware_hash_test.dart updated for these new requirements 
3. content_aware_hash_test.dart allows for forcing powershell on mac for
testing
4. updated docs/tool/Engine-artifacts.md documentation.

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.

<!-- end_revert_body -->

Co-authored-by: auto-submit[bot] <flutter-engprod-team@google.com>
This commit is contained in:
auto-submit[bot] 2025-08-01 23:37:49 +00:00 committed by GitHub
parent 0420eee2d4
commit c50dd2ea93
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 19 additions and 313 deletions

View File

@ -13,11 +13,13 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4
- name: Fetch base commit and origin/master
run: |
git fetch --no-tags --prune --depth=1 origin ${{ github.event.pull_request.base.sha }}
- name: Generate Hash
run: |
# IMPORTANT: Keep the list of files in sync with bin/internal/content_aware_hash.sh
# We call this directly here as we're expected to be in the merge queue (not master)
engine_content_hash=$(git ls-tree --format "%(objectname) %(path)" HEAD -- DEPS engine bin/internal/release-candidate-branch.version | git hash-object --stdin)
engine_content_hash=$(bin/internal/content_aware_hash.sh)
# test notice annotation for retrival from api
echo "::notice ::{\"engine_content_hash\": \"${engine_content_hash}\"}"
# test summary writing

View File

@ -5,7 +5,7 @@
# ---------------------------------- NOTE ---------------------------------- #
#
# Please keep the logic in this file consistent with the logic in the
# `content_aware_hash.sh` script in the same directory to ensure that Flutter
# `content_aware_hash.ps1` script in the same directory to ensure that Flutter
# continues to work across all platforms!
#
# -------------------------------------------------------------------------- #
@ -23,51 +23,10 @@ $flutterRoot = (Get-Item $progName).parent.parent.FullName
# Cannot use '*' for files in this command
# DEPS: tracks third party dependencies related to building the engine
# engine: all the code in the engine folder
# bin/internal/release-candidate-branch.version: release marker
$trackedFiles = "DEPS", "engine", "bin/internal/release-candidate-branch.version"
$baseRef = "HEAD"
$currentBranch = (git -C "$flutterRoot" rev-parse --abbrev-ref HEAD).Trim()
# By default, the content hash is based on HEAD.
# For local development branches, we want to base the hash on the merge-base
# with the remote tracking branch, so that we don't rebuild the world every
# time we make a change to the engine.
# bin/internal/content_aware_hash.ps1: script for calculating the hash on windows
# bin/internal/content_aware_hash.sh: script for calculating the hash on mac/linux
# .github/workflows/content-aware-hash.yml: github action for CI/CD hashing
#
# The following conditions are exceptions where we want to use HEAD.
# 1. The current branch is a release branch (main, master, stable, beta).
# 2. The current branch is a GitHub temporary merge branch.
# 3. The current branch is a release candidate branch.
# 4. The current checkout is a shallow clone.
$isShallow = Test-Path -Path (Join-Path "$flutterRoot" ".git/shallow")
if (($currentBranch -ne "main") -and
($currentBranch -ne "master") -and
($currentBranch -ne "stable") -and
($currentBranch -ne "beta") -and
(-not $currentBranch.StartsWith("gh-readonly-queue/master/pr-")) -and
(-not ($currentBranch -like "flutter-*-candidate.*")) -and
(-not $isShallow)) {
# This is a development branch. Find the merge-base.
# We will fallback to origin if upstream is not detected.
$remote = "origin"
$ErrorActionPreference = 'SilentlyContinue'
git -C "$flutterRoot" remote get-url upstream *> $null
if ($LASTEXITCODE -eq 0) {
$remote = "upstream"
}
# Try to find the merge-base with master, then main.
$mergeBase = (git -C "$flutterRoot" merge-base HEAD "$remote/master" 2>$null).Trim()
if ([string]::IsNullOrEmpty($mergeBase)) {
$mergeBase = (git -C "$flutterRoot" merge-base HEAD "$remote/main" 2>$null).Trim()
}
$ErrorActionPreference = "Stop"
if ($mergeBase) {
$baseRef = "$mergeBase"
}
}
# Removing the "cmd" requirement enables powershell usage on other hosts
# 1. git ls-tree | Out-String - combines output of pipeline into a single string
# rather than an array for each line.
@ -77,6 +36,6 @@ if (($currentBranch -ne "main") -and
# 3. Out-File -NoNewline -Encoding ascii outputs 8bit ascii
# 4. git hash-object with stdin from a pipeline consumes UTF-16, so consume
#. the contents of hash.txt
(git -C "$flutterRoot" ls-tree --format "%(objectname) %(path)" "$baseRef" -- $trackedFiles | Out-String) -replace "`r`n", "`n" | Out-File -NoNewline -Encoding ascii hash.txt
(git -C "$flutterRoot" ls-tree --format "%(objectname) %(path)" HEAD DEPS engine bin/internal/release-candidate-branch.version | Out-String) -replace "`r`n", "`n" | Out-File -NoNewline -Encoding ascii hash.txt
git hash-object hash.txt
Remove-Item hash.txt

View File

@ -22,48 +22,7 @@ unset GIT_WORK_TREE
# Cannot use '*' for files in this command
# DEPS: tracks third party dependencies related to building the engine
# engine: all the code in the engine folder
# bin/internal/release-candidate-branch.version: release marker
TRACKEDFILES=(DEPS engine bin/internal/release-candidate-branch.version)
BASEREF="HEAD"
CURRENT_BRANCH="$(git -C "$FLUTTER_ROOT" rev-parse --abbrev-ref HEAD)"
# By default, the content hash is based on HEAD.
# For local development branches, we want to base the hash on the merge-base
# with the remote tracking branch, so that we don't rebuild the world every
# time we make a change to the engine.
#
# The following conditions are exceptions where we want to use HEAD.
# 1. The current branch is a release branch (main, master, stable, beta).
# 2. The current branch is a GitHub temporary merge branch.
# 3. The current branch is a release candidate branch.
# 4. The current checkout is a shallow clone.
if [[ "$CURRENT_BRANCH" != "main" && \
"$CURRENT_BRANCH" != "master" && \
"$CURRENT_BRANCH" != "stable" && \
"$CURRENT_BRANCH" != "beta" && \
"$CURRENT_BRANCH" != "gh-readonly-queue/master/pr-"* && \
"$CURRENT_BRANCH" != "flutter-"*"-candidate."* && \
! -f "$FLUTTER_ROOT/.git/shallow" ]]; then
# This is a development branch. Find the merge-base.
# We will fallback to origin if upstream is not detected.
REMOTE="origin"
set +e
git -C "$FLUTTER_ROOT" remote get-url upstream >/dev/null 2>&1
if [[ $? -eq 0 ]]; then
REMOTE="upstream"
fi
# Try to find the merge-base with master, then main.
MERGEBASE=$(git -C "$FLUTTER_ROOT" merge-base HEAD "$REMOTE/master" 2>/dev/null)
if [[ -z "$MERGEBASE" ]]; then
MERGEBASE=$(git -C "$FLUTTER_ROOT" merge-base HEAD "$REMOTE/main" 2>/dev/null)
fi
set -e
if [[ -n "$MERGEBASE" ]]; then
BASEREF="$MERGEBASE"
fi
fi
git -C "$FLUTTER_ROOT" ls-tree --format "%(objectname) %(path)" "$BASEREF" -- "${TRACKEDFILES[@]}" | git hash-object --stdin
# bin/internal/content_aware_hash.ps1: script for calculating the hash on windows
# bin/internal/content_aware_hash.sh: script for calculating the hash on mac/linux
# .github/workflows/content-aware-hash.yml: github action for CI/CD hashing
git -C "$FLUTTER_ROOT" ls-tree --format "%(objectname) %(path)" HEAD DEPS engine bin/internal/release-candidate-branch.version | git hash-object --stdin

View File

@ -22,18 +22,6 @@ import 'package:test/test.dart';
//////////////////////////////////////////////////////////////////////
void main() {
// Want to test the powershell (content_aware_hash.ps1) file, but running
// a macOS or Linux machine? You can install powershell and then opt-in to
// running `pwsh bin/internal/content_aware_hash.ps1`.
//
// macOS: https://learn.microsoft.com/en-us/powershell/scripting/install/installing-powershell-on-macos
// linux: https://learn.microsoft.com/en-us/powershell/scripting/install/installing-powershell-on-linux
//
// Then, set this variable to true:
final bool usePowershellOnPosix = io.Platform.environment['FORCE_POWERSHELL'] == 'true';
print('env: ${io.Platform.environment}');
const FileSystem localFs = LocalFileSystem();
final _FlutterRootUnderTest flutterRoot = _FlutterRootUnderTest.findWithin();
@ -70,22 +58,15 @@ void main() {
return result;
}
setUpAll(() async {
if (usePowershellOnPosix) {
final io.ProcessResult result = io.Process.runSync('pwsh', <String>['--version']);
print('Using Powershell (${result.stdout}) on POSIX for local debugging and testing');
}
});
setUp(() async {
tmpDir = localFs.systemTempDirectory.createTempSync('content_aware_hash.');
testRoot = _FlutterRootUnderTest.fromPath(tmpDir.childDirectory('flutter').path);
environment = <String, String>{};
if (const LocalPlatform().isWindows || usePowershellOnPosix) {
if (const LocalPlatform().isWindows) {
// Copy a minimal set of environment variables needed to run the update_engine_version script in PowerShell.
const List<String> powerShellVariables = <String>['SystemRoot', 'PATH', 'PATHEXT'];
const List<String> powerShellVariables = <String>['SystemRoot', 'Path', 'PATHEXT'];
for (final String key in powerShellVariables) {
final String? value = io.Platform.environment[key];
if (value != null) {
@ -125,11 +106,6 @@ void main() {
final List<String> args;
if (const LocalPlatform().isWindows) {
executable = 'powershell';
// "ExecutionPolicy Bypass" is required to execute scripts from temp
// folders on Windows 11 machines.
args = <String>['-ExecutionPolicy', 'Bypass', '-File', testRoot.contentAwareHashPs1.path];
} else if (usePowershellOnPosix) {
executable = 'pwsh';
args = <String>[testRoot.contentAwareHashPs1.path];
} else {
executable = testRoot.contentAwareHashSh.path;
@ -138,26 +114,9 @@ void main() {
return run(executable, args);
}
/// Sets up and fetches a [remote] (such as `upstream` or `origin`) for [testRoot.root].
///
/// The remote points at itself (`testRoot.root.path`) for ease of testing.
void setupRemote({required String remote, String? rootPath}) {
run('git', <String>[
'remote',
'add',
remote,
rootPath ?? testRoot.root.path,
], workingPath: rootPath);
run('git', <String>['fetch', remote], workingPath: rootPath);
}
/// Initializes a blank git repo in [testRoot.root].
void initGitRepoWithBlankInitialCommit({
String? workingPath,
String branch = 'master',
String remote = 'upstream',
}) {
run('git', <String>['init', '--initial-branch', branch], workingPath: workingPath);
void initGitRepoWithBlankInitialCommit({String? workingPath}) {
run('git', <String>['init', '--initial-branch', 'master'], workingPath: workingPath);
// autocrlf is very important for tests to work on windows.
run('git', 'config --local core.autocrlf true'.split(' '), workingPath: workingPath);
run('git', <String>[
@ -174,13 +133,6 @@ void main() {
'-m',
'Initial commit',
], workingPath: workingPath);
setupRemote(remote: remote, rootPath: workingPath);
}
String gitShaFor(String ref, {String? workingPath}) {
return (run('git', <String>['rev-parse', ref], workingPath: workingPath).stdout as String)
.trim();
}
void writeFileAndCommit(File file, String contents) {
@ -189,139 +141,11 @@ void main() {
run('git', <String>['commit', '--all', '-m', 'changed ${file.basename} to $contents']);
}
void gitSwitchBranch(String branch, {bool create = true}) {
run('git', <String>['switch', if (create) '-c', branch]);
}
// Downstream flutter user tests: (origin|upstream)/(main|master), stable, and
// beta should work.
test('generates a hash or upstream/master', () async {
test('generates a hash', () async {
initGitRepoWithBlankInitialCommit();
expect(runContentAwareHash(), processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'));
});
test('generates a hash for origin/master', () {
initGitRepoWithBlankInitialCommit(remote: 'origin');
expect(runContentAwareHash(), processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'));
});
test('generates a hash for origin/main', () {
initGitRepoWithBlankInitialCommit(remote: 'origin', branch: 'main');
expect(runContentAwareHash(), processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'));
});
test('generates a hash for upstream/main', () {
initGitRepoWithBlankInitialCommit(branch: 'main');
expect(runContentAwareHash(), processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'));
});
group('stable branches calculate hash locally', () {
test('with no changes', () {
initGitRepoWithBlankInitialCommit(branch: 'main');
gitSwitchBranch('stable');
expect(runContentAwareHash(), processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'));
});
test('with engine changes', () {
initGitRepoWithBlankInitialCommit(branch: 'main');
gitSwitchBranch('stable');
writeFileAndCommit(testRoot.deps, 'deps changed');
expect(runContentAwareHash(), processStdout('f049fdcd4300c8c0d5041b5e35b3d11c2d289bdf'));
});
});
group('beta branches calculate hash locally', () {
test('with no changes', () {
initGitRepoWithBlankInitialCommit(branch: 'main');
gitSwitchBranch('beta');
expect(runContentAwareHash(), processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'));
});
test('with engine changes', () {
initGitRepoWithBlankInitialCommit(branch: 'main');
gitSwitchBranch('beta');
writeFileAndCommit(testRoot.deps, 'deps changed');
expect(runContentAwareHash(), processStdout('f049fdcd4300c8c0d5041b5e35b3d11c2d289bdf'));
});
});
group('release branches calculate hash locally', () {
test('with no changes', () {
initGitRepoWithBlankInitialCommit(branch: 'main');
gitSwitchBranch('flutter-4.35-candidate.2');
expect(runContentAwareHash(), processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'));
});
test('with engine changes', () {
initGitRepoWithBlankInitialCommit(branch: 'main');
gitSwitchBranch('flutter-4.35-candidate.2');
writeFileAndCommit(testRoot.deps, 'deps changed');
expect(runContentAwareHash(), processStdout('f049fdcd4300c8c0d5041b5e35b3d11c2d289bdf'));
});
});
test('github special merge group branches calculate hash locally', () {
initGitRepoWithBlankInitialCommit(
remote: 'origin',
branch: 'gh-readonly-queue/master/pr-1234-abcd',
);
writeFileAndCommit(testRoot.deps, 'deps changed');
expect(runContentAwareHash(), processStdout('f049fdcd4300c8c0d5041b5e35b3d11c2d289bdf'));
}); //gh-readonly-queue/master/pr
test('generates a hash for shallow clones', () {
initGitRepoWithBlankInitialCommit(remote: 'origin', branch: 'blip');
final String headSha = gitShaFor('HEAD');
testRoot.root
.childFile(localFs.path.joinAll('.git/shallow'.split('/')))
.writeAsStringSync(headSha);
writeFileAndCommit(testRoot.deps, 'deps changed');
expect(runContentAwareHash(), processStdout('f049fdcd4300c8c0d5041b5e35b3d11c2d289bdf'));
});
group('ignores local engine for', () {
test('upstream', () {
initGitRepoWithBlankInitialCommit();
gitSwitchBranch('engineTest');
testRoot.deps.writeAsStringSync('deps changed');
expect(
runContentAwareHash(),
processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'),
reason: 'content hash from master for non-committed file',
);
writeFileAndCommit(testRoot.deps, 'deps changed');
expect(
runContentAwareHash(),
processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'),
reason: 'content hash from master for committed file',
);
});
test('origin', () {
initGitRepoWithBlankInitialCommit(remote: 'origin');
gitSwitchBranch('engineTest');
testRoot.deps.writeAsStringSync('deps changed');
expect(
runContentAwareHash(),
processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'),
reason: 'content hash from master for non-committed file',
);
writeFileAndCommit(testRoot.deps, 'deps changed');
expect(
runContentAwareHash(),
processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'),
reason: 'content hash from master for committed file',
);
});
});
group('generates a different hash when', () {
setUp(() {
initGitRepoWithBlankInitialCommit();
@ -367,20 +191,6 @@ void main() {
testRoot.flutterReadMe.writeAsStringSync('codefu was here');
expect(runContentAwareHash(), processStdout('3bbeb6a394378478683ece4f8e8663c42f8dc814'));
});
test('missing merge-base defaults to HEAD', () {
initGitRepoWithBlankInitialCommit();
run('git', <String>['branch', '-m', 'no-merge-base'], workingPath: testRoot.root.path);
run('git', <String>['remote', 'remove', 'upstream'], workingPath: testRoot.root.path);
writeFileAndCommit(testRoot.deps, 'deps changed');
expect(
runContentAwareHash(),
processStdout('f049fdcd4300c8c0d5041b5e35b3d11c2d289bdf'),
reason: 'content hash from HEAD when no merge-base',
);
});
}
/// A FrUT, or "Flutter Root"-Under Test (parallel to a SUT, System Under Test).

View File

@ -77,30 +77,6 @@ On Cocoon (Flutter's internal CI/CD) we _often_ set
[^2]: I.e. experimental branches that do not fall into one of the above.
[^3]: Only updated through `flutter-x.x-candidate.x` branches.
## Content Hashing
The content hash is a fingerprint of the assets used in producing engine artifacts.
These include:
- `DEPS`: Used to pull third_party dependencies.
- `engine/`: The entire engine subfolder[^4].
- `bin/internal/release-candidate-branch.version`: A signal for release builds, keeping builds hermetic.
The Flutter project has a plethora of users: engineers working from local branches, release branches, GitHub merge queues, and downstream shallow consumers to name the known ones. The following table shows where the content hash is calculated from:
| Branch | Hashed From |
| ----------------------- | -------------------------------------------------------------------- |
| `main` | HEAD |
| `master` | HEAD |
| `stable` | HEAD |
| `beta` | HEAD |
| GitHub Merge Queue | HEAD |
| `flutter-*-candidate.x` | HEAD |
| Shallow Clones | HEAD |
| **Everything Else**. | `merge-base` between `HEAD` and`(origin or upstream)/(main or master)` |
[^4]: This is suboptimal from an artifact building perspective, but optimal for the speed of each `dart` and `flutter` call. Flutter is called more often than it is built.
## References
The script(s) that compute (and test the computation of) the engine version: