From cfafb9105f74a17ee7ad3d863bc014d74e0fd460 Mon Sep 17 00:00:00 2001
From: keyonghan <54558023+keyonghan@users.noreply.github.com>
Date: Thu, 27 Jul 2023 08:16:09 -0700
Subject: [PATCH] Add DeviceLab build+test separation doc (#131365)

This PR add instructions to separate DeviceLab tasks to build and test model.
---
 dev/devicelab/README.md | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/dev/devicelab/README.md b/dev/devicelab/README.md
index 39cd020fe50..afa8300f153 100644
--- a/dev/devicelab/README.md
+++ b/dev/devicelab/README.md
@@ -19,6 +19,7 @@ for information on using the dashboards.
 * [Adding tests to continuous
   integration](#adding-tests-to-continuous-integration)
 * [Adding tests to presubmit](#adding-tests-to-presubmit)
+* [Migrating to build and test model](#migrating-to-build-and-test-model)
 
 ## How the DeviceLab runs tests
 
@@ -226,3 +227,33 @@ target for each operating system.
 
 Flutter's DeviceLab has a limited capacity in presubmit. File an infra ticket
 to investigate feasibility of adding a test to presubmit.
+
+## Migrating to build and test model
+
+To better utilize limited DeviceLab testbed resources and speed up commit validation
+time, it is now supported to separate building artifacts (.apk/.app) from testing them.
+The artifact will be built on a host only bot, a VM or physical bot without a device,
+and the test will run based on the artifact against a testbed with a device.
+
+Steps:
+
+1. Update the task class to extend [`BuildTestTask`](https://github.com/flutter/flutter/blob/master/dev/devicelab/lib/tasks/build_test_task.dart)
+   - Override function `getBuildArgs`
+   - Override function `getTestArgs`
+   - Override function `parseTaskResult`
+   - Override function `getApplicationBinaryPath`
+2. Update the `bin/tasks/{TEST}.dart` to point to the new task class
+3. Validate the task locally
+   - build only: `dart bin/test_runner.dart test -t {NAME_OR_PATH_OF_TEST} --task-args build --task-args application-binary-path={PATH_TO_ARTIFACT}`
+   - test only: `dart bin/test_runner.dart test -t {NAME_OR_PATH_OF_TEST} --task-args test --task-args application-binary-path={PATH_TO_ARTIFACT}`
+4. Add tasks to continuous integration
+   - Mirror a target with platform `Linux_build_test` or `Mac_build_test`
+   - The only difference from regular targets is the artifact property: if omitted, it will use the `task_name`.
+5. Once validated in CI, enable the target in `PROD` by removing `bringup: true` and deleting the old target entry without build+test model.
+
+Take gallery tasks for example:
+
+1. Linux android
+   - Separating PR: https://github.com/flutter/flutter/pull/103550
+   - Switching PR: https://github.com/flutter/flutter/pull/110533
+2. Mac iOS: https://github.com/flutter/flutter/pull/111164
\ No newline at end of file