12 Commits

Author SHA1 Message Date
Michael Klimushyn
1e1eb4d34a
Minor update to the Robolectric test harness (#11067)
Make sure that any `android.util.Log`s during the tests are reported to
stdout.
2019-08-16 16:25:59 -07:00
liyuqian
a7c965caec
Hide verbose dart snapshot during run_test.py (#11040)
Without this change, a single dart test will print out 306 lines.
With this change, a single dart test will only print out 33 lines.
This helps a lot in local tests.

In order to return to the old logging, set `--verbose-dart-snapshot` flag.
2019-08-15 16:40:30 -07:00
Michael Klimushyn
c92a0d992a
Report JUnit test failures (#10941)
Previously this script was not reporting any JUNit failures and somehow
ended up included a non-existent jar for Java tests to test against.

It looks like one of the JUnit tests is now failing. Disabling it for
now to turn on CI again as soon as possible, will fix and enable it in a
follow up.
2019-08-12 12:17:15 -07:00
Dan Field
33c53c876b
Specify which android variant for tests (#10717) 2019-08-08 16:16:51 -07:00
liyuqian
620b8998f8
Revert "Remove semi-redundant try-jobs. (#10485)" (#10705)
This reverts commit 16c0058f17403b219a596de993ece5dd7f768b3d.

Our infra should be capable to run all those deleted tests as now we've throttled the Fuchsia auto-rollers.

The current throttle is 1 PR / 3 hours for the 4 rollers:
- https://cs.corp.google.com/skia/buildbot/autoroll/config/fuchsia-mac-toolchain-flutter-engine.json?l=53
- https://cs.corp.google.com/skia/buildbot/autoroll/config/fuchsia-mac-sdk-flutter-engine.json?g=0&l=53
- https://cs.corp.google.com/skia/buildbot/autoroll/config/fuchsia-linux-toolchain-flutter-engine.json?g=0&l=56
- https://cs.corp.google.com/skia/buildbot/autoroll/config/fuchsia-linux-sdk-flutter-engine.json?g=0&l=56

We're changing them to 1 PR / 12 hours: http://skbug.com/9322
2019-08-07 16:21:41 -07:00
Dan Field
4c406ba976
Test perf overlay gold on Linux (#10703) 2019-08-07 13:47:00 -07:00
Shi-Hao Hong
797f032b7b
Rolls engine to Android SDK 29 and its corresponding tools (#10692)
* Rolls engine to Android SDK 29 and its corresponding tools

* Update from android-28 to android-29 in relevant files

* Roll buildroot to tip of tree
2019-08-07 12:34:33 -04:00
Chinmay Garde
16c0058f17
Remove semi-redundant try-jobs. (#10485)
We are severely resource constrained on the tryjobs. This is hindering progress on ToT due to scheduling failures and rate limits. There are no plans to requisition more resources in this infra scheme as the migration to tryjobs on LUCI is the preferred way forward. This patch removes presubmits that are somewhat redundant. Each removed presubmit and the justification is given below:

* `build_and_test_linux_opt_release`: The AOT variant is tested by `build_and_test_linux_opt_profile` (actually its replacement, see the next point).
* `build_and_test_linux_opt_profile`: When we build the host profile variant for Android, we test it as well instead of building the extra variant (since we already need a host for a target build).
* `build_and_test_android_unopt_debug`: `gen_javadoc` have been moved to the profile variant. `Roboelectric` tests have been moved to profile variant. JIT infra will be tested by the the host tests.
* ToT framework checks: These will be performed (by the autoroller)[https://autoroll.skia.org/r/flutter-engine-flutter-autoroll?tab=status] which also has a sheriff rotation. This will also avoid us having to ignore red presubmits on manual changes.

Everything else is left as-is. To be clear, this causes strictly less checks to be performed on each commit.
2019-08-06 16:16:06 -07:00
Michael Klimushyn
8ed5da8b65
Add working Robolectric tests (#9954)
`gclient sync` now grabs Robolectric, JUnit, and their transitive
runtime dependencies. They're being stored in a new CIPD package,
`flutter/android/robolectric_bundle`.

`shell/platform/android/BUILD.gn` has a new target for building the
tests, `robolectric_tests`. `testing/run_tests.py` has been extended to
build and run the new target. Runs the android tests under
"build_and_test_android" on CI.

This also adds some very simple sample tests to start with and a README
to the java tests directory.
2019-07-23 09:06:19 -07:00
Jason Simmons
da0fcf8284
Updates to the engine test runner script (#9934)
* Use separate filters for engine executables and Dart test scripts
* Do not run the benchmarks on Cirrus
2019-07-19 17:00:18 -07:00
Chinmay Garde
9eed783e94
Selectively enable tests that work on Windows and file issues for ones that don't. (#9852)
This is in preparation for the  tryjobs to run these tests. The LUCI harness will also be updated so that the tests to run are specified in the repo instead of the recipe.
2019-07-16 13:30:57 -07:00
Chinmay Garde
a59b6a6e38
Convert run_tests to python, allow running on Mac/Windows and allow filters for tests. (#9818)
Sample usage:

To run only the embedder_unittests in the engine with the profile variant, the command would be
```
./flutter/testing/run_tests.py --variant host_profile_unopt --type engine --filter embedder_unittests
``

To run only the geometry in Dart with the debug variant, the command would be
```
./flutter/testing/run_tests.py --variant host_debug_unopt --type dart --filter geometry_test
``

Without any argument, the behavior is identical to `run_tests.sh`.

In a subsequent patch, I will enable running unit-tests on Windows in the tryjobs. The lack of compatibility of the shell script on Windows made it so that we never ran any Windows unit-tests in the tryjobs.
2019-07-15 17:46:36 -07:00