The JSON codec is awkward to use in the wrapper (since the client has to build and link one of the JSON libraries to do so). Since it would be very cumbersome to wrap in a C API, and there's essentially no reason to use it instead of the standard codec, this removes it from the wrapper entirely.
Since some system channels (internal to the engine) still use it, it's moved into common/cpp instead of being eliminated entirely. Internally we always use RapidJSON though, so the jsoncpp implementation is removed. Also adds some unit test coverage, since there wasn't any.
Fixes#30669
Currently every Linux runner has this code to allow relative resource paths; this moves it into the framework so that any embedder can get this behavior without that code needing to be in the template.
Rolls buildroot to pick up std::filesystem support in our libc++
* The outputs of all commands are not printed and not just commands that fail.
* The stdout and stderr are now printed in order.
* Clear dividers mark logs from specific subprocesses or errors.
* The test whose run failed should now be exactly on top of the error message and code.
`//flutter/testing` now contains a lot of utilities used by other test targets.
This includes stuff like working with render targets that use either OpenGL or
Metal, fixtures for interacting with the Dart VM, test assertion predicates,
etc.. However, these utilities themselves are not tested as part of a standalone
test suite. Instead, only the test targets that include it exercise these
utilities. Since these are no longer trivial, a new test target has been added
that tests the testing utilities directly.
Make flutter_tester support multithreaded testing, and run all Dart tests in both single and multithreaded configurations
This also modifies Shell::GetUIIsolateLastError() and Shell::EngineHasLivePorts() so that they must be called from the UI task runner.
The tests we write must be resilient to the order in which they are run in the
harness. That is, they must not rely on global state set by other tests that
have already run in the process. Also, these tests must themselves be
repeatable. That is, they must correctly clean up after themselves and be able
to run successfully again in the same process.
This patch adds some safeguards against (but does NOT guarantee) the addition of
tests that violate the dictum.
Additionally, test failures must be easily reproducible for folks investigating
the test failure. Also, tests that assert correctness of unrelated code must not
stop progress on the authors patch.
This changes does not hinder reproducibility of test failures because the random
seed is printed in the logs before running each test. Developers attempting to
reproduce the failure locally can do the same via the following invocation
`--gtest_shuffle --gtest_repeat=<the count> --gtest_random_seed=<seed from failing run>`.
This change does introduce potential burden on patch authors that may see
failures in unrelated code as a newly failing shuffle seed is used on their
runs. To ameliorate this, we will formulate guidance for them to aggressively
mark such tests as disabled and file bugs to enable the same.
The test seed is intentionally kept low because it’s purpose is to test that
individual tests are repeatable. It must not be used as a replacement for
fuzzing.
Without this change, a single dart test will print out 306 lines.
With this change, a single dart test will only print out 33 lines.
This helps a lot in local tests.
In order to return to the old logging, set `--verbose-dart-snapshot` flag.
Previously this script was not reporting any JUNit failures and somehow
ended up included a non-existent jar for Java tests to test against.
It looks like one of the JUnit tests is now failing. Disabling it for
now to turn on CI again as soon as possible, will fix and enable it in a
follow up.
We are severely resource constrained on the tryjobs. This is hindering progress on ToT due to scheduling failures and rate limits. There are no plans to requisition more resources in this infra scheme as the migration to tryjobs on LUCI is the preferred way forward. This patch removes presubmits that are somewhat redundant. Each removed presubmit and the justification is given below:
* `build_and_test_linux_opt_release`: The AOT variant is tested by `build_and_test_linux_opt_profile` (actually its replacement, see the next point).
* `build_and_test_linux_opt_profile`: When we build the host profile variant for Android, we test it as well instead of building the extra variant (since we already need a host for a target build).
* `build_and_test_android_unopt_debug`: `gen_javadoc` have been moved to the profile variant. `Roboelectric` tests have been moved to profile variant. JIT infra will be tested by the the host tests.
* ToT framework checks: These will be performed (by the autoroller)[https://autoroll.skia.org/r/flutter-engine-flutter-autoroll?tab=status] which also has a sheriff rotation. This will also avoid us having to ignore red presubmits on manual changes.
Everything else is left as-is. To be clear, this causes strictly less checks to be performed on each commit.
`gclient sync` now grabs Robolectric, JUnit, and their transitive
runtime dependencies. They're being stored in a new CIPD package,
`flutter/android/robolectric_bundle`.
`shell/platform/android/BUILD.gn` has a new target for building the
tests, `robolectric_tests`. `testing/run_tests.py` has been extended to
build and run the new target. Runs the android tests under
"build_and_test_android" on CI.
This also adds some very simple sample tests to start with and a README
to the java tests directory.
This is in preparation for the tryjobs to run these tests. The LUCI harness will also be updated so that the tests to run are specified in the repo instead of the recipe.
Sample usage:
To run only the embedder_unittests in the engine with the profile variant, the command would be
```
./flutter/testing/run_tests.py --variant host_profile_unopt --type engine --filter embedder_unittests
``
To run only the geometry in Dart with the debug variant, the command would be
```
./flutter/testing/run_tests.py --variant host_debug_unopt --type dart --filter geometry_test
``
Without any argument, the behavior is identical to `run_tests.sh`.
In a subsequent patch, I will enable running unit-tests on Windows in the tryjobs. The lack of compatibility of the shell script on Windows made it so that we never ran any Windows unit-tests in the tryjobs.
