* carrying code
* more changes for carrying the code
* rebase changes onto ios-screenshot tests
* adding screenshot capability to text_editing e2e test
* address some comments
* change enable flag for isUnitTestsScreenshotsAvailable
* addressing the reviewer comments
* change the dependency for path
* add to licencense file
* changing goldens commit no. the new commit has the screenshot goldens
* update readme file
* firefox tests needs LUCI changes
* change to release mode since screenshots were taken in release mode
* change window size
* some argument changes
* small comment change
* test the chrome linux tests again
* use roboto font instead of default font
* addressing reviewer comments
* change commit for goldens
* enable ios safari screenshot tests
* this is the value used for screenshots in the repo.
* change revision
* fix the error made in the gaps
* do not try to fetch on LUCI
* lunix luci migth break existing prs. only skip fetching for mac
* add a placeholder tests for shadow golden test
* try to use iPhone 11 Pro on LUCI
* change the scale and the smoke test image
* add unmerged goldens PR to tests all the screenshots. will be removed later
* change the goldens lock back to flutter/goldens
* change wrong comment on screenshot tests block
* address reviewer comments
* change the commit number for goldens file
* skip canvas blend mode tests
* debugging LUCI error
* debugging LUCI error printing directory contents
* skip one test, remove the debug logs
* change the revision number to include the correct chrome files
* running screenshot tests on ios-safari unit tests
* fixing the golden_smoke tests. changes to the documentation
* addressing reviewer comments
* cropping footer from the simulator screenshot. addressing some reviewer comments
* use .dart_tools for recording the screenshots
* fix the usage of the method
* adding TODO's for missing documentation and not supported windows tests
* addressing comments
* changing to incremental counter for file names
* add comment to the counter
* fix anaylze issues
* using takescreenshot method from the iosSimulator.
* address reviewer comments
* fix the scaling issue. disable eronous test
* change the smoke file for top gap 282
* change the variable name for scale factor
* change from build_runner to dart2js
* add internalBootstrapBrowserTest to some of the tests
* add internalBootstrapBrowserTest to all remaining tests
* make tests build in paralel. Total time dropped from 586 to 177 seconds for 8 core MacBook
* change isolates with pool
* fixing analysis errors
* skipping canvaskit tests for ios-safari
* copy image files to the build directory
* adding internalBootstrapBrowserTest to newly added tests
* add internalBootstrapBrowserTest to faling path iterator test
* necessary changes to make chrome windows work
* in windows test in chrome instead of edge. our edge code was for legacy edge
* do not run golden unit tests on Windows LUCI bots for now
* addressing reviewer comments. Adding a method for deciding when to run integration tests.
* remove lines that I forgot to remove
* fixing analysis error. add issue for todo
* add bootstap to a test file
* adding bootstrap to another test
* add internalBootstrapBrowserTest to a golden test
* return test result in bat file. use archieve package to unzip
* fixing logs for chrome_installer
* use archieve and archieve entity instead of dynamic
* adding comments for windows platform archieve part
* addressing reviewer comments
* change readme file
* runs ios unit tests, if the simulator is already booted.
* address reviewer comments
* changing comments for the Safari desktop tests
* addressing reviewer comments. removing the timeout.
* change mobile browsers ver name for safari.dart
* removing then calback on browser exit, only leaving the exception callback
* Reland "add limited analysis options (#17332)"
This time I'm starting without Cirrus. Will add Cirrus serpartely from code changes.
This reverts commit 6d33ee1a2ca7d404ebd5cb434d8ab859bf8ac0b7.
* disable Cirrus analysis check
* changes to add firefox as one of the browser options to test_platform and test_runner
* Creating a supported_browsers file to put all the different browser related maps and utilities
* Remove accidentaly forgotten commented out lines. Clear imports
* fix error in screenshot handler
* Addressing PR comments.
* addressing PR comments part 2:
* Add Firefox installing functionality to test platform. For Linux only. Refactor test platform code
* remove download.dart. Not complete for now
* uncomment firefox.dart. Adding new CL parameters.
* Licence headers added.
* adding more comments to firefox_installer
* adding test for firefox download
* address pr comments. change directory for test in .cirrus.yml
* change directory for test_web_engine_firefox_script
* removing the system test.
* Import all golden files and tests from internal repo.
* Adapt test files to new screenshot API, and tweak some settings.
(Check PR to see individual changes to each file)
Fixes https://github.com/flutter/flutter/issues/40975
New features for golden tests (for web):
* Height/width of headless browser instance extracted to constants
* Remove border from the host iframe
* Added 'region' to matchGoldenFile, so screenshots can capture only
a subset of the viewport
* Added image-format-awareness (png) to screenshot comparison, so it
doesn't only look at raw bytes
* When a test fails, output a diff image alongside the unexpected
output.
1. Various functionalities offered by this tool are now organized into commands (e.g. `felt test`, `felt check-licenses`).
2. The felt tool can now be invoked from anywhere, not necessarily from the web_ui directory.
3. This new structure helps us scale better as we add more commands (e.g. soon a `build/watch` command is coming).
- Add a custom `PlatformPlugin` that spins up a server waiting for test to request a screenshot.
- When a screenshot is requested the plugin talks to Chrome via the debug port, captures a screenshot, and compares it with a golden file
- This PR also adds proper CLI for `dev/test.dart` with an `ArgParser` for future extension into a proper developer tool for the team.
- As a first couple of features, it adds `--debug` option to launch Chrome in debug mode, a `--target` option to choose a single test to run rather than all tests, and `--shard` option to choose a subset of tests to run.
Limitations:
- While screenshot test will run on Cirrus, they are configured to not fail on Cirrus. Need to solve Chrome version skew.
- Stack maps do not work yet (you get stacks, but they are nonsensical)
- When requesting a single test, build_runner builds all tests anyway
- Nothing but desktop Chrome is supported
- Nothing but Linux is supported
- There's no Chrome version pinning; currently assumed stable Chrome channel
