* enabling firefox screenshot tests. adding to documentation
* test with correct goldens
* update goldens SHA
* change the goldens with flutter/goldens repo
* do not run preparation step twice. this will cause test_results to be deleted
* carrying code
* more changes for carrying the code
* rebase changes onto ios-screenshot tests
* adding screenshot capability to text_editing e2e test
* address some comments
* change enable flag for isUnitTestsScreenshotsAvailable
* addressing the reviewer comments
* change the dependency for path
* add to licencense file
* changing goldens commit no. the new commit has the screenshot goldens
* update readme file
* firefox tests needs LUCI changes
* change to release mode since screenshots were taken in release mode
* change window size
* some argument changes
* small comment change
* test the chrome linux tests again
* use roboto font instead of default font
* addressing reviewer comments
* change commit for goldens
* enable ios safari screenshot tests
* this is the value used for screenshots in the repo.
* change revision
* fix the error made in the gaps
* do not try to fetch on LUCI
* lunix luci migth break existing prs. only skip fetching for mac
* add a placeholder tests for shadow golden test
* try to use iPhone 11 Pro on LUCI
* change the scale and the smoke test image
* add unmerged goldens PR to tests all the screenshots. will be removed later
* change the goldens lock back to flutter/goldens
* change wrong comment on screenshot tests block
* address reviewer comments
* change the commit number for goldens file
* skip canvas blend mode tests
* debugging LUCI error
* debugging LUCI error printing directory contents
* skip one test, remove the debug logs
* change the revision number to include the correct chrome files
* run safari desktop tests on luci
* fix safari issue. focus on dom element when new transform is received. add transform to test cases
* Update text_editing.dart
minor change to retrigger tests (recipe change is merged)
* running screenshot tests on ios-safari unit tests
* fixing the golden_smoke tests. changes to the documentation
* addressing reviewer comments
* cropping footer from the simulator screenshot. addressing some reviewer comments
* use .dart_tools for recording the screenshots
* fix the usage of the method
* adding TODO's for missing documentation and not supported windows tests
* addressing comments
* changing to incremental counter for file names
* add comment to the counter
* fix anaylze issues
* using takescreenshot method from the iosSimulator.
* address reviewer comments
* fix the scaling issue. disable eronous test
* change the smoke file for top gap 282
* change the variable name for scale factor
* change from build_runner to dart2js
* add internalBootstrapBrowserTest to some of the tests
* add internalBootstrapBrowserTest to all remaining tests
* make tests build in paralel. Total time dropped from 586 to 177 seconds for 8 core MacBook
* change isolates with pool
* fixing analysis errors
* skipping canvaskit tests for ios-safari
* copy image files to the build directory
* adding internalBootstrapBrowserTest to newly added tests
* add internalBootstrapBrowserTest to faling path iterator test
* necessary changes to make chrome windows work
* in windows test in chrome instead of edge. our edge code was for legacy edge
* do not run golden unit tests on Windows LUCI bots for now
* addressing reviewer comments. Adding a method for deciding when to run integration tests.
* remove lines that I forgot to remove
* fixing analysis error. add issue for todo
* add bootstap to a test file
* adding bootstrap to another test
* add internalBootstrapBrowserTest to a golden test
* return test result in bat file. use archieve package to unzip
* fixing logs for chrome_installer
* use archieve and archieve entity instead of dynamic
* adding comments for windows platform archieve part
* addressing reviewer comments
* change readme file
* skip ios safari tests on felt level
* instead of skipping use the new web_installers repo
* re-skip the test. led run failed with another error
* remove led json
* running ios unit tests from felt by booting/shutingdown simulators. create simulator with felt
* changing the readme file for ios,mac,xcode spellings
* addressing reviewer comments
* fix analyse error
* fixing cleanup issue. solution: adding and await and using for loop instead of foreach
* try to run integration tests on luci
* adds extra logs to check if chrome is on lUCI
* use the cipd chrome package for integration tests in LUCI
* add driver version for chrome 83.(cipd package has 83 now)
* no headless mode didn't worked on the bots. try the other option
* On LUCI also use chrome driver as a cipd package
* change the directory name to fit the cpid package
* adding blacklist functionality
* remove logs added for troubleshooting. remove the check that blocks int-test runs on LUCI
* add more comments to blacklists
* also use CIPD package chrome for unit tests
* addressing reviewer comments
* run integration tests first, upon reviewer request
* fix bug. keep running integration tests after unit tests
* update the logs for LUCI
* fix todo comments
* changing felt script to fetch flutter
* changing the clone_flutter.sh code
* running integration tests with felt on cirrus. fetch framework in CI (not in local).
* only run cirrus tests on chrome. fix a comma in the flutter command path
* adding comments to public flags
* use local engine parameter for flutter pub get
* change flutter executable used for flutter drive command
* fix a cleanup issue. address comments. add toolException. enable web in flutter
* address reviwer comments. fix issue with local-engine
* address reviwer comments. fix issue with local-engine
* using engine/flutter/.dart_tools as clone directory. enabling clone script for local usage
* clean flutter repo with felt clean. add a flag to skip cloning the repo. always clone the repo even for local development, unless this flag is set
* fixing typos. updating readme for the new flag.
* fix directory error
* addressing reviewer comments
* adding an apple script to quit safari
* making changes to utils and adding cleanup after unit tests failed
* switch saai quit to pkill instead of applescript
* adding toolexception
* adding a cleanup function option to felt.
* removing unnecassary await's added for the previos version of the PR
* removing unnecassary async
* fix analyze error
* adding arguments to felt for running different type of tests
* adding test type enums to felt
* more changes to felt for testing branches
* more additions to code
* more changes to felt
* adding code for cloning web_drivers
* validating test directories. running the integration tests. update the readme file partially
* Removing todo lists used for development
* addressing reviewers comments
* remove unused imports
* addressing more reviewer comments
* addressing more reviewer comments
* addressing reviewer comments.
* addressing reviewer comments.
* fixing typos
* using chromedriverinstaller as a library. Fixing the commit number used from web_installers repo
* clean drivers directory after tests finish
* addressing more reviewer comments
* throwing all critical exceptions instead of logging and returning boolean flags
* adding todos for items we can do for making felt easier to use for local development
* do not run integration tests on CIs. Added an issue to the TODO.
* changing todo's with issues.
* Add Edge support for Flutter Engine unit-tests. This does not include the launcher
* Add mapping from browser name to platform.
* remove ie from supported browser maps
* changes to add firefox as one of the browser options to test_platform and test_runner
* Creating a supported_browsers file to put all the different browser related maps and utilities
* Remove accidentaly forgotten commented out lines. Clear imports
* fix error in screenshot handler
* Addressing PR comments.
* addressing PR comments part 2:
* Add Firefox installing functionality to test platform. For Linux only. Refactor test platform code
* remove download.dart. Not complete for now
* uncomment firefox.dart. Adding new CL parameters.
* Licence headers added.
* adding more comments to firefox_installer
* adding test for firefox download
* address pr comments. change directory for test in .cirrus.yml
* change directory for test_web_engine_firefox_script
* removing the system test.
1. Various functionalities offered by this tool are now organized into commands (e.g. `felt test`, `felt check-licenses`).
2. The felt tool can now be invoked from anywhere, not necessarily from the web_ui directory.
3. This new structure helps us scale better as we add more commands (e.g. soon a `build/watch` command is coming).
- Add a custom `PlatformPlugin` that spins up a server waiting for test to request a screenshot.
- When a screenshot is requested the plugin talks to Chrome via the debug port, captures a screenshot, and compares it with a golden file
- This PR also adds proper CLI for `dev/test.dart` with an `ArgParser` for future extension into a proper developer tool for the team.
- As a first couple of features, it adds `--debug` option to launch Chrome in debug mode, a `--target` option to choose a single test to run rather than all tests, and `--shard` option to choose a subset of tests to run.
Limitations:
- While screenshot test will run on Cirrus, they are configured to not fail on Cirrus. Need to solve Chrome version skew.
- Stack maps do not work yet (you get stacks, but they are nonsensical)
- When requesting a single test, build_runner builds all tests anyway
- Nothing but desktop Chrome is supported
- Nothing but Linux is supported
- There's no Chrome version pinning; currently assumed stable Chrome channel
