4 Commits

Author SHA1 Message Date
Kate Lovett
616f9bcffa
Add tag support for executing reduced test sets (#86821) 2021-08-30 15:41:02 -07:00
Greg Spencer
738ce43d97
Randomize tests, exclude tests that fail with randomization. (#86793)
* Randomize tests, exclude tests that fail with randomization.

* Disable some more tool tests
2021-07-26 23:40:49 -07:00
Greg Spencer
6adea05276
Revert "Randomize Framework tests, opt out some tests that currently fail. (#85159)" (#85673)
This reverts commit b5f9612 because it is taking about 40% longer to run the tests, which is causing timeouts.
2021-06-30 19:38:09 -07:00
Greg Spencer
b5f9612cac
Randomize Framework tests, opt out some tests that currently fail. (#85159)
This turns on order shuffling for all tests that don't fail with it on, marking those tests that do fail with a tag so that they will be run without shuffling on.

To determine which tests fail with it on, I ran all the tests 100 times with different random shuffle seeds, and then also ran it with the date seeds from today until the end of July, and tagged all of the test suites (files) that fail, with a seed that caused them to fail.
2021-06-29 13:46:13 -07:00