13 Commits

Author SHA1 Message Date
rami-a
615a0e4061
Snapshot Testing Proof of Concept (#5887)
* Snapshot Testing Proof of Concept (#5754)

### The problem
We currently do not have UI tests on a component level. Integrating snapshot tests would allow us to have peace of mind with each PR that it isn't going to introduce any changes to the UI unless its intended to.

### The solution
* Integrate `ios-snapshot-test-case` pod to handle generating and diffing images of components. 
* Integrate `git-lfs` to handle storage of the goldens.

This PR creates one test to showcase the ability to do snapshot tests. Upon merging this PR, you must install git-lfs in order to properly have the images pulled down. The 3 steps to do this:
1. `brew install git-lfs`
2. `git lfs install`
3. `git lfs pull`

Additionally, the golden is generated using an iOS 11 simulator at 2x scale so that the kokoro jobs are happy.

### Related bugs
Closes #5740

### Difference from #5754 
**Note:** This is a re-revert of #5754 that aims to fix issues with Travis CI by ensuring the snapshot test only runs on a single iOS version. I've opened #5888 to expand on this in the future. A few things changed in the approach in this PR:
 * We only run the snapshot test for iOS 11.2.0 until we can have an elegant solution for supporting multiple OS''s (see #5888)
 * The snapshot tests now live in their own test target to avoid issues with having to require an App host for all tests.
* Additionally, a dummy swift file was required for compilation of this new test target (see https://forums.developer.apple.com/thread/88451 for context)
2018-12-05 09:49:41 -05:00
Robert Moore
13781e6d45
[Catalog] Revert "Snapshot Testing Proof of Concept" (#5883)
Reverts material-components/material-components-ios#5754

Travis CI on `develop` branch testing has failed since snapshot testing was introduced. Rolling it back.

https://travis-ci.org/material-components/material-components-ios/builds
2018-11-30 09:20:03 -05:00
rami-a
c6a300ff2c
Snapshot Testing Proof of Concept (#5754)
### The problem
We currently do not have UI tests on a component level. Integrating snapshot tests would allow us to have peace of mind with each PR that it isn't going to introduce any changes to the UI unless its intended to.

### The solution
* Integrate `ios-snapshot-test-case` pod to handle generating and diffing images of components. 
* Integrate `git-lfs` to handle storage of the goldens.

This PR creates one test to showcase the ability to do snapshot tests. Upon merging this PR, you must install git-lfs in order to properly have the images pulled down. The 3 steps to do this:
1. `brew install git-lfs`
2. `git lfs install`
3. `git lfs pull`

Additionally, the golden is generated using an iOS 11 simulator at 2x scale so that the kokoro jobs are happy.

### Related bugs
Closes #5740
2018-11-29 15:56:06 -05:00
featherless
531b372815
[CI] Fix the test_all script. (#5538)
## Prior to this change

Our test_all script was using an Xcode schemes search and filter to identify any test schemes that exist in the repository and test them. Unfortunately, the script was not also testing to confirm that it found at least *some* tests, meaning if it did not find any tests it would silently succeed.

The script was essentially optimized for us adding multiple test schemes in the future. In practice, the frequency with which we add new test schemes is low (we've added only one in the past year via MaterialComponentsAlpha).

## The root cause

With d238c86d47eb072617d285106147ace613321aee, our Xcode test scheme names changed and our scheme search was no longer returning our unit test schemes, resulting in test_all succeeding even though it hadn't run any tests.

## After this change

The test_all script will explicitly test specific schemes. If these schemes change or move in the future, this script will fail until we also update the script's schemes.

This script now also pipes the test output through xcpretty if it is available.

Closes https://github.com/material-components/material-components-ios/issues/5539
2018-10-26 15:29:01 -04:00
Will Larche
cf6da2e8c9 [Testing] Travis testing devices updates. (#1595)
* [Testing] Updating test_all for multiple devices.

* [Testing] Setting Travis device matrix to use a 32-bit device.

* [Testing] Comments in Travis file.

* [Testing] Bumping Travis testing 'latest' version to 10.3.

* [Testing] Correcting Travis device target.
2017-07-05 14:40:45 -04:00
Adrian Secord
508deb4262 Modified scripts/test_all to use latest simulator available. (#1559) 2017-06-29 13:33:23 -04:00
Adrian Secord
8ffd18be47 [Testing] Remove interaction tests from Travis CI (#1190)
* Add ability to specify test to run on the command line.

* Run only unit tests.
2017-02-13 16:36:30 -05:00
Adrian Secord
f6f8480d4f Added comment about Travis CI's 10.1 device bug. (#1135) 2017-01-05 10:19:33 -05:00
Adrian Secord
5220eb3390 Enable test code coverage reports in Xcode and in Travis CI. (#1115) 2016-12-27 18:47:14 -05:00
ianegordon
82dc874d6d Enable Travis CI integration (#1095)
* Add file to kick off Travis

* Checking Travis CI's installed Xcode and simulator devices.

* Adding Cocoapods init to Travis CI config.

* Enabling building in Travis CI config.

* Added temporary dumping of the log file on error.

* Switched Shrine to manual signing.

* Dump Travis code signing options.

* Adding CocoaPods version to debug output.

* Remove code signing reporting from Travis CI config.

* Forcing `build_all` to skip code signing.

* Switching Shrine back to automatic provisioning style.

* Re-enabling testing in Travis CI.

* Bump up verbosity of testing.

* Changing test device from 10.2 to 10.1 to work around Travis CI bug.

* Remove the printing of available devices, Xcode will print them out on error.

* Added more context on failure for Travis CI builds only.

* Re-enable CI for our normal branches.

* Remove "do not merge" file.

* Removing our development team and switching Catalog to automatic profiles.

* Removing our development team and switching Bare to automatic profiles.

* Removing our development team and switching Pesto to automatic profiles.
2016-12-27 17:46:34 -05:00
Adrian Secord
a11d01a092 Made build/test scripts exit with failure if anything failed. (#1088) 2016-12-21 17:07:56 -05:00
Adrian Secord
ab2037afc6 Update scripts/test_all to run all test targets (#1004)
* WIP

* WIP

* Simplified build script and fixed it to build all schemes.

* WIP

* Renamed script to build_all and added better output.

* Add scripts/list_xcode_workspaces.

* Filtered out the CocoaPods schemes.

* Added Xcode workspace helpers.

Switched build_all to a whitelist approach instead of a blacklist.

* Refactored scripts/test_all to find all test schemes.

* Fixed test_all log filename.

* Add comment to scripts/test_all.
2016-12-12 11:55:10 -05:00
Adrian Secord
989caa912e Added a "test_all" script. 2016-12-08 08:32:25 -05:00