Context
-------
Prior to this PR we were unable to build our Swift source code as part of the Bazel toolchain because our source supports CocoaPods-style framework imports out of the box. E.g.:
```swift
import MaterialComponents.MaterialAppBar
```
Supporting CocoaPods imports enables us to use a pure open source workflow with our code that ensures we're using and validating the latest community technologies (including CocoaPods and native Xcode build systems). Changing our source to Bazel-first was explored as an option for this PR, but was rejected in service to maintaining our CocoaPods support.
The way we solved this import problem before this PR was to perform a global rewrite of import statements prior to invoking any `bazel` commands. This rewrite phase happened as part of the `.kokoro` script. This meant that the only way to run `bazel` commands on our repo was through the `.kokoro` bootstrap script.
This change does not yet entirely remove our need for the `.kokoro` bootstrap script, but it does remove the need for it to rewrite Swift imports. This change is one part of #5491.
The change
----------
This change introduces a new drop-in replacement for swift_library called `_mdc_cocoapods_compatible_swift_library` (for private use within the `material_components_ios.bzl` file) and `mdc_swift_library` for use in all BUILD files. `_mdc_cocoapods_compatible_swift_library` is made use of by several `mdc_` rules, including the new `mdc_swift_library` rule which should now be used instead of `swift_library` throughout our repo.
The new rule's implementation is documented in the source.
The impact
----------
Prior to this change, running the following command:
```
bazel build //components/Typography:SwiftExamples
```
Would result in the following error:
```
components/Typography/examples/TypographyFontListExample.swift:15:8: error: no such module 'MaterialComponents.MaterialTypography'
import MaterialComponents.MaterialTypography
```
After this change, running that same command will give the following successful result:
```
INFO: Invocation ID: 324c8887-d733-42ce-be06-20d9823bf718
INFO: Build options have changed, discarding analysis cache.
INFO: Analysed target //components/Typography:SwiftExamples (3 packages loaded, 365 targets configured).
INFO: Found 1 target...
Target //components/Typography:SwiftExamples up-to-date:
bazel-bin/components/Typography/libSwiftExamples.a
bazel-bin/components/Typography/components_Typography_SwiftExamples.swiftdoc
bazel-bin/components/Typography/components_Typography_SwiftExamples.swiftmodule
INFO: Elapsed time: 4.299s, Critical Path: 4.01s
INFO: 2 processes: 2 darwin-sandbox.
INFO: Build completed successfully, 3 total actions
```
Note that as a result of this change, build output for the transformed src will point to the generated source files rather than the original source files. E.g.:
```
ERROR: /Users/featherless/workbench/material-components-ios/components/Typography/BUILD:55:1: SwiftCompile components/Typography/components_Typography_SwiftExamples.swiftmodule failed (Exit 1) bazel_xcode_wrapper failed: error executing command bazel-out/host/bin/external/build_bazel_rules_swift/tools/wrappers/bazel_xcode_wrapper bazel-out/host/bin/external/build_bazel_rules_swift/tools/wrappers/swift_wrapper /usr/bin/xcrun swiftc ... (remaining 3 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
bazel-out/darwin-fastbuild/genfiles/components/Typography/examples/TypographyFontListExample.bazel_imports.swift:20:1: error: expressions are not allowed at the top level
typo
^
bazel-out/darwin-fastbuild/genfiles/components/Typography/examples/TypographyFontListExample.bazel_imports.swift:20:1: error: use of unresolved identifier 'typo'
typo
^~~~
```
This is the downside to this change, but it only materially impacts eng workflows if we start using Bazel to generate Xcode projects. As command line output, it's fairly straightforward to transform the path to the correct original source path. If we do someday move to a Bazel-generated Xcode project workflow, then we should also revisit our assumption that our source should support CocoaPods out of the box. As noted at the beginning of this PR, however, this consideration is deemed out of scope for this PR.
This file enables support for https://github.com/bazelbuild/bazelisk.
The `bazelisk` command is a drop-in replacement for the `bazel` command that automatically loads the correct `bazel` version (in this case 0.20.0) required by the workspace.
This PR accomplishes two things:
1. It defines which Bazel version our workspace supports in alignment with emerging Bazel community conventions encouraged by bazelisk.
2. It sets us up to remove explicit Bazel version management in our `.kokoro` script.
Closes https://github.com/material-components/material-components-ios/issues/9364
This reverts commit ca81e6e42826956e1912665a8e8077ab0d9705be.
The reverted commit does not actually fix the ancestry problem for a given release and the change was adding an unnecessary merge operation as a result. Simply branching off of `develop` is sufficient to create history from `stable` to `develop`.
In a follow-up change, I will address the actual root problem of `develop` not having an ancestral relations to the `stable` release commits.
Prior to this change, our release process was merging `develop` directly into `stable` without first branching off of `stable`.
The result of this process was that `stable` shares no common ancestor with `develop`, making it difficult to calculate differences between a commit on `develop` and a commit on `stable`.
One example of where this becomes problematic is in generating internal CLs for our PRs and for `develop`. It is not possible to compute the delta between the most recent `stable` commit that was mirrored internally and a given `develop` commit, making it difficult for tooling to apply deltas to the internal codebase without simply blowing away the internal directory and checking out the desired `develop` commit.
To fix this problem, we need to create a common ancestor between `develop` and `stable`. To do so, when cutting a release we need to cut the `release-candidate` from `stable` rather than `develop`, and then immediately merge `develop` into the release-candidate. This creates a common ancestor between `develop` and `stable`.
Note that this change causes us to deviate from the pure git flow model we have historically been following, as shown in the before/after of the model below (note the new red lines connecting stable to develop).
| Before | After |
|:--|:--|
|  |  |
The problem is that when you have a fresh clone (and you're not doing a hotfix) the "branch" variable in `scripts/release` points to `origin/develop`. There is no notion of a local stable branch. So commands like the one this PR changes need to specify a remote.
Closes#8585.
To keep .gitattributes correct in both develop and stable branches, 2 additions were added, one to the cut and one to the merge steps.
In cut we checkout the .gitattributes from stable before opening the PR to merge from release-candidate to stable, as release-candidate was created from Develop, and therefore without checking out the .gitattributes version from stable, when merged it would take the develop version.
In merge, we do a no-commit merge and then reset the .gitattributes so that develop branch won't take the release-candidate/stable version.
Closes#8392
Icons were not fully integrated into the catalog project and the relevant
scripts were out of date. This PR addressed the following issues:
* The Icons source files had been hand-modified and did not match the
script outputs.
* The Icons unit tests were not included in the Catalog or Dragons
projects.
It addresses these issues through the following:
* Updates `scripts/sync_icons.sh` to:
- Match the copyright header format used by the project.
- Run clang-format on each generated source file.
- Add Icons unit tests to the MaterialComponents podspec.
- Remove duplicate import of `MaterialIcons.h` in the source
implementation files.
* Adds the Icons unit test targets to the Catalog and Dragons Xcode schemes.
Discovered while working on #8498
Context is in the associated issue.
This affects the perl expression that prepends `docs/` to local urls. Specifically, it makes the perl expression only modify urls that link to a sibling markdown file composed of `\w` characters (e.g. `theming.md`). This is not perfect, but it addresses our needs for now. If/when we encounter other types of local urls that we can address them separately.
For now, this change fixes the incorrect behavior described in the associated bug.
Closes https://github.com/material-components/material-components-ios/issues/7203
Context is described in the associated issue.
The fix here is to force jazzy to only look at a given component when resolving headers. This means the generate_readme script will generate some import errors, so I've added a notice to the script indicating that these errors are likely ok.
Closes https://github.com/material-components/material-components-ios/issues/7195
We have had errors when generating the docs using jazzy specifically around components giving a fatal error around not finding imports when depending on other components. This is because each component for jazzy is isolated and on its own unless we provide the framework root. This caused our website generator to crash each time and fail to generate a new website.
This should resolve this problem.
QA=
Before the fix when running the website generator we got these errors:
```
/material-components-ios/components/BottomAppBar/src/MDCBottomAppBarView.h:17:9: fatal error: 'MaterialButtons.h' file not found
building site
building search index
^C/Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge/lexer.rb:458:in `load': Interrupt
from /Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge/lexer.rb:458:in `load_lexer'
from /Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge.rb:50:in `block in <top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge.rb:49:in `each'
from /Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge.rb:49:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/jazzy_markdown.rb:2:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/jazzy_markdown.rb:2:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/doc.rb:7:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/doc.rb:7:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/config.rb:5:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/config.rb:5:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy.rb:1:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy.rb:1:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/bin/jazzy:13:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/bin/jazzy:13:in `<top (required)>'
from /usr/local/bin/jazzy:22:in `load'
from /usr/local/bin/jazzy:22:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:74:in `load'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:74:in `kernel_load'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:28:in `run'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:463:in `exec'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:27:in `dispatch'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:18:in `start'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/exe/bundle:30:in `block in <top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/exe/bundle:22:in `<top (required)>'
from /usr/local/bin/bundle:22:in `load'
from /usr/local/bin/bundle:22:in `<main>'
/material-components-site-generator/scripts/lib/reporter.js:36
throw e;
^
Error: Command failed: bundle exec jazzy --output "/material-components-site-generator/.stage/ios/catalog/bottomnavigation/api-docs/" --theme "/material-components-site-generator/ios-api-docs-src/theme" --head '/components' --use-safe-filenames
at checkExecSyncError (child_process.js:601:13)
at execSync (child_process.js:641:13)
at JazzyApiGenerator.build (/material-components-site-generator/scripts/lib/jazzy-api-generator.js:34:5)
at PlatformSite.generateApiDocs (/material-components-site-generator/scripts/lib/platform-site.js:212:17)
at platformSites.forEach (/material-components-site-generator/scripts/build:86:14)
at Array.forEach (<anonymous>)
at reporter.step (/material-components-site-generator/scripts/build:85:21)
at Reporter.step (/material-components-site-generator/scripts/lib/reporter.js:32:22)
at Object.<anonymous> (/material-components-site-generator/scripts/build:84:14)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:191:16)
at bootstrap_node.js:612:3
```
Now we no longer get fatal errors or crash when running the script.
When looking at which files are included in bazel targets and which are
not, we should exclude any files that were deleted. If not, any PR that
deletes a file could potentially produce a false positive and get
blocked from submission.
Closes#6428
Instead of updating both RubyGems and Homebrew at the beginning of the kokoro
script, they should only be updated if they're going to be used for an install
or update command.
Closes#6256
The affected_targets script was looking at the entire log of changes within a
branch or history. Instead, it should only look at the differences between
HEAD and the diff base.
Since all of the `Snapshot` code is so far completely unbuildable on bazel
(because of the dependency on the snapshot library), changes to it should be
excluded from the bazel coverage job.
To make it clearer which `scripts` files are executable and which only provide
libraries of functions, the two library scripts are being moved to a separate
directory.
The job was broken due to a few things:
* It did not update bazel on the kokoro runner after install.
* It used the wrong bazel query to determine if a file was included in a BUILD package.
* It was not posting comments to the PR, forcing authors to hunt through the logs (which not all authors can access).
Commit eaa9d12554a2d22275124bd7e34c48fab148b0b4 (in this PR's history) originally included several files to test the behavior of the job. Since I cannot turn the job on for the `develop` branch until these changes are made (because it will fail with spurious errors), the testing took place in this branch independently.
Since posting comments can be helpful outside of the .kokoro script,
extracting the shared logic to a separate file will allow reuse of the
code and ensure fixes are applied to all uses.
In order to prevent us from having example code that cannot be compiled internally, we should detect any PRs that might have new code not covered by a bazel rule. Being included by a bazel rule will provide greater chance for the source to be compiled and perhaps even tested.
Supports #2641
Supports #5370
- The affected_targets script now accepts a range.
- The .kokoro script now determines the correct range of changes using the target branch.
- Changes to `*.bzl`, `WORKSPACE`, `.kokoro`, `affected_targets`, or `.gitattributes` will now result in a complete build.
E.g. this PR should result in a complete build because it is modifying the .kokoro file.
We can now test `scripts/affected_targets` with various commits like so:
```bash
> scripts/affected_targets test 026a2005e74fb75411295b8f44fd778eb2f87db0...8297e5dca167f72ee1b69d3acfec5507fccceaf7
//components/Tabs:unit_tests
//components/Tabs:unit_tests_IPAD_PRO_12_9_IN_9_3
//components/Tabs:unit_tests_IPAD_PRO_12_9_IN_9_3_test_bundle
//components/Tabs:unit_tests_IPHONE_5_IN_8_1
//components/Tabs:unit_tests_IPHONE_5_IN_8_1_test_bundle
//components/Tabs:unit_tests_IPHONE_7_PLUS_IN_10_3
//components/Tabs:unit_tests_IPHONE_7_PLUS_IN_10_3_test_bundle
//components/Tabs:unit_tests_IPHONE_X_IN_11_0
//components/Tabs:unit_tests_IPHONE_X_IN_11_0_test_bundle
```
This new bazel job will only build targets that are affected by the change.
Affected targets are determined via the scripts/affected_targets command.
This job has some rough edges that need to be smoothed out:
- Some changes should result in all targets being affected. E.g. the material_components_ios.bzl file.
- We're only picking up srcs and hdrs modifications right now. We will need to pick up other assets as well (bundles, assets, etc...).
This new job will be enabled, but not required, until we've ensured that its coverage is accurate.
The release script was continuing after errors failed to generate the
correct local clone of the repository for the API Diff step. This is
because the main release script suppresses non-zero return codes by
default. Instead, we should explicitly exit of any command returns an
unhandled non-zero value.
The `scripts/release apidiff` command needs to create two local temporary clones of the repo in order to generate the API diff. Unfortunately, git lfs fails to clone local copies of a git repo, particularly when not using a `file://` prefix.
The fix is to disable git LFS cloning prior to the API diff temporary clone commands.
```bash
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
Downloading snapshot_test_goldens/goldens_64/MDCCardSnapshotTests/testDefaultCard_11_2@2x.png (1.9 KB)
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
Error downloading object: snapshot_test_goldens/goldens_64/MDCCardSnapshotTests/testDefaultCard_11_2@2x.png (96d48d5): Smudge error: Error downloading snapshot_test_goldens/goldens_64/MDCCardSnapshotTests/testDefaultCard_11_2@2x.png (96d48d51f415dc94427e49a361cf68cdadc86d0e00a6706894a7f82b5d13b086): batch request: missing protocol: "file:///Users/featherless/workbench/material-components-ios/.git/info/lfs"
Errors logged to /private/tmp/mdcios/mdc-ios/.git/lfs/logs/20181218T112727.676554.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: snapshot_test_goldens/goldens_64/MDCCardSnapshotTests/testDefaultCard_11_2@2x.png: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'
```
This is an automated change generated by replacing all instances of MaterialComponentsAlpha with MaterialComponentsBeta. This is not a breaking change because changes to Alpha/Beta components (including renaming them) are not considered breaking.
The MaterialComponentsAlpha podspec was mistakenly named Alpha, when what we meant was more close to Beta. The distinction is that Alpha components are not expected to be used by clients, while Beta components are.
* Snapshot Testing Proof of Concept (#5754)
### The problem
We currently do not have UI tests on a component level. Integrating snapshot tests would allow us to have peace of mind with each PR that it isn't going to introduce any changes to the UI unless its intended to.
### The solution
* Integrate `ios-snapshot-test-case` pod to handle generating and diffing images of components.
* Integrate `git-lfs` to handle storage of the goldens.
This PR creates one test to showcase the ability to do snapshot tests. Upon merging this PR, you must install git-lfs in order to properly have the images pulled down. The 3 steps to do this:
1. `brew install git-lfs`
2. `git lfs install`
3. `git lfs pull`
Additionally, the golden is generated using an iOS 11 simulator at 2x scale so that the kokoro jobs are happy.
### Related bugs
Closes#5740
### Difference from #5754
**Note:** This is a re-revert of #5754 that aims to fix issues with Travis CI by ensuring the snapshot test only runs on a single iOS version. I've opened #5888 to expand on this in the future. A few things changed in the approach in this PR:
* We only run the snapshot test for iOS 11.2.0 until we can have an elegant solution for supporting multiple OS''s (see #5888)
* The snapshot tests now live in their own test target to avoid issues with having to require an App host for all tests.
* Additionally, a dummy swift file was required for compilation of this new test target (see https://forums.developer.apple.com/thread/88451 for context)
### The problem
We currently do not have UI tests on a component level. Integrating snapshot tests would allow us to have peace of mind with each PR that it isn't going to introduce any changes to the UI unless its intended to.
### The solution
* Integrate `ios-snapshot-test-case` pod to handle generating and diffing images of components.
* Integrate `git-lfs` to handle storage of the goldens.
This PR creates one test to showcase the ability to do snapshot tests. Upon merging this PR, you must install git-lfs in order to properly have the images pulled down. The 3 steps to do this:
1. `brew install git-lfs`
2. `git lfs install`
3. `git lfs pull`
Additionally, the golden is generated using an iOS 11 simulator at 2x scale so that the kokoro jobs are happy.
### Related bugs
Closes#5740
### Context
When determining whether a change affects multiple components we want to ignore changes to CocoaPods files that live outside of the components directories.
### The problem + fix
Prior to this change, if a commit modified MaterialComponentsAlpha.podspec or catalog/Podfile it was being picked up as a multi-component change in the release notes.
After this change, commits affecting those files will not be treated as multi-component changes unless they also touch other files in the repo.
### Opportunities for further improvement
It's likely fair to say that any modification to files outside of the `components/` directory can be ignored. I'm not sure what the best way to filter these types of changes is, but that would be a welcome follow-up simplification of this change that would require less on-going maintenance.
The script now outputs four groups of changelogs:
1. Breaking changes for specific components.
2. Breaking changes across multiple components.
3. Changes for specific components.
4. Changes acrosss multiple components.
This will make it easier for us to keep all google-owned repositories running the same clang-format version in tandem.
Also moved from a submodule to a clone + version checkout so that our CI gracefully upgrades as we release new non-breaking release to clang-format-ci.
Relies on upstream changes in https://github.com/material-foundation/clang-format-ci/pull/4
{MDC iOS} No longer using `-init` for Color Scheme.
Based on the discussions in go/mdc-ios-theming, we should not use the `-init` default initializer. Instead, we should use an explicit set of defaults.
Search regex: '\[\[MDCSemanticColorScheme alloc\] init\]'
