Context is in the associated issue.
This affects the perl expression that prepends `docs/` to local urls. Specifically, it makes the perl expression only modify urls that link to a sibling markdown file composed of `\w` characters (e.g. `theming.md`). This is not perfect, but it addresses our needs for now. If/when we encounter other types of local urls that we can address them separately.
For now, this change fixes the incorrect behavior described in the associated bug.
Closes https://github.com/material-components/material-components-ios/issues/7203
Context is described in the associated issue.
The fix here is to force jazzy to only look at a given component when resolving headers. This means the generate_readme script will generate some import errors, so I've added a notice to the script indicating that these errors are likely ok.
Closes https://github.com/material-components/material-components-ios/issues/7195
We have had errors when generating the docs using jazzy specifically around components giving a fatal error around not finding imports when depending on other components. This is because each component for jazzy is isolated and on its own unless we provide the framework root. This caused our website generator to crash each time and fail to generate a new website.
This should resolve this problem.
QA=
Before the fix when running the website generator we got these errors:
```
/material-components-ios/components/BottomAppBar/src/MDCBottomAppBarView.h:17:9: fatal error: 'MaterialButtons.h' file not found
building site
building search index
^C/Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge/lexer.rb:458:in `load': Interrupt
from /Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge/lexer.rb:458:in `load_lexer'
from /Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge.rb:50:in `block in <top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge.rb:49:in `each'
from /Library/Ruby/Gems/2.3.0/gems/rouge-3.1.1/lib/rouge.rb:49:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/jazzy_markdown.rb:2:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/jazzy_markdown.rb:2:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/doc.rb:7:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/doc.rb:7:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/config.rb:5:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy/config.rb:5:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy.rb:1:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/lib/jazzy.rb:1:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/bin/jazzy:13:in `require'
from /Library/Ruby/Gems/2.3.0/gems/jazzy-0.9.3/bin/jazzy:13:in `<top (required)>'
from /usr/local/bin/jazzy:22:in `load'
from /usr/local/bin/jazzy:22:in `<top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:74:in `load'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:74:in `kernel_load'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli/exec.rb:28:in `run'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:463:in `exec'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:27:in `dispatch'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/cli.rb:18:in `start'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/exe/bundle:30:in `block in <top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
from /Library/Ruby/Gems/2.3.0/gems/bundler-2.0.1/exe/bundle:22:in `<top (required)>'
from /usr/local/bin/bundle:22:in `load'
from /usr/local/bin/bundle:22:in `<main>'
/material-components-site-generator/scripts/lib/reporter.js:36
throw e;
^
Error: Command failed: bundle exec jazzy --output "/material-components-site-generator/.stage/ios/catalog/bottomnavigation/api-docs/" --theme "/material-components-site-generator/ios-api-docs-src/theme" --head '/components' --use-safe-filenames
at checkExecSyncError (child_process.js:601:13)
at execSync (child_process.js:641:13)
at JazzyApiGenerator.build (/material-components-site-generator/scripts/lib/jazzy-api-generator.js:34:5)
at PlatformSite.generateApiDocs (/material-components-site-generator/scripts/lib/platform-site.js:212:17)
at platformSites.forEach (/material-components-site-generator/scripts/build:86:14)
at Array.forEach (<anonymous>)
at reporter.step (/material-components-site-generator/scripts/build:85:21)
at Reporter.step (/material-components-site-generator/scripts/lib/reporter.js:32:22)
at Object.<anonymous> (/material-components-site-generator/scripts/build:84:14)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:191:16)
at bootstrap_node.js:612:3
```
Now we no longer get fatal errors or crash when running the script.
When looking at which files are included in bazel targets and which are
not, we should exclude any files that were deleted. If not, any PR that
deletes a file could potentially produce a false positive and get
blocked from submission.
Closes#6428
Instead of updating both RubyGems and Homebrew at the beginning of the kokoro
script, they should only be updated if they're going to be used for an install
or update command.
Closes#6256
The affected_targets script was looking at the entire log of changes within a
branch or history. Instead, it should only look at the differences between
HEAD and the diff base.
Since all of the `Snapshot` code is so far completely unbuildable on bazel
(because of the dependency on the snapshot library), changes to it should be
excluded from the bazel coverage job.
To make it clearer which `scripts` files are executable and which only provide
libraries of functions, the two library scripts are being moved to a separate
directory.
The job was broken due to a few things:
* It did not update bazel on the kokoro runner after install.
* It used the wrong bazel query to determine if a file was included in a BUILD package.
* It was not posting comments to the PR, forcing authors to hunt through the logs (which not all authors can access).
Commit eaa9d12554a2d22275124bd7e34c48fab148b0b4 (in this PR's history) originally included several files to test the behavior of the job. Since I cannot turn the job on for the `develop` branch until these changes are made (because it will fail with spurious errors), the testing took place in this branch independently.
Since posting comments can be helpful outside of the .kokoro script,
extracting the shared logic to a separate file will allow reuse of the
code and ensure fixes are applied to all uses.
In order to prevent us from having example code that cannot be compiled internally, we should detect any PRs that might have new code not covered by a bazel rule. Being included by a bazel rule will provide greater chance for the source to be compiled and perhaps even tested.
Supports #2641
Supports #5370
- The affected_targets script now accepts a range.
- The .kokoro script now determines the correct range of changes using the target branch.
- Changes to `*.bzl`, `WORKSPACE`, `.kokoro`, `affected_targets`, or `.gitattributes` will now result in a complete build.
E.g. this PR should result in a complete build because it is modifying the .kokoro file.
We can now test `scripts/affected_targets` with various commits like so:
```bash
> scripts/affected_targets test 026a2005e74fb75411295b8f44fd778eb2f87db0...8297e5dca167f72ee1b69d3acfec5507fccceaf7
//components/Tabs:unit_tests
//components/Tabs:unit_tests_IPAD_PRO_12_9_IN_9_3
//components/Tabs:unit_tests_IPAD_PRO_12_9_IN_9_3_test_bundle
//components/Tabs:unit_tests_IPHONE_5_IN_8_1
//components/Tabs:unit_tests_IPHONE_5_IN_8_1_test_bundle
//components/Tabs:unit_tests_IPHONE_7_PLUS_IN_10_3
//components/Tabs:unit_tests_IPHONE_7_PLUS_IN_10_3_test_bundle
//components/Tabs:unit_tests_IPHONE_X_IN_11_0
//components/Tabs:unit_tests_IPHONE_X_IN_11_0_test_bundle
```
This new bazel job will only build targets that are affected by the change.
Affected targets are determined via the scripts/affected_targets command.
This job has some rough edges that need to be smoothed out:
- Some changes should result in all targets being affected. E.g. the material_components_ios.bzl file.
- We're only picking up srcs and hdrs modifications right now. We will need to pick up other assets as well (bundles, assets, etc...).
This new job will be enabled, but not required, until we've ensured that its coverage is accurate.
The release script was continuing after errors failed to generate the
correct local clone of the repository for the API Diff step. This is
because the main release script suppresses non-zero return codes by
default. Instead, we should explicitly exit of any command returns an
unhandled non-zero value.
The `scripts/release apidiff` command needs to create two local temporary clones of the repo in order to generate the API diff. Unfortunately, git lfs fails to clone local copies of a git repo, particularly when not using a `file://` prefix.
The fix is to disable git LFS cloning prior to the API diff temporary clone commands.
```bash
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
Downloading snapshot_test_goldens/goldens_64/MDCCardSnapshotTests/testDefaultCard_11_2@2x.png (1.9 KB)
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent. See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
Error downloading object: snapshot_test_goldens/goldens_64/MDCCardSnapshotTests/testDefaultCard_11_2@2x.png (96d48d5): Smudge error: Error downloading snapshot_test_goldens/goldens_64/MDCCardSnapshotTests/testDefaultCard_11_2@2x.png (96d48d51f415dc94427e49a361cf68cdadc86d0e00a6706894a7f82b5d13b086): batch request: missing protocol: "file:///Users/featherless/workbench/material-components-ios/.git/info/lfs"
Errors logged to /private/tmp/mdcios/mdc-ios/.git/lfs/logs/20181218T112727.676554.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: snapshot_test_goldens/goldens_64/MDCCardSnapshotTests/testDefaultCard_11_2@2x.png: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'
```
This is an automated change generated by replacing all instances of MaterialComponentsAlpha with MaterialComponentsBeta. This is not a breaking change because changes to Alpha/Beta components (including renaming them) are not considered breaking.
The MaterialComponentsAlpha podspec was mistakenly named Alpha, when what we meant was more close to Beta. The distinction is that Alpha components are not expected to be used by clients, while Beta components are.
* Snapshot Testing Proof of Concept (#5754)
### The problem
We currently do not have UI tests on a component level. Integrating snapshot tests would allow us to have peace of mind with each PR that it isn't going to introduce any changes to the UI unless its intended to.
### The solution
* Integrate `ios-snapshot-test-case` pod to handle generating and diffing images of components.
* Integrate `git-lfs` to handle storage of the goldens.
This PR creates one test to showcase the ability to do snapshot tests. Upon merging this PR, you must install git-lfs in order to properly have the images pulled down. The 3 steps to do this:
1. `brew install git-lfs`
2. `git lfs install`
3. `git lfs pull`
Additionally, the golden is generated using an iOS 11 simulator at 2x scale so that the kokoro jobs are happy.
### Related bugs
Closes#5740
### Difference from #5754
**Note:** This is a re-revert of #5754 that aims to fix issues with Travis CI by ensuring the snapshot test only runs on a single iOS version. I've opened #5888 to expand on this in the future. A few things changed in the approach in this PR:
* We only run the snapshot test for iOS 11.2.0 until we can have an elegant solution for supporting multiple OS''s (see #5888)
* The snapshot tests now live in their own test target to avoid issues with having to require an App host for all tests.
* Additionally, a dummy swift file was required for compilation of this new test target (see https://forums.developer.apple.com/thread/88451 for context)
### The problem
We currently do not have UI tests on a component level. Integrating snapshot tests would allow us to have peace of mind with each PR that it isn't going to introduce any changes to the UI unless its intended to.
### The solution
* Integrate `ios-snapshot-test-case` pod to handle generating and diffing images of components.
* Integrate `git-lfs` to handle storage of the goldens.
This PR creates one test to showcase the ability to do snapshot tests. Upon merging this PR, you must install git-lfs in order to properly have the images pulled down. The 3 steps to do this:
1. `brew install git-lfs`
2. `git lfs install`
3. `git lfs pull`
Additionally, the golden is generated using an iOS 11 simulator at 2x scale so that the kokoro jobs are happy.
### Related bugs
Closes#5740
### Context
When determining whether a change affects multiple components we want to ignore changes to CocoaPods files that live outside of the components directories.
### The problem + fix
Prior to this change, if a commit modified MaterialComponentsAlpha.podspec or catalog/Podfile it was being picked up as a multi-component change in the release notes.
After this change, commits affecting those files will not be treated as multi-component changes unless they also touch other files in the repo.
### Opportunities for further improvement
It's likely fair to say that any modification to files outside of the `components/` directory can be ignored. I'm not sure what the best way to filter these types of changes is, but that would be a welcome follow-up simplification of this change that would require less on-going maintenance.
The script now outputs four groups of changelogs:
1. Breaking changes for specific components.
2. Breaking changes across multiple components.
3. Changes for specific components.
4. Changes acrosss multiple components.
This will make it easier for us to keep all google-owned repositories running the same clang-format version in tandem.
Also moved from a submodule to a clone + version checkout so that our CI gracefully upgrades as we release new non-breaking release to clang-format-ci.
Relies on upstream changes in https://github.com/material-foundation/clang-format-ci/pull/4
{MDC iOS} No longer using `-init` for Color Scheme.
Based on the discussions in go/mdc-ios-theming, we should not use the `-init` default initializer. Instead, we should use an explicit set of defaults.
Search regex: '\[\[MDCSemanticColorScheme alloc\] init\]'
## Context
Two libraries have graduated from the material-components-ios repo so that they can be used more generally across other open source projects: [clang-format-ci](https://github.com/material-foundation/clang-format-ci) and [github-comment](https://github.com/material-foundation/github-comment).
## What's going on here
Both of these libraries have been added as external submodule dependencies and the scripts have been updated to refer to the new external dependency paths.
## Prior to this change
Our test_all script was using an Xcode schemes search and filter to identify any test schemes that exist in the repository and test them. Unfortunately, the script was not also testing to confirm that it found at least *some* tests, meaning if it did not find any tests it would silently succeed.
The script was essentially optimized for us adding multiple test schemes in the future. In practice, the frequency with which we add new test schemes is low (we've added only one in the past year via MaterialComponentsAlpha).
## The root cause
With d238c86d47eb072617d285106147ace613321aee, our Xcode test scheme names changed and our scheme search was no longer returning our unit test schemes, resulting in test_all succeeding even though it hadn't run any tests.
## After this change
The test_all script will explicitly test specific schemes. If these schemes change or move in the future, this script will fail until we also update the script's schemes.
This script now also pipes the test output through xcpretty if it is available.
Closes https://github.com/material-components/material-components-ios/issues/5539
This add the shape theming doc generation for our components. Same as we did for color, we are adding the shape theming template so then it can be easily generated for our docs. I will generate the shape theming docs for our components in a follow up PR.
Prior to this change, `Component+Extension` additions would be incorrectly listed as a "New component" in the API diff.
After this change, `Componet+Extension` additions will be listed as "New extension" in the API diff.
This ensures that if a piece of feedback moves around within a given file (e.g. because lines were added or removed above this feedback) that we won't erroneously post the same comment again.
This job is our first official linter.
<img width="830" alt="screen shot 2018-08-30 at 5 12 35 pm" src="https://user-images.githubusercontent.com/45670/44880230-d8049780-ac79-11e8-926b-836b16c86f9f.png">
See https://github.com/material-components/material-components-ios/pull/4954 for an example thread in which I purposefully made some style regressions and let the tool post back to the PR until I resolved them.
This new job will apply clang-format *only* to the lines that have been changed by a given pull request. If any changes are suggested, the diff will be shown in the job's build logs and individual changes will be posted back to the PR at each suggested line.
The provided feedback can be addressed in multiple ways:
1. Manually, by copying the suggested changes into your code.
2. By running git-clang-format on your local branch. The tool will suggest this, including an ideal command line invocation to do so.
This job can be run locally, though it requires a fair amount of configuration to do so. See the job's documentation for more details on which environment variables are required.
---
This change also required modifications to the github-comment command line tool. Notably:
1. A very light-weight diff parser was added. This API takes a git diff [unified diff](https://www.gnu.org/software/diffutils/manual/html_node/Example-Unified.html#Example-Unified) file and parses it into individual [hunks](https://www.gnu.org/software/diffutils/manual/html_node/Hunks.html).
2. A hunk-correlator was added. This is the magic that translates clang-format's suggested diff into individual positions on a GitHub pull request.
3. A new GitHub API method for fetching a single file was added. We require this in order to fetch the pull request diff.
4. A new set of command line parameters was added to the github-comment tool. Use of these new parameters turn a suggested diff into a series of GitHub pull request line comments.
Before posting any new comments to the PR, the command enumerates all of the existing comments into a Set. If desired comment already exists in the set of comments then it will not be posted again. This is the mechanism by which we minimize noise generated from the job on subsequent updates to the PR.
---
The essential workflow for this tool is (on kokoro):
1. Install the clang-format version we're using internally by fetching it from our pre-built binary repository: https://github.com/material-foundation/clang-format
2. Install the git-clang-format tool.
3. Verify the checksums of both tools.
4. Run git-clang-format on the pull request.
5. If there have been no changes, exit with status 0 and post success back to the PR. Deletes any comment that had been made previously about style (this matches the apidiff behavior).
6. Otherwise, output the diff results and feed the diff to github-comment.
7. github-comment parses the pull request diff and the suggested diff, correlates the suggested hunks with the pull request hunks, and then posts individual line comments with the suggested changes.
8. If a summary comment hasn't already been posted to the PR, a summary comment is posted. Otherwise, we update the existing comment.
9. Post a failure status to the PR.
This resolves a warning that was appearing when building/running the github-comment project:
```
warning: Ignoring duplicate product 'github-comment'
```
Tested by running:
```
cd scripts/github-comment/
swift run github-comment
```
And verifying that the above warning did not appear and that the command successfully built and executed.
The umbrellas command is fed in to the apidiff tool as a filter for which components to look at.
Prior to this change, the umbrellas command was ignoring the apidiff base sha and always generating results from origin/stable.
After this change, the umbrellas and files commands will both accept an optional base sha from which to generate results.
Tested by running:
```bash
./scripts/release umbrellas HEAD
# Should output nothing
```
The release engineer is expected to fill these sections in by gathering the information from the commits. If information is not available in a given commit, then the release engineer is expected to reach out to the commit author and ask them to provide the additional information (and encouraging them to do so in their commits in the future).
