1# external_updater 2 3external updater is a tool to automatically update libraries in external/. 4 5The documentation on this page is for users of `external_updater`. If you're 6looking for developer docs, see [docs/dev.md](docs/dev.md). 7 8## Usage 9 10In each of the examples below, `$PROJECT_PATH` is the path to the project to 11operate on. If more than one path is given, external_updater will operate on 12each in turn. 13 14Make sure you have initialized AOSP main source code. The default remote for 15external updater is AOSP. 16 17If you are trying to upgrade a project in other remotes, you can pass 18`--remote-name $REMOTE` to the `update` parser. We strongly recommend updating 19projects in AOSP and allowing automerger to merge the upgrade CL with other 20branches. 21 22To use this tool, a METADATA file must present at the root of the 23repository. The full definition can be found in 24[metadata.proto](https://android.googlesource.com/platform/tools/external_updater/+/refs/heads/main/metadata.proto). 25Or 26[external/toybox/METADATA](https://android.googlesource.com/platform/external/toybox/+/refs/heads/main/METADATA) 27is a concrete example. 28 29From within your working directory, source the `envsetup.sh` script to set up 30your build environment and pick a target to build with the `lunch` command. You 31can pass any target that you want. After upgrading a project, external_updater 32starts building for the selected lunch target: 33 34```shell 35source build/envsetup.sh 36lunch aosp_cf_x86_64_phone-trunk_staging-eng 37``` 38 39Check updates for a library or verify METADATA is valid: 40 41```shell 42tools/external_updater/updater.sh check PROJECT_PATH 43``` 44 45Update a library, commit, and upload the change to Gerrit: 46 47```shell 48tools/external_updater/updater.sh update PROJECT_PATH 49``` 50 51PROJECT_PATH can be the path to a library under external/, e.g. 52external/kotlinc, or external/python/cpython3. You can press Tab to complete the 53path. 54 55The following options can be passed to `update` parser: 56```shell 57--no-build Skip building 58--no-upload Does not upload to Gerrit after upgrade 59--bug BUG Bug number for this update 60--custom-version CUSTOM_VERSION Custom version we want to upgrade to. 61--skip-post-update Skip post_update script if post_update script exists 62--keep-local-changes Updates the current branch instead of creating a new branch 63--no-verify Pass --no-verify to git commit 64--remote-name REMOTE_NAME Remote repository name, the default is set to aosp 65--exclude$EXCLUDE Names of projects to exclude. These are just the final part of the path with no directories. 66--refresh Run update and refresh to the current version. 67--keep-date Run update and do not change date in METADATA. 68--json-output JSON_OUTPUT Path of a json file to write result to. 69``` 70 71For example: 72 73```shell 74tools/external_updater/updater.sh update --custom-version $VERSION $PROJECT_PATH 75``` 76 77## Configure 78 79The most important part in the file is a list of urls. 80`external_updater` will go through all urls and uses the first 81supported url. 82 83### Git upstream 84 85If the url type is `Git`, the URL must be a git upstream 86(the one you can use with `git clone`). And the version field must 87be either a version tag, or SHA. The tool will find the latest 88version tag or sha based on it. 89 90When upgrade, the tool will simply run `git merge tag/sha`. 91 92IMPORTANT: It is suggested to set up a `upstream-main` branch to 93replicate upstream. Because most users don't have the privilege to 94upload changes not authored by themselves. This can be done by 95filing a bug to componentid:99104. 96 97#### SHA 98 99If the version is a SHA, the tool will always try to upgrade to the 100top of upstream. As long as there is any new change upstream, local 101library will be treated as stale. 102 103#### Version tag 104 105If the version is not a SHA, the tool will try to parse the version 106to get a numbered version. Currently the supported version format is: 107 108```markdown 109<prefix><version_number><suffix> 110``` 111 112version_number part can be numbers separated by `.` or `-` or `_`. 113 114If you have project where this isn't working, file a bug so we can take a look. 115 116#### Local changes 117 118It is suggested to verify all local changes when upgrading. This can 119be done easily in Gerrit, by comparing parent2 and the patchset. 120 121 122### GitHub archive 123 124If the url type is `Archive`, and the url is from GitHub, `external_updater` 125will upgrade a library based on GitHub tags/releases. 126 127If you have the choice between archives and git tags, choose tags. 128Because that makes it easier to manage local changes. 129 130The tool will query GitHub to get the latest release from: 131 132```url 133https://github.com/user/proj/releases/latest 134``` 135 136If the tag of latest release is not equal to version in METADATA file, a 137new version is found. The tool will download the tarball and overwrite the 138library with it. 139 140If there are multiple archives in one GitHub release, the one most 141[similar](https://en.wikipedia.org/wiki/Edit_distance) to previous 142(from METADATA) will be used. 143 144After upgrade, files not present in the new tarball will be removed. But we 145explicitly keep files famous in Android tree. 146See [update_package.sh](https://android.googlesource.com/platform/tools/external_updater/+/refs/heads/main/update_package.sh). 147 148If more files need to be reserved, a post_update.sh can be created to copy 149these files over. 150See [example](https://android.googlesource.com/platform/external/kotlinc/+/refs/heads/main/post_update.sh). 151 152#### Local patches 153 154Local patches can be kept as patches/*.diff. They will be applied after 155upgrade. [example](https://cs.android.com/android/platform/superproject/main/+/main:external/jsmn/patches/header.diff) 156 157## Email notification 158 159There is some support to automatically check updates for all external 160libraries every hour, send email and change. Currently this is done by 161running the following script on a desktop machine. 162 163```shell 164#!/bin/bash 165 166cd /src/aosp 167while true 168do 169 repo abandon tmp_auto_upgrade 170 repo forall -c git checkout . 171 repo forall -c git clean -xdf 172 repo sync -c 173 source build/envsetup.sh 174 lunch aosp_arm-eng 175 mmma tools/external_updater 176 177 out/soong/host/linux-x86/bin/external_updater_notifier \ 178 --history ~/updater/history \ 179 --recipients=android_external_lib_updates@google.com \ 180 --generate_change \ 181 --all 182 date 183 echo "Sleeping..." 184 sleep 3600 185done 186``` 187