1# external_updater 2 3external updater is a tool to automatically update libraries in external/. 4 5## Usage 6 7Check updates for a library or verify METADATA is valid: 8 9```shell 10tools/external_updater/updater.sh check ${LIBNAME} 11``` 12 13Update a library, commit, and upload the change to Gerrit: 14 15```shell 16tools/external_updater/updater.sh update ${LIBNAME} 17``` 18 19Update a library without committing and uploading to Gerrit: 20 21```shell 22tools/external_updater/updater.sh update --no-upload ${LIBNAME} 23``` 24 25Update a library on top of the local changes in the current branch, commit, and upload the change to Gerrit: 26 27```shell 28tools/external_updater/updater.sh update --keep-local-changes ${LIBNAME} 29``` 30 31Update a library without building: 32 33```shell 34tools/external_updater/updater.sh update --no-build ${LIBNAME} 35``` 36 37LIBNAME can be the path to a library under external/, e.g. kotlinc, or 38python/cpython3. 39 40## Configure 41 42To use this tool, a METADATA file must present at the root of the 43repository. The full definition can be found 44[here](https://android.googlesource.com/platform/tools/external_updater/+/refs/heads/master/metadata.proto). 45Or see example [here](https://android.googlesource.com/platform/external/ImageMagick/+/refs/heads/master/METADATA) 46 47The most important part in the file is a list of urls. 48`external_updater` will go through all urls and uses the first 49supported url. 50 51### Git upstream 52 53If type of a URL is set to GIT, the URL must be a git upstream 54(the one you can use with `git clone`). And the version field must 55be either a version tag, or SHA. The tool will find the latest 56version tag or sha based on it. 57 58When upgrade, the tool will simply run `git merge tag/sha`. 59 60IMPORTANT: It is suggested to set up a `upstream-master` branch to 61replicate upstream. Because most users don't have the privilege to 62upload changes not authored by themselves. This can be done by 63filing a bug to componentid:99104. 64 65#### SHA 66 67If the version is a SHA, the tool will always try to upgrade to the 68top of upstream. As long as there is any new change upstream, local 69library will be treated as stale. 70 71#### Version tag 72 73If the version is not a SHA, the tool will try to parse the version 74to get a numbered version. Currently the supported version format is: 75 76```markdown 77<prefix><version_number><suffix> 78``` 79 80version_number part can be numbers separated by `.` or `-` or `_`. 81 82If you have project where this isn't working, file a bug so we can take a look. 83 84#### Local changes 85 86It is suggested to verify all local changes when upgrading. This can 87be done easily in Gerrit, by comparing parent2 and the patchset. 88 89 90### GitHub archive 91 92If the url type is ARCHIVE, and the url is from GitHub, `external_updater` 93can upgrade a library based on GitHub releases. 94 95If you have the choice between archives and git tags, choose tags. 96Because that makes it easier to manage local changes. 97 98The tool will query GitHub to get the latest release from: 99 100```url 101https://github.com/user/proj/releases/latest 102``` 103 104If the tag of latest release is not equal to version in METADATA file, a 105new version is found. The tool will download the tarball and overwrite the 106library with it. 107 108If there are multiple archives in one GitHub release, the one most 109[similar](https://en.wikipedia.org/wiki/Edit_distance) to previous 110(from METADATA) will be used. 111 112After upgrade, files not present in the new tarball will be removed. But we 113explicitly keep files famous in Android tree. 114See [here](https://android.googlesource.com/platform/tools/external_updater/+/refs/heads/master/update_package.sh). 115 116If more files need to be reserved, a post_update.sh can be created to copy 117these files over. 118See [example](https://android.googlesource.com/platform/external/kotlinc/+/refs/heads/master/post_update.sh). 119 120#### Local patches 121 122Local patches can be kept as patches/*.diff. They will be applied after 123upgrade. [example](https://cs.android.com/android/platform/superproject/+/master:external/jsmn/patches/header.diff) 124 125## Email notification 126 127There is some support to automatically check updates for all external 128libraries every hour, send email and change. Currently this is done by 129running the following script on a desktop machine. 130 131```shell 132#!/bin/bash 133 134cd /src/aosp 135while true 136do 137 repo abandon tmp_auto_upgrade 138 repo forall -c git checkout . 139 repo forall -c git clean -xdf 140 repo sync -c 141 source build/envsetup.sh 142 lunch aosp_arm-eng 143 mmma tools/external_updater 144 145 out/soong/host/linux-x86/bin/external_updater_notifier \ 146 --history ~/updater/history \ 147 --recipients=android_external_lib_updates@google.com \ 148 --generate_change \ 149 --all 150 date 151 echo "Sleeping..." 152 sleep 3600 153done 154``` 155