1# external_updater 2 3external updater is a tool to automatically updates libraries in external/ . 4 5## Usage 6 7Check updates for a library, or verify METADATA is valid: 8 9```shell 10tools/external_updater/updater.sh check ${LIBNAME} 11``` 12 13Check updates for all libraries in external/: 14 15```shell 16tools/external_updater/updater.sh check --all 17``` 18 19Update a library: 20 21```shell 22tools/external_updater/updater.sh update ${LIBNAME} 23``` 24 25LIBNAME can be the path to a library under external/. E.g. kotlinc, or 26python/cpython3. 27 28## Configure 29 30To use this tool, a METADATA file must present at the root of the 31repository. The full definition can be found 32[here](https://android.googlesource.com/platform/tools/external_updater/+/refs/heads/master/metadata.proto). 33Or see example [here](https://android.googlesource.com/platform/external/ImageMagick/+/refs/heads/master/METADATA) 34 35The most important part in the file is a list of urls. 36`external_updater` will go through all urls and uses the first 37supported url. 38 39### Git upstream 40 41If type of a URL is set to GIT, the URL must be a git upstream 42(the one you can use with `git clone`). And the version field must 43be either a version tag, or SHA. The tool will find the latest 44version tag or sha based on it. 45 46When upgrade, the tool will simply run `git merge tag/sha`. 47 48IMPORTANT: It is suggested to set up a `upstream-master` branch to 49replicate upstream. Because most users don't have the privilege to 50upload changes not authored by themselves. This can be done by 51filing a bug to componentid:99104. 52 53#### SHA 54 55If the version is a SHA, the tool will always try to upgrade to the 56top of upstream. As long as there is any new change upstream, local 57library will be treated as stale. 58 59#### Version tag 60 61If the version is not a SHA, the tool will try to parse the version 62to get a numbered version. Currently the supported version format is: 63 64```markdown 65<prefix><version_number><suffix> 66``` 67 68version_number part can be numbers separated by `.` or `-` or `_`. 69 70If you have project where this isn't working, file a bug so we can take a look. 71 72#### Local changes 73 74It is suggested to verify all local changes when upgrading. This can 75be done easily in Gerrit, by comparing parent2 and the patchset. 76 77 78### GitHub archive 79 80If the url type is ARCHIVE, and the url is from GitHub, `external_updater` 81can upgrade a library based on GitHub releases. 82 83If you have the choice between archives and git tags, choose tags. 84Because that makes it easier to manage local changes. 85 86The tool will query GitHub to get the latest release from: 87 88```url 89https://github.com/user/proj/releases/latest 90``` 91 92If the tag of latest release is not equal to version in METADATA file, a 93new version is found. The tool will download the tarball and overwrite the 94library with it. 95 96If there are multiple archives in one GitHub release, the one most 97[similar](https://en.wikipedia.org/wiki/Edit_distance) to previous 98(from METADATA) will be used. 99 100After upgrade, files not present in the new tarball will be removed. But we 101explicitly keep files famous in Android tree. 102See [here](https://android.googlesource.com/platform/tools/external_updater/+/refs/heads/master/update_package.sh). 103 104If more files need to be reserved, a post_update.sh can be created to copy 105these files over. 106See [example](https://android.googlesource.com/platform/external/kotlinc/+/refs/heads/master/post_update.sh). 107 108Local patches can be kept as patches/*.diff. They will be applied after 109upgrade. [example](https://cs.corp.google.com/android/external/jsmn/patches/header.diff) 110 111## Email notification 112 113There is some support to automatically check updates for all external 114libraries every hour, send email and change. Currently this is done by 115running the following script on a desktop machine. 116 117```shell 118#!/bin/bash 119 120cd /src/aosp 121while true 122do 123 repo abandon tmp_auto_upgrade 124 repo forall -c git checkout . 125 repo forall -c git clean -xdf 126 repo sync -c 127 source build/envsetup.sh 128 lunch aosp_arm-eng 129 mmma tools/external_updater 130 131 out/soong/host/linux-x86/bin/external_updater_notifier \ 132 --history ~/updater/history \ 133 --recipients=android_external_lib_updates@google.com \ 134 --generate_change \ 135 --all 136 date 137 echo "Sleeping..." 138 sleep 3600 139done 140``` 141