1Android 2======= 3 4Mesa hardware drivers can be built for Android one of two ways: built 5into the Android OS using the ndk-build build system on older versions 6of Android, or out-of-tree using the Meson build system and the 7Android NDK. 8 9The ndk-build build system has proven to be hard to maintain, as one 10needs a built Android tree to build against, and it has never been 11tested in CI. The Meson build system flow is frequently used by 12Chrome OS developers for building and testing Android drivers. 13 14When building llvmpipe or lavapipe for Android the ndk-build workflow 15is also used, but there are additional steps required to add the driver 16to the Android OS image. 17 18Building using the Android NDK 19------------------------------ 20 21Download and install the NDK using whatever method you normally would. 22Then, create your Meson cross file to use it, something like this 23``~/.local/share/meson/cross/android-aarch64`` file: 24 25.. code-block:: ini 26 27 [binaries] 28 ar = 'NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android-ar' 29 c = ['ccache', 'NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android34-clang'] 30 cpp = ['ccache', 'NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android34-clang++', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables', '--start-no-unused-arguments', '-static-libstdc++', '--end-no-unused-arguments'] 31 c_ld = 'lld' 32 cpp_ld = 'lld' 33 strip = 'NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android-strip' 34 # Android doesn't come with a pkg-config, but we need one for Meson to be happy not 35 # finding all the optional deps it looks for. Use system pkg-config pointing at a 36 # directory we get to populate with any .pc files we want to add for Android 37 pkg-config = ['env', 'PKG_CONFIG_LIBDIR=NDKDIR/pkgconfig', '/usr/bin/pkg-config'] 38 39 [host_machine] 40 system = 'android' 41 cpu_family = 'aarch64' 42 cpu = 'armv8' 43 endian = 'little' 44 45Now, use that cross file for your Android build directory (as in this 46one cross-compiling the turnip driver for a stock Pixel phone) 47 48.. code-block:: sh 49 50 meson setup build-android-aarch64 \ 51 --cross-file android-aarch64 \ 52 -Dplatforms=android \ 53 -Dplatform-sdk-version=34 \ 54 -Dandroid-stub=true \ 55 -Dgallium-drivers= \ 56 -Dvulkan-drivers=freedreno \ 57 -Dfreedreno-kmds=kgsl 58 meson compile -C build-android-aarch64 59 60Replacing Android drivers on stock Android 61------------------------------------------ 62 63The vendor partition with the drivers is normally mounted from a 64read-only disk image on ``/vendor``. To be able to replace them for 65driver development, we need to unlock the device and remount 66``/vendor`` read/write. 67 68.. code-block:: sh 69 70 adb disable-verity 71 adb reboot 72 adb remount -R 73 adb remount 74 75Now you can replace drivers as in: 76 77.. code-block:: sh 78 79 adb push build-android-aarch64/src/freedreno/vulkan/libvulkan_freedreno.so /vendor/lib64/hw/vulkan.sdm710.so 80 81Note this command doesn't quite work because libvulkan wants the 82SONAME to match. You can use ``patchelf`` to fix this: 83 84.. code-block:: sh 85 86 cp build-android-aarch64/src/freedreno/vulkan/libvulkan_freedreno.so /tmp/vulkan.sdm710.so 87 patchelf --set-soname vulkan.sdm710.so /tmp/vulkan.sdm710.so 88 adb push /tmp/vulkan.sdm710.so /vendor/lib64/hw/ 89 90Replacing Android drivers on Chrome OS 91-------------------------------------- 92 93Chrome OS's ARC++ is an Android container with hardware drivers inside 94of it. The vendor partition with the drivers is normally mounted from 95a read-only squashfs image on disk. For doing rapid driver 96development, you don't want to regenerate that image. So, we'll take 97the existing squashfs image, copy it out on the host, and then use a 98bind mount instead of a loopback mount so we can update our drivers 99using scp from outside the container. 100 101On your device, you'll want to make ``/`` read-write. ssh in as root 102and run: 103 104.. code-block:: sh 105 106 crossystem dev_boot_signed_only=0 107 /usr/share/vboot/bin/make_dev_ssd.sh --remove_rootfs_verification --partitions 4 108 reboot 109 110Then, we'll switch Android from using an image for ``/vendor`` to using a 111bind-mount from a directory we control. 112 113.. code-block:: sh 114 115 cd /opt/google/containers/android/ 116 mkdir vendor-ro 117 mount -o loop vendor.raw.img vendor-ro 118 cp -a vendor-ro vendor-rw 119 emacs config.json 120 121In the ``config.json``, you want to find the block for ``/vendor`` and 122change it to:: 123 124 { 125 "destination": "/vendor", 126 "type": "bind", 127 "source": "/opt/google/containers/android/vendor-rw", 128 "options": [ 129 "bind", 130 "rw" 131 ] 132 }, 133 134Now, restart the UI to do a full reload: 135 136.. code-block:: sh 137 138 restart ui 139 140At this point, your android container is restarted with your new 141bind-mount ``/vendor``, and if you use ``android-sh`` to shell into it 142then the ``mount`` command should show:: 143 144 /dev/root on /vendor type ext2 (rw,seclabel,relatime) 145 146Now, replacing your DRI driver with a new one built for Android should 147be a matter of: 148 149.. code-block:: sh 150 151 scp msm_dri.so $HOST:/opt/google/containers/android/vendor-rw/lib64/dri/ 152 153You can do your build of your DRI driver using ``emerge-$BOARD 154arc-mesa-freedreno`` (for example) if you have a source tree with 155ARC++, but it should also be possible to build using the NDK as 156described above. There are currently rough edges with this, for 157example the build will require that you have your arc-libdrm build 158available to the NDK, assuming you're building anything but the 159Freedreno Vulkan driver for KGSL. You can mostly put things in place 160with: 161 162.. code-block:: sh 163 164 scp $HOST:/opt/google/containers/android/vendor-rw/lib64/libdrm.so \ 165 NDKDIR/sysroot/usr/lib/aarch64-linux-android/lib/ 166 167 ln -s \ 168 /usr/include/xf86drm.h \ 169 /usr/include/libsync.h \ 170 /usr/include/libdrm \ 171 NDKDIR/sysroot/usr/include/ 172 173It seems that new invocations of an application will often reload the 174DRI driver, but depending on the component you're working on you may 175find you need to reload the whole Android container. To do so without 176having to log in to Chrome again every time, you can just kill the 177container and let it restart: 178 179.. code-block:: sh 180 181 kill $(cat /run/containers/android-run_oci/container.pid ) 182 183Adding out-of-tree drivers to Android OS image 184---------------------------------------------- 185 186When building your own Android OS images it's possible to add 187drivers built out of tree directly into the OS image. For 188running llvmpipe and lavapipe on Android this step is required 189to ensure Android is able to load the drivers correctly. 190 191The following steps provide and example for building 192the android cuttlefish image following the official Android 193documentation from https://source.android.com/docs/setup 194 195When building llvmpipe or lavapipe for Android, it is required 196to do this so that the permissions for accessing the library 197are set correctly. 198 199Following the Android documentation, we can run the following 200commands 201 202.. code-block:: sh 203 204 repo init -b main -u https://android.googlesource.com/platform/manifest 205 repo sync -c -j8 206 207 source build/envsetup.sh 208 lunch aosp_cf_x86_64_phone-trunk_staging-userdebug 209 210Be aware that the sync command can take a long time to run as 211it will download all of the source code. This will set up 212the ``aosp_cf_x86_64_phone-trunk_staging-userdebug`` build target 213for Android. Please note that the x86_64 cuttlefish target will require 214you to build mesa for 32bit and 64bit. Next we need to copy the build 215driver libraries into the source tree of Android and patch the binary names. 216 217.. code-block:: sh 218 219 mkdir prebuilts/mesa 220 mkdir prebuilts/mesa/x86_64 221 mkdir prebuilts/mesa/x86 222 cp ${INSTALL_PREFIX_64}/lib/libEGL.so prebuilts/mesa/x86_64/ 223 cp ${INSTALL_PREFIX_64}/lib/libgallium_dri.so prebuilts/mesa/x86_64/ 224 cp ${INSTALL_PREFIX_64}/lib/libGLESv1_CM.so prebuilts/mesa/x86_64/ 225 cp ${INSTALL_PREFIX_64}/lib/libGLESv2.so prebuilts/mesa/x86_64/ 226 cp ${INSTALL_PREFIX_64}/lib/libvulkan_lvp.so prebuilts/mesa/x86_64/ 227 cp ${INSTALL_PREFIX_32}/lib/libEGL.so prebuilts/mesa/x86 228 cp ${INSTALL_PREFIX_32}/lib/libgallium_dri.so prebuilts/mesa/x86/ 229 cp ${INSTALL_PREFIX_32}/lib/libGLESv1_CM.so prebuilts/mesa/x86 230 cp ${INSTALL_PREFIX_32}/lib/libGLESv2.so prebuilts/mesa/x86 231 cp ${INSTALL_PREFIX_32}/lib/libvulkan_lvp.so prebuilts/mesa/x86 232 233 patchelf --set-soname libEGL_lp.so prebuilts/mesa/x86_64/libEGL.so 234 patchelf --set-soname libGLESv1_CM_lp.so prebuilts/mesa/x86_64/libGLESv1_CM.so 235 patchelf --set-soname libGLESv2_lp.so prebuilts/mesa/x86_64/libGLESv2.so 236 patchelf --set-soname vulkan.lvp.so prebuilts/mesa/x86_64/libvulkan_lvp.so 237 patchelf --set-soname libEGL_lp.so prebuilts/mesa/x86/libEGL.so 238 patchelf --set-soname libGLESv1_CM_lp.so prebuilts/mesa/x86/libGLESv1_CM.so 239 patchelf --set-soname libGLESv2_lp.so prebuilts/mesa/x86/libGLESv2.so 240 patchelf --set-soname vulkan.lvp.so prebuilts/mesa/x86/libvulkan_lvp.so 241 242We then need to create an ``prebuilts/mesa/Android.bp`` build file to include 243the libraries in the build. 244 245.. code-block:: 246 247 cc_prebuilt_library_shared { 248 name: "libgallium_dri", 249 arch: { 250 x86_64: { 251 srcs: ["x86_64/libgallium_dri.so"], 252 }, 253 x86: { 254 srcs: ["x86/libgallium_dri.so"], 255 }, 256 }, 257 strip: { 258 none: true, 259 }, 260 relative_install_path: "egl", 261 shared_libs: ["libc", "libdl", "liblog", "libm"], 262 check_elf_files: false, 263 vendor: true 264 } 265 266 cc_prebuilt_library_shared { 267 name: "libEGL_lp", 268 arch: { 269 x86_64: { 270 srcs: ["x86_64/libEGL.so"], 271 }, 272 x86: { 273 srcs: ["x86/libEGL.so"], 274 }, 275 }, 276 strip: { 277 none: true, 278 }, 279 relative_install_path: "egl", 280 shared_libs: ["libc", "libdl", "liblog", "libm", "libcutils", "libdrm", "libhardware", "liblog", "libnativewindow", "libsync"], 281 check_elf_files: false, 282 vendor: true 283 } 284 285 cc_prebuilt_library_shared { 286 name: "libGLESv1_CM_lp", 287 arch: { 288 x86_64: { 289 srcs: ["x86_64/libGLESv1_CM.so"], 290 }, 291 x86: { 292 srcs: ["x86/libGLESv1_CM.so"], 293 }, 294 }, 295 strip: { 296 none: true, 297 }, 298 relative_install_path: "egl", 299 shared_libs: ["libc", "libdl", "liblog", "libm"], 300 check_elf_files: false, 301 vendor: true 302 } 303 304 cc_prebuilt_library_shared { 305 name: "libGLESv2_lp", 306 arch: { 307 x86_64: { 308 srcs: ["x86_64/libGLESv2.so"], 309 }, 310 x86: { 311 srcs: ["x86_64/libGLESv2.so"], 312 }, 313 }, 314 strip: { 315 none: true, 316 }, 317 relative_install_path: "egl", 318 shared_libs: ["libc", "libdl", "liblog", "libm"], 319 check_elf_files: false, 320 vendor: true 321 } 322 323 cc_prebuilt_library_shared { 324 name: "vulkan.lvp", 325 arch: { 326 x86_64: { 327 srcs: ["x86_64/libvulkan_lvp.so"], 328 }, 329 x86: { 330 srcs: ["x86/libvulkan_lvp.so"], 331 }, 332 }, 333 strip: { 334 none: true, 335 }, 336 relative_install_path: "hw", 337 shared_libs: ["libc", "libdl", "liblog", "libm", "libcutils", "libdrm", "liblog", "libnativewindow", "libsync", "libz"], 338 vendor: true 339 } 340 341 342Next we need to update the device configuration to include the libraries 343in the build, as well as set the appropriate system properties. We can 344create the file 345``device/google/cuttlefish/shared/mesa/device_vendor.mk`` 346 347 348.. code-block:: makefile 349 350 PRODUCT_SOONG_NAMESPACES += prebuilts/mesa 351 PRODUCT_PACKAGES += libglapi \ 352 libGLESv1_CM_lp \ 353 libGLESv2_lp \ 354 libEGL_lp \ 355 libgallium_dri.so \ 356 vulkan.lvp 357 PRODUCT_VENDOR_PROPERTIES += \ 358 ro.hardware.egl=lp \ 359 ro.hardware.vulkan=lvp \ 360 mesa.libgl.always.software=true \ 361 mesa.android.no.kms.swrast=true \ 362 debug.hwui.renderer=opengl \ 363 ro.gfx.angle.supported=false \ 364 debug.sf.disable_hwc_vds=1 \ 365 ro.vendor.hwcomposer.mode=client 366 367Also the file ``device/google/cuttlefish/shared/mesa/BoardConfig.mk`` 368 369.. code-block:: makefile 370 371 BOARD_VENDOR_SEPOLICY_DIRS += \ 372 device/google/cuttlefish/shared/mesa/sepolicy 373 374Next the file ``device/google/cuttlefish/shared/mesa/sepolicy/file_contexts`` 375 376.. code-block:: sh 377 378 /vendor/lib(64)?/egl/libEGL_lp\.so u:object_r:same_process_hal_file:s0 379 /vendor/lib(64)?/egl/libGLESv1_CM_lp\.so u:object_r:same_process_hal_file:s0 380 /vendor/lib(64)?/egl/libGLESv2_lp\.so u:object_r:same_process_hal_file:s0 381 /vendor/lib(64)?/libglapi\.so u:object_r:same_process_hal_file:s0 382 /vendor/lib(64)?/libgallium_dri\.so u:object_r:same_process_hal_file:s0 383 /vendor/lib(64)?/hw/vulkan\.lvp\.so u:object_r:same_process_hal_file:s0 384 385After creating these files we need to modify the existing config files 386to include these build files. First we modify 387``device/google/cuttlefish/shared/phone/device_vendor.mk`` 388to add the below code in the spot where other device_vendor 389files are included. 390 391.. code-block:: sh 392 393 $(call inherit-product, device/google/cuttlefish/shared/mesa/device_vendor.mk) 394 395Lastly we modify 396``device/google/cuttlefish/vsoc_x86_64/BoardConfig.mk`` to include 397the following line where the other BoardConfig files are included 398 399.. code-block:: sh 400 401 -include device/google/cuttlefish/shared/mesa/BoardConfig.mk 402 403Then we are set to continue following the official instructions to 404build the cuttlefish target and run it in the cuttlefish emulator. 405