1Android OverlayFS Integration with adb Remount 2============================================== 3 4Introduction 5------------ 6 7Users working with userdebug or eng builds expect to be able to remount the 8system partition as read-write and then add or modify any number of files 9without reflashing the system image, which is efficient for a development cycle. 10 11Limited memory systems use read-only types of file systems or dynamic 12Android partitions (DAPs). These file systems land system partition images 13right-sized, and have been deduped at the block level to compress the content. 14This means that a remount either isn’t possible, or isn't useful because of 15space limitations or support logistics. 16 17OverlayFS resolves these debug scenarios with the _adb disable-verity_ and 18_adb remount_ commands, which set up backing storage for a writable file 19system as an upper reference, and mount the lower reference on top. 20 21Performing a remount 22-------------------- 23 24Use the following sequence to perform the remount. 25 26 $ adb root 27 $ adb disable-verity 28 $ adb reboot 29 $ adb wait-for-device 30 $ adb root 31 $ adb remount 32 33Then enter one of the following sequences: 34 35 $ adb shell stop 36 $ adb sync 37 $ adb shell start 38 $ adb reboot 39 40*or* 41 42 $ adb push <source> <destination> 43 $ adb reboot 44 45Note that you can replace these two lines in the above sequence: 46 47 $ adb disable-verity 48 $ adb reboot 49 50with this line: 51 52 $ adb remount -R 53 54**Note:** _adb remount -R_ won’t reboot if the device is already in the adb remount state. 55 56None of this changes if OverlayFS needs to be engaged. 57The decisions whether to use traditional direct file-system remount, 58or one wrapped by OverlayFS is automatically determined based on 59a probe of the file-system types and space remaining. 60 61### Backing Storage 62 63When *OverlayFS* logic is feasible, it uses either the 64**/cache/overlay/** directory for non-A/B devices, or the 65**/mnt/scratch/overlay** directory for A/B devices that have 66access to *LRAP*. 67It is also possible for an A/B device to use the system_<other> partition 68for backing storage. eg: if booting off system_a+vendor_a, use system_b. 69The backing store is used as soon as possible in the boot 70process and can occur at first stage init, or when the 71*mount_all* commands are run in init RC scripts. 72 73By attaching OverlayFS early, SEpolicy or init can be pushed and used after the exec phases of each stage. 74 75Caveats 76------- 77 78- Backing storage requires more space than immutable storage, as backing is 79 done file by file. Be mindful of wasted space. For example, defining 80 **BOARD_IMAGE_PARTITION_RESERVED_SIZE** has a negative impact on the 81 right-sizing of images and requires more free dynamic partition space. 82- The kernel requires **CONFIG_OVERLAY_FS=y**. overlayfs is used 'as is' as of 83 android 16, no modifications are required. 84- In order for overlayfs to work, overlays are mounted in the overlay_remounter 85 domain, defined here: system/sepolicy/private/overlay_remounter.te. This domain 86 must have full access to the files on the underlying volumes, add any other file 87 and directory types here 88- For devices with dynamic partitions, we use a simpler logic to decide which 89 partitions to remount, being all logical ones. In case this isn't correct, 90 we added the overlay=on and overlay=off mount flags to allow detailed control. 91- _adb enable-verity_ frees up OverlayFS and reverts the device to the state 92 prior to content updates. The update engine performs a full OTA. 93- _adb remount_ overrides are incompatible with OTA resources, so the update 94 engine may not run if fs_mgr_overlayfs_is_setup() returns true. 95- If a dynamic partition runs out of space, making a logical partition larger 96 may fail because of the scratch partition. If this happens, clear the scratch 97 storage by running either either _fastboot flashall_ or _adb enable-verity_. 98 Then reinstate the overrides and continue. 99- For implementation simplicity on retrofit dynamic partition devices, 100 take the whole alternate super (eg: if "*a*" slot, then the whole of 101 "*system_b*"). 102 Since landing a filesystem on the alternate super physical device 103 without differentiating if it is setup to support logical or physical, 104 the alternate slot metadata and previous content will be lost. 105- There are other subtle caveats requiring complex logic to solve. 106 Have evaluated them as too complex or not worth the trouble, please 107 File a bug if a use case needs to be covered. 108 - The backing storage is treated fragile, if anything else has 109 issue with the space taken, the backing storage will be cleared 110 out and we reserve the right to not inform, if the layering 111 does not prevent any messaging. 112 - Space remaining threshold is hard coded. If 1% or more space 113 still remains, OverlayFS will not be used, yet that amount of 114 space remaining is problematic. 115 - Flashing a partition via bootloader fastboot, as opposed to user 116 space fastbootd, is not detected, thus a partition may have 117 override content remaining. adb enable-verity to wipe. 118 - Space is limited, there is near unlimited space on userdata, 119 we have made an architectural decision to not utilize 120 /data/overlay/ at this time. Acquiring space to use for 121 backing remains an ongoing battle. 122 - First stage init, or ramdisk, can not be overriden. 123 - Backing storage will be discarded or ignored on errors, leading 124 to confusion. When debugging using **adb remount** it is 125 currently advised to confirm update is present after a reboot 126 to develop confidence. 127- File bugs or submit fixes for review. 128