1Triggering the GC to reclaim non-Java memory 2-------------------------------------------- 3 4Android applications and libraries commonly allocate "native" (i.e. C++) objects that are 5effectively owned by a Java object, and reclaimed when the garbage collector determines that the 6owning Java object is no longer reachable. Various mechanisms are used to accomplish this. These 7include traditional Java finalizers, more modern Java `Cleaner`s, Android's `SystemCleaner` and, 8for platform code, `NativeAllocationRegistry`. Internally, these all rely on the garbage 9collector's processing of `java.lang.ref.Reference`s. 10 11Historically, we have encountered issues when large volumes of such "native" objects are owned by 12Java objects of significantly smaller size. The Java garbage collector normally decides when to 13collect based on occupancy of the Java heap. It would not collect if there are few bytes of newly 14allocated Java objects in the Java heap, even if they "own" large amounts of C++ memory, which 15cannot be reclaimed until the Java objects are reclaimed. 16 17This led to the development of the `VMRuntime.registerNativeAllocation` API, and eventually the 18`NativeAllocationRegistry` API. Both of these allow the programmer to inform the ART GC that a 19certain amount of C++ memory had been allocated, and could only be reclaimed as the result of a 20Java GC. 21 22This had the advantage that the GC would theoretically know exactly how much native memory was 23owned by Java objects. However, its major problem was that registering the exact size of such 24native objects frequently turned out to be impractical. Often a Java object does not just own a 25single native object, but instead owns a complex native data structure composed of many objects in 26the C++ heap. Their sizes commonly depended on prior computations by C++ code. Some of these might 27be shared between Java objects, and could only be reclaimed when all of those Java objects became 28unreachable. Often at least some of the native objects were allocated by third-party libraries 29that did not make the sizes of its internal objects available to clients. 30 31In extreme cases, underestimation of native object sizes could cause native memory use to be so 32excessive that the device would become unstable. At one time, a particularly nasty arithmetic 33expression would cause the Google Calculator app to allocate sufficiently large native arrays of 34digits backing Java `BigInteger`s to force a restart of system processes. (This has since been 35addressed in other ways as well.) 36 37Thus we switched to a scheme in which sizes of native objects are commonly no longer directly 38provided by clients. The GC instead occasionally calls `mallinfo()` to determine how much native 39memory has been allocated, and assumes that any of this may need the collector's help to reclaim. 40C++ memory allocated by means other than `malloc`/`new` and owned by Java objects should still be 41explicitly registered. This loses information about Java ownership, but results in much more 42accurate information about the total size of native objects, something that was previously very 43difficult to approximate, even to within an order of magnitude. 44 45The triggering heuristic 46------------------------ 47 48We compute the total amount of native memory allocated as the sum of 49 501. memory allocated, but not yet deallocated, by the system memory allocator, as reported by 51 `mallinfo()`, and 52 532. the number of bytes registered via `VMRuntime.registerNativeAllocation()` and not yet 54 unregistered via `VMRuntime.registerNativeFree()`. This includes non-malloc-allocated objects 55 allocated via `NativeAllocationRegistry`. 56 57Though we use mallinfo() to track native allocation, this call itself can be expensive, and thus 58we perform this check fairly rarely. More precisely, we do so only after the application has 59called `NativeAllocationRegistry.registerNativeAllocation()` a certain number of times or 60with a sufficiently large argument, or after `VMRuntime.registerNativeAllocation` is called. 61Thus an application not using these APIs, e.g. because it is running almost entirely native 62code, may never do so. This can be useful, in that an application that runs basically only 63native code, and thus deallocates its own native memory, does not trigger the GC. 64 65The actual computation for triggering a native-allocation-GC is performed by 66`Heap::NativeMemoryOverTarget()`. This computes and compares two quantities: 67 681. An adjusted heap size for GC triggering. This consists of the Java heap size at which we would 69 normally trigger a GC plus an allowance for native heap size. This allowance currently consists 70 of one half (background processes) or three halves (foreground processes) of 71 `NativeAllocationGcWatermark()`. The latter is `HeapMaxFree` (typically 32MB) plus 1/8 of the 72 currently targeted heap size. For a foreground process, this allowance would typically be in 73 the 50-100 MB range for something other than a low-end device. 74 752. An adjusted count of current bytes allocated. This is basically the number of bytes currently 76 allocated in the Java heap, plus half the net number of native bytes allocated since the last 77 GC. The latter is computed as the change since the last GC in the total-bytes-allocated 78 obtained from `mallinfo()` plus `native_bytes_registered_`. The `native_bytes_registered_` 79 field tracks the bytes explicitly registered, and not yet unregistered via 80 `registerNativeAllocation` APIs. It excludes bytes registered as malloc-allocated via 81 `NativeAllocationRegistry`. (The computation also considers the total amount of native memory 82 currently allocated by this metric, as opposed to the change since the last GC. But that is 83 currently de-weighted to the point of insignificance.) 84 85A background GC is triggered if the second quantity exceeds the first. A stop-the-world-GC is 86triggered if the second quantity is at least 4 times larger than the first, and the native heap 87currently occupies a large fraction of device memory, suggesting that the GC is falling behind and 88endangering device usability. 89 90The fact that we consider both Java and native memory use at once means that we are more likely to 91trigger a native GC when we are closer to the normal Java GC threshold. 92 93The actual use of `mallinfo()` to compute native heap occupancy reflects experiments with 94different `libc` implementations. These have different performance characteristics, and sometimes 95disagree on the interpretation of `struct mallinfo` fields. We believe our current implementation 96is solid with scudo and jemalloc on Android, and minimally usable for testing elsewhere. 97 98(Some of this assumes typical current (May 2024) configuration constants, and may need to be 99updated.) 100 101Workaround for excessive native GCs 102----------------------------------- 103 104If an application routinely allocates and deallocates large amounts of memory without requiring 105the GC, it may be preferable to bypass the system allocator, for example by using mmap directly. 106This will avoid unnecessary GC triggering. This is clearly much more convenient for a small number 107of large allocations than for small allocations. It could also be addressed with a new ART API, 108but so far we have not found this necessary. 109