Malloc Debug
============

Malloc debug is a method of debugging native memory problems. It can help
detect memory corruption, memory leaks, and use after free issues.

This documentation describes how to enable this feature on Android N or later
versions of the Android OS.

The documentation for malloc debug on older versions of Android is
[here](README_marshmallow_and_earlier.md).

In order to enable malloc debug, you must be able to set special system
properties using the setprop command from the shell. This requires the
ability to run as root on the device.

When malloc debug is enabled, it works by adding a shim layer that replaces
the normal allocation calls. The replaced calls are:

* `malloc`
* `free`
* `calloc`
* `realloc`
* `posix_memalign`
* `memalign`
* `malloc_usable_size`

On 32 bit systems, these two deprecated functions are also replaced:

* `pvalloc`
* `valloc`

Any errors detected by the library are reported in the log.

Controlling Malloc Debug Behavior
---------------------------------
Malloc debug is controlled by individual options. Each option can be enabled
individually, or in a group of other options. Every single option can be
combined with every other option.

Option Descriptions
-------------------
### front\_guard[=SIZE\_BYTES]
Enables a small buffer placed before the allocated data. This is an attempt
to find memory corruption occuring to a region before the original allocation.
On first allocation, this front guard is written with a specific pattern (0xaa).
When the allocation is freed, the guard is checked to verify it has not been
modified. If any part of the front guard is modified, an error will be reported
in the log indicating what bytes changed.

If the backtrace option is also enabled, then any error message will include
the backtrace of the allocation site.

If SIZE\_BYTES is present, it indicates the number of bytes in the guard.
The default is 32 bytes, the max bytes is 16384. SIZE\_BYTES will be
padded so that it is a multiple of 8 bytes on 32 bit systems and 16 bytes
on 64 bit systems to make sure that the allocation returned is aligned
properly.

This option adds a special header to all allocations that contains the guard
and information about the original allocation.

Example error:

    04-10 12:00:45.621  7412  7412 E malloc_debug: +++ ALLOCATION 0x12345678 SIZE 100 HAS A CORRUPTED FRONT GUARD
    04-10 12:00:45.622  7412  7412 E malloc_debug:   allocation[-32] = 0x00 (expected 0xaa)
    04-10 12:00:45.622  7412  7412 E malloc_debug:   allocation[-15] = 0x02 (expected 0xaa)

### rear\_guard[=SIZE\_BYTES]
Enables a small buffer placed after the allocated data. This is an attempt
to find memory corruption occuring to a region after the original allocation.
On first allocation, this rear guard is written with a specific pattern (0xbb).
When the allocation is freed, the guard is checked to verify it has not been
modified. If any part of the rear guard is modified, an error will be reported
in the log indicating what bytes changed.

If SIZE\_BYTES is present, it indicates the number of bytes in the guard.
The default is 32 bytes, the max bytes is 16384.

This option adds a special header to all allocations that contains
information about the original allocation.

Example error:

    04-10 12:00:45.621  7412  7412 E malloc_debug: +++ ALLOCATION 0x12345678 SIZE 100 HAS A CORRUPTED REAR GUARD
    04-10 12:00:45.622  7412  7412 E malloc_debug:   allocation[130] = 0xbf (expected 0xbb)
    04-10 12:00:45.622  7412  7412 E malloc_debug:   allocation[131] = 0x00 (expected 0xbb)

### guard[=SIZE\_BYTES]
Enables both a front guard and a rear guard on all allocations.

If SIZE\_BYTES is present, it indicates the number of bytes in both guards.
The default is 32 bytes, the max bytes is 16384.

### backtrace[=MAX\_FRAMES]
Enable capturing the backtrace of each allocation site.
This option will slow down allocations by an order of magnitude. If the
system runs too slowly with this option enabled, decreasing the maximum number
of frames captured will speed the allocations up.

Note that any backtrace frames that occur within the malloc backtrace library
itself are not recorded.

If MAX\_FRAMES is present, it indicates the maximum number of frames to
capture in a backtrace. The default is 16 frames, the maximumum value
this can be set to is 256.

This option adds a special header to all allocations that contains the
backtrace and information about the original allocation.

### backtrace\_enable\_on\_signal[=MAX\_FRAMES]
Enable capturing the backtrace of each allocation site. If the
backtrace capture is toggled when the process receives the signal
SIGRTMAX - 19 (which is 45 on most Android devices). When this
option is used alone, backtrace capture starts out disabled until the signal
is received. If both this option and the backtrace option are set, then
backtrace capture is enabled until the signal is received.

If MAX\_FRAMES is present, it indicates the maximum number of frames to
capture in a backtrace. The default is 16 frames, the maximumum value
this can be set to is 256.

This option adds a special header to all allocations that contains the
backtrace and information about the original allocation.

### fill\_on\_alloc[=MAX\_FILLED\_BYTES]
Any allocation routine, other than calloc, will result in the allocation being
filled with the value 0xeb. When doing a realloc to a larger size, the bytes
above the original usable size will be set to 0xeb.

If MAX\_FILLED\_BYTES is present, it will only fill up to the specified number
of bytes in the allocation. The default is to fill the entire allocation.

### fill\_on\_free[=MAX\_FILLED\_BYTES]
When an allocation is freed, fill it with 0xef.

If MAX\_FILLED\_BYTES is present, it will only fill up to the specified number
of bytes in the allocation. The default is to fill the entire allocation.

### fill[=MAX\_FILLED\_BYTES]
This enables both the fill\_on\_alloc option and the fill\_on\_free option.

If MAX\_FILLED\_BYTES is present, it will only fill up to the specified number
of bytes in the allocation. The default is to fill the entire allocation.

### expand\_alloc[=EXPAND\_BYTES]
Add an extra amount to allocate for every allocation.

If XX is present, it is the number of bytes to expand the allocation by.
The default is 16 bytes, the max bytes is 16384.

### free\_track[=ALLOCATION\_COUNT]
When a pointer is freed, do not free the memory right away, but add it to
a list of freed allocations. In addition to being added to the list, the
entire allocation is filled with the value 0xef, and the backtrace at
the time of the free is recorded. The backtrace recording is completely
separate from the backtrace option, and happens automatically if this
option is enabled. By default, a maximum of 16 frames will be recorded,
but this value can be changed using the free\_track\_backtrace\_num\_frames
option. It can also be completely disabled by setting the option to zero.
See the full description of this option below.

When the list is full, an allocation is removed from the list and is
checked to make sure that none of the contents have been modified since
being placed on the list. When the program terminates, all of the allocations
left on the list are verified.

If ALLOCATION\_COUNT is present, it indicates the total number of allocations
in the list. The default is to record 100 freed allocations, the max
allocations to record is 16384.

This option adds a special header to all allocations that contains
information about the original allocation.

Example error:

    04-15 12:00:31.304  7412  7412 E malloc_debug: +++ ALLOCATION 0x12345678 USED AFTER FREE
    04-15 12:00:31.305  7412  7412 E malloc_debug:   allocation[20] = 0xaf (expected 0xef)
    04-15 12:00:31.305  7412  7412 E malloc_debug:   allocation[99] = 0x12 (expected 0xef)
    04-15 12:00:31.305  7412  7412 E malloc_debug: Backtrace at time of free:
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #00  pc 00029310  /system/lib/libc.so
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #01  pc 00021438  /system/lib/libc.so (newlocale+160)
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #02  pc 000a9e38  /system/lib/libc++.so
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #03  pc 000a28a8  /system/lib/libc++.so

In addition, there is another type of error message that can occur if
an allocation has a special header applied, and the header is corrupted
before the verification occurs. This is the error message that will be found
in the log:

    04-15 12:00:31.604  7412  7412 E malloc_debug: +++ ALLOCATION 0x12345678 HAS CORRUPTED HEADER TAG 0x1cc7dc00 AFTER FREE

### free\_track\_backtrace\_num\_frames[=MAX\_FRAMES]
This option only has meaning if free\_track is set. It indicates how many
backtrace frames to capture when an allocation is freed.

If MAX\_FRAMES is present, it indicates the number of frames to capture.
If the value is set to zero, then no backtrace will be captured when the
allocation is freed. The default is to record 16 frames, the max number of
frames to to record is 256.

### leak\_track
Track all live allocations. When the program terminates, all of the live
allocations will be dumped to the log. If the backtrace option was enabled,
then the log will include the backtrace of the leaked allocations. This
option is not useful when enabled globally because a lot of programs do not
free everything before the program terminates.

This option adds a special header to all allocations that contains
information about the original allocation.

Example leak error found in the log:

    04-15 12:35:33.304  7412  7412 E malloc_debug: +++ APP leaked block of size 100 at 0x2be3b0b0 (leak 1 of 2)
    04-15 12:35:33.304  7412  7412 E malloc_debug: Backtrace at time of allocation:
    04-15 12:35:33.305  7412  7412 E malloc_debug:           #00  pc 00029310  /system/lib/libc.so
    04-15 12:35:33.305  7412  7412 E malloc_debug:           #01  pc 00021438  /system/lib/libc.so (newlocale+160)
    04-15 12:35:33.305  7412  7412 E malloc_debug:           #02  pc 000a9e38  /system/lib/libc++.so
    04-15 12:35:33.305  7412  7412 E malloc_debug:           #03  pc 000a28a8  /system/lib/libc++.so
    04-15 12:35:33.305  7412  7412 E malloc_debug: +++ APP leaked block of size 24 at 0x7be32380 (leak 2 of 2)
    04-15 12:35:33.305  7412  7412 E malloc_debug: Backtrace at time of allocation:
    04-15 12:35:33.305  7412  7412 E malloc_debug:           #00  pc 00029310  /system/lib/libc.so
    04-15 12:35:33.305  7412  7412 E malloc_debug:           #01  pc 00021438  /system/lib/libc.so (newlocale+160)
    04-15 12:35:33.305  7412  7412 E malloc_debug:           #02  pc 000a9e38  /system/lib/libc++.so
    04-15 12:35:33.305  7412  7412 E malloc_debug:           #03  pc 000a28a8  /system/lib/libc++.so

### record\_allocs[=TOTAL\_ENTRIES]
Keep track of every allocation/free made on every thread and dump them
to a file when the signal SIGRTMAX - 18 (which is 46 on most Android devices)
is received.

If TOTAL\_ENTRIES is set, then it indicates the total number of
allocation/free records that can be retained. If the number of records
reaches the TOTAL\_ENTRIES value, then any further allocations/frees are
not recorded. The default value is 8,000,000 and the maximum value this
can be set to is 50,000,000.

Once the signal is received, and the current records are written to the
file, all current records are deleted. Any allocations/frees occuring while
the data is being dumped to the file are ignored.

**NOTE**: This option is not available until the O release of Android.

The allocation data is written in a human readable format. Every line begins
with the THREAD\_ID returned by gettid(), which is the thread that is making
the allocation/free. If a new thread is created, no special line is added
to the file. However, when a thread completes, a special entry is added to
the file indicating this.

The thread complete line is:

**THREAD\_ID**: thread\_done 0x0

Example:

    187: thread_done 0x0

Below is how each type of allocation/free call ends up in the file dump.

pointer = malloc(size)

**THREAD\_ID**: malloc pointer size

Example:

    186: malloc 0xb6038060 20

free(pointer)

**THREAD\_ID**: free pointer

Example:

    186: free 0xb6038060

pointer = calloc(nmemb, size)

**THREAD\_ID**: calloc pointer nmemb size

Example:

    186: calloc 0xb609f080 32 4

new\_pointer = realloc(old\_pointer, size)

**THREAD\_ID**: realloc new\_pointer old\_pointer size

Example:

    186: realloc 0xb609f080 0xb603e9a0 12

pointer = memalign(alignment, size)

**THREAD\_ID**: memalign pointer alignment size

posix\_memalign(&pointer, alignment, size)

**THREAD\_ID**: memalign pointer alignment size

Example:

    186: memalign 0x85423660 16 104

pointer = valloc(size)

**THREAD\_ID**: memalign pointer 4096 size

Example:

    186: memalign 0x85423660 4096 112

pointer = pvalloc(size)

**THREAD\_ID**: memalign pointer 4096 <b>SIZE\_ROUNDED\_UP\_TO\_4096</b>

Example:

    186: memalign 0x85423660 4096 8192

### record\_allocs\_file[=FILE\_NAME]
This option only has meaning if record\_allocs is set. It indicates the
file where the recorded allocations will be found.

If FILE\_NAME is set, then it indicates where the record allocation data
will be placed.

**NOTE**: This option is not available until the O release of Android.

Additional Errors
-----------------
There are a few other error messages that might appear in the log.

### Use After Free
    04-15 12:00:31.304  7412  7412 E malloc_debug: +++ ALLOCATION 0x12345678 USED AFTER FREE (free)
    04-15 12:00:31.305  7412  7412 E malloc_debug: Backtrace of original free:
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #00  pc 00029310  /system/lib/libc.so
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #01  pc 00021438  /system/lib/libc.so (newlocale+160)
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #02  pc 000a9e38  /system/lib/libc++.so
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #03  pc 000a28a8  /system/lib/libc++.so
    04-15 12:00:31.305  7412  7412 E malloc_debug: Backtrace at time of failure:
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #00  pc 00029310  /system/lib/libc.so
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #01  pc 00021438  /system/lib/libc.so (newlocale+160)
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #02  pc 000a9e38  /system/lib/libc++.so
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #03  pc 000a28a8  /system/lib/libc++.so

This indicates that code is attempting to free an already freed pointer. The
name in parenthesis indicates that the application called the function
*free* with the bad pointer.

For example, this message:

    04-15 12:00:31.304  7412  7412 E malloc_debug: +++ ALLOCATION 0x12345678 USED AFTER FREE (realloc)

Would indicate that the application called the *realloc* function
with an already freed pointer.

### Invalid Tag
    04-15 12:00:31.304  7412  7412 E malloc_debug: +++ ALLOCATION 0x12345678 HAS INVALID TAG 1ee7d000 (malloc_usable_size)
    04-15 12:00:31.305  7412  7412 E malloc_debug: Backtrace at time of failure:
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #00  pc 00029310  /system/lib/libc.so
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #01  pc 00021438  /system/lib/libc.so (newlocale+160)
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #02  pc 000a9e38  /system/lib/libc++.so
    04-15 12:00:31.305  7412  7412 E malloc_debug:           #03  pc 000a28a8  /system/lib/libc++.so

This indicates that a function (malloc\_usable\_size) was called with
a pointer that is either not allocated memory, or that the memory of
the pointer has been corrupted.

As with the other error message, the function in parenthesis is the
function that was called with the bad pointer.

Examples
========
Enable backtrace tracking of all allocation for all processes:

    adb shell stop
    adb shell setprop libc.debug.malloc.options backtrace
    adb shell start

Enable backtrace tracking for a specific process (ls):

    adb shell setprop libc.debug.malloc.options backtrace
    adb shell setprop libc.debug.malloc.program ls
    adb shell ls

Enable backtrace tracking for the zygote and zygote based processes:

    adb shell stop
    adb shell setprop libc.debug.malloc.program app_process
    adb shell setprop libc.debug.malloc.options backtrace
    adb shell start

Enable multiple options (backtrace and guards):

    adb shell stop
    adb shell setprop libc.debug.malloc.options "\"backtrace guards\""
    adb shell start

Note: The two levels of quoting in the adb shell command is necessary.
The outer layer of quoting is for the shell on the host, to ensure that the
inner layer of quoting is sent to the device, to make 'backtrace guards'
a single argument.

Enable malloc debug using an environment variable (pre-O Android release):

    adb shell
    # setprop libc.debug.malloc.env_enabled 1
    # setprop libc.debug.malloc.options backtrace
    # export LIBC_DEBUG_MALLOC_ENABLE=1
    # ls

Enable malloc debug using an environment variable (Android O or later):

    adb shell
    # export LIBC_DEBUG_MALLOC_OPTIONS=backtrace
    # ls

Any process spawned from this shell will run with malloc debug enabled
using the backtrace option.

    adb shell stop
    adb shell setprop libc.debug.malloc.options backtrace
    adb shell start
    adb shell am dumpheap -n <PID_TO_DUMP> /data/local/tmp/heap.txt

It is possible to use the backtrace\_enable\_on\_signal option as well,
but, obviously, it must be enabled through the signal before the file will
contain any data.

To analyze the data produced by the dumpheap command, run this script:

    development/scripts/native_heapdump_viewer.py

In order for the script to properly symbolize the stacks in the file,
make sure the script is executed from the tree that built the image.
Below is an example of how to execute the script using the dump created by the
above command:

    adb shell pull /data/local/tmp/heap.txt .
    development/scripts/native_heapdump_viewer.py heap.txt > heap_info.txt

Enable malloc debug for a specific program/application (Android O or later):

    adb shell setprop wrap.<APP> '"LIBC_DEBUG_MALLOC_OPTIONS=backtrace logwrapper"'

For example, to enable malloc debug for the google search box (Android O or later):

    adb shell setprop wrap.com.google.android.googlequicksearchbox '"LIBC_DEBUG_MALLOC_OPTIONS=backtrace logwrapper"'
    adb shell am force-stop com.google.android.googlequicksearchbox

NOTE: On pre-O versions of the Android OS, property names had a length limit
of 32. This meant that to create a wrap property with the name of the app, it
was necessary to truncate the name to fit. On O, property names can be
an order of magnitude larger, so there should be no need to truncate the name
at all.

Native Memory Tracking using libc Callbacks
-------------------------------------------
Malloc debug can be used to get information on all of the live allocations
in a process. The libc library in Android exports two calls that can be
used to gather this data from a process. This tracking can be enabled using
either the backtrace option or the backtrace\_enabled\_on\_signal option.

The function to gather the data:

`extern "C" void get_malloc_leak_info(uint8_t** info, size_t* overall_size, size_t* info_size, size_t* total_memory, size_t* backtrace_size);`

*info* is set to a buffer allocated by the call that contains all of
the allocation information.
*overall\_size* is set to the total size of the buffer returned. If this
*info\_size*
value is zero, then there are no allocation being tracked.
*total\_memory* is set to the sum of all allocation sizes that are live at
the point of the function call. This does not include the memory allocated
by the malloc debug library itself.
*backtrace\_size* is set to the maximum number of backtrace entries
that are present for each allocation.

In order to free the buffer allocated by the function, call:

`extern "C" void free_malloc_leak_info(uint8_t* info);`

### Format of info Buffer
    size_t size_of_original_allocation
    size_t num_allocations
    uintptr_t pc1
    uintptr_t pc2
    uintptr_t pc3
    .
    .
    .

The number of *uintptr\_t* values is determined by the value
*backtrace\_size* as returned by the original call to
*get\_malloc\_leak\_info*. This value is not variable, it is the same
for all the returned data. The value
*num\_allocations* contains the total number of allocations with the same
backtrace and size as this allocation. On Android Nougat, this value was
incorrectly set to the number of frames in the backtrace.
Each *uintptr\_t* is a pc of the callstack. If the total number
of backtrace entries is less than *backtrace\_size*, the rest of the
entries are zero.
The calls from within the malloc debug library are automatically removed.

For 32 bit systems, *size\_t* and *uintptr\_t* are both 4 byte values.

For 64 bit systems, *size\_t* and *uintptr\_t* are both 8 byte values.

The total number of these structures returned in *info* is
*overall\_size* divided by *info\_size*.

Note, the size value in each allocation data structure will have bit 31 set
if this allocation was created in a process forked from the Zygote process.
This helps to distinguish between native allocations created by the application.

Malloc Debug
============

Malloc debug is a method of debugging native memory problems. It can help
detect memory corruption, memory leaks, and use after free issues.

This documentation describes how to enable this feature on versions of
the Android OS, Marshmallow or older. Note: malloc debug was full of bugs
and was not fully functional until KitKat, so using it on a version older
than that is not guaranteed to work at all.

The documentation for malloc debug on newer versions of Android is
[here](README.md).

On these old versions of the OS, you must be able to set system properties
using the setprop command from the shell. This requires the ability to
run as root on the device.

When malloc debug is enabled, it works by adding a shim layer that replaces
the normal allocation calls. The replaced calls are:

* `malloc`
* `free`
* `calloc`
* `realloc`
* `posix_memalign`
* `memalign`
* `malloc_usable_size`

On 32 bit systems, these two deprecated functions are also replaced:

* `pvalloc`
* `valloc`

Any errors detected by the library are reported in the log.

Controlling Malloc Debug Behavior
---------------------------------
Malloc debug is controlled by a system property that takes a numeric value
named libc.debug.malloc. It has only a few distinct modes that enables a
set of different malloc debug checks at once.

Value 1
--------
When enabled, this value creates a special header to all allocations
that contains information about the allocation.

### Backtrace at Allocation Creation
Enable capturing the backtrace of each allocation site. Only the
first 16 frames of the backtrace will be captured.
This option will slow down allocations by an order of magnitude, and
might cause timeouts when trying to start a device.

### Track Live Allocations
All of the currently live allocations will be tracked and can be retrieved
by a call to get\_malloc\_leak\_info (see README\_api.md for details).

Note: If multiple allocations have the same exact backtrace, then only one
entry is returned in the list.

Value 5
-------
When enabled, this value does not create a special header. It only modifies
the content of allocations.

Whenever an allocation is created, initialize the data with a known
pattern (0xeb). This does not happen for the calloc calls.
Whenever an allocation is freed, write a known pattern over the data (0xef).

Value 10
--------
When enabled, this value creates a special header to all allocations
that contains information about the allocation.

This value enables everything enabled with value 1 plus these other options.

### Allocation Guards
A 32 byte buffer is placed before the returned allocation (known as
a front guard). This buffer is filled with the pattern (0xaa). In addition,
a 32 byte buffer is placed after the data for the returned allocation (known
as a rear guard). This buffer is filled with the pattern (0xbb).

When the allocation is freed, both of these guards are verified to contain
the expected patterns. If not, then an error message is printed to the log.

### Free Memory Tracking
When a pointer is freed, do not free the memory right away, but add it to
a list of freed allocations. In addition to being added to the list, the
entire allocation is filled with the value 0xef, and the backtrace at
the time of the free is recorded. As with the backtrace on allocation,
only up to 16 frames will be recorded.

When the list of freed allocations reaches 100, the oldest allocation
on the list is removed and verified that it still contains the pattern 0xef.
If the entire allocation is not filled with this value, an error is printed
to the log.

### Log Leaks
When the program completes, all of the allocations that are still live
are printed to the log as leaks. This isn't very useful since it tends
to display a lot of false positive because many programs do not free
everything before terminating.

Option 20
---------
Do not use this option value, it only works on the emulator. It has not
been verified, so it may or may not work.

Enable on Certain Processes
---------------------------
Using the special system property, libc.debug.malloc.program, will
cause malloc debug to only be used on processes with that name. For example,
if the property is set to ls, then only the program named ls will have malloc
debug enabled.

Examples
========
Enable malloc debug for all allocations for all processes:

    adb shell stop
    adb shell setprop libc.debug.malloc 1
    adb shell start

Enable malloc debug for a particular process:

    adb shell setprop libc.debug.malloc.program ls
    adb shell setprop libc.debug.malloc 10
    adb shell ls /data/local/tmp