• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1page.title=Reading Bug Reports
2@jd:body
3
4<!--
5    Copyright 2015 The Android Open Source Project
6
7    Licensed under the Apache License, Version 2.0 (the "License");
8    you may not use this file except in compliance with the License.
9    You may obtain a copy of the License at
10
11        http://www.apache.org/licenses/LICENSE-2.0
12
13    Unless required by applicable law or agreed to in writing, software
14    distributed under the License is distributed on an "AS IS" BASIS,
15    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16    See the License for the specific language governing permissions and
17    limitations under the License.
18-->
19<div id="qv-wrapper">
20  <div id="qv">
21    <h2>In this document</h2>
22    <ol id="auto-toc">
23    </ol>
24  </div>
25</div>
26
27<p>Bugs are a reality in any type of development&#8212;and bug reports are
28critical to identifying and solving problems. All versions of Android support
29capturing bug reports with <a href="http://developer.android.com/tools/help/adb.html">Android
30Debug Bridge (adb)</a>; Android versions 4.2 and higher support a
31<a href="http://developer.android.com/tools/device.html#developer-device-options">Developer
32Option</a> for taking bug reports and sharing via email, Drive, etc.</p>
33
34<p>Android bug reports contain <code>dumpsys</code>, <code>dumpstate</code>, and
35<code>logcat</code> data in text (.txt) format, enabling you to easily search
36for specific content. The following sections detail bug report components,
37describe common problems, and give helpful tips and <code>grep</code> commands
38for finding logs associated with those bugs. Most sections also include examples
39for <code>grep</code> command and output and/or <code>dumpsys</code> output.</p>
40
41<h2 id="logcat">Logcat</h2>
42<p>The <code>logcat</code> log is a string-based dump of all <code>logcat</code>
43information. The <strong>system</strong> part is reserved for the framework and
44has a longer history than <strong>main</strong> which contains everything else.
45Each line starts with <code>timestamp PID TID log-level</code>.</p>
46
47<div class="toggle-content closed">
48  <p><a href="#" onclick="return toggleContent(this)">
49    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
50    <strong><span class="toggle-content-text">Show example</span>
51    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
52  </a></p>
53
54  <div class="toggle-content-toggleme">
55    <p><pre>------ SYSTEM LOG (logcat -v threadtime -d *:v) ------
56--------- beginning of system
57<i>Blah</i>
58<i>Blah</i>
59<i>Blah</i>
60
61--------- beginning of main
62<i>Blah </i>
63<i>Blah</i>
64<i>Blah</i></pre></p>
65  </div>
66</div>
67
68<h3 id="event-log">Viewing the event log</h3>
69<p>This log contains string representations of binary-formatted log messages. It
70is less noisy than the <code>logcat</code> log but also a little harder to read.
71When viewing event logs, you can search this section for specific process ID
72(PID) to see what a process has been doing. The basic format is:
73<code>timestamp PID TID log-level log-tag tag-values</code>.</p>
74
75<p>Log levels include the following:</p>
76<ul>
77<li>V: verbose</li>
78<li>D: debug</li>
79<li>I: information</li>
80<li>W: warning</li>
81<li>E: error</li>
82</ul>
83
84<div class="toggle-content closed">
85  <p><a href="#" onclick="return toggleContent(this)">
86    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
87    <strong><span class="toggle-content-text">Show example</span>
88    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
89  </a></p>
90
91  <div class="toggle-content-toggleme">
92    <p><pre>------ EVENT LOG (logcat -b events -v threadtime -d *:v) ------
9309-28 13:47:34.179   785  5113 I am_proc_bound: [0,23054,com.google.android.gms.unstable]
9409-28 13:47:34.777   785  1975 I am_proc_start: [0,23134,10032,com.android.chrome,broadcast,com.android.chrome/org.chromium.chrome.browser.precache.PrecacheServiceLauncher]
9509-28 13:47:34.806   785  2764 I am_proc_bound: [0,23134,com.android.chrome]
96...</pre></p>
97  </div>
98</div>
99
100<h2 id="anrs-deadlocks">ANRs and deadlocks</h2>
101<p>Bugreports can help you identify what's causing
102<a href="http://developer.android.com/training/articles/perf-anr.html">Application
103Not Responding (ANR)</a> errors and deadlock events.</p>
104
105<h3 id="determine-anr-app">Identifying unresponsive apps</h3>
106<p>When an application does not respond within a certain time, usually due to a
107blocked or busy main thread, the system kills the process and dumps the stack to
108<code>/data/anr</code>. To discover the culprit behind an ANR, grep for
109<code>am_anr</code> in the binary event log.</p>
110
111<div class="toggle-content closed">
112  <p><a href="#" onclick="return toggleContent(this)">
113    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
114    <strong><span class="toggle-content-text">Show example</span>
115    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
116  </a></p>
117  <div class="toggle-content-toggleme">
118    <p><pre>grep "am_anr" bugreport-2015-10-01-18-13-48.txt
11910-01 18:12:49.599  4600  4614 I am_anr  : [0,29761,com.google.android.youtube,953695941,executing service com.google.android.youtube/com.google.android.apps.youtube.app.offline.transfer.OfflineTransferService]
12010-01 18:14:10.211  4600  4614 I am_anr  : [0,30363,com.google.android.apps.plus,953728580,executing service com.google.android.apps.plus/com.google.android.apps.photos.service.PhotosService]</pre></p>
121  </div>
122</div>
123<p></p>
124<p>You can also grep for <code>ANR in</code> in the <code>logcat</code>log,
125which contains more information about what was using CPU at the time of the ANR.
126</p>
127
128<div class="toggle-content closed">
129  <p><a href="#" onclick="return toggleContent(this)">
130    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
131    <strong><span class="toggle-content-text">Show example</span>
132    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
133  </a></p>
134
135  <div class="toggle-content-toggleme">
136    <p><pre>grep "ANR in" bugreport-2015-10-01-18-13-48.txt
13710-01 18:13:11.984  4600  4614 E ActivityManager: ANR in com.google.android.youtube
13810-01 18:14:31.720  4600  4614 E ActivityManager: ANR in com.google.android.apps.plus
13910-01 18:14:31.720  4600  4614 E ActivityManager: PID: 30363
14010-01 18:14:31.720  4600  4614 E ActivityManager: Reason: executing service com.google.android.apps.plus/com.google.android.apps.photos.service.PhotosService
14110-01 18:14:31.720  4600  4614 E ActivityManager: Load: 35.27 / 23.9 / 16.18
14210-01 18:14:31.720  4600  4614 E ActivityManager: CPU usage from 16ms to 21868ms later:
14310-01 18:14:31.720  4600  4614 E ActivityManager:   74% 3361/mm-qcamera-daemon: 62% user + 12% kernel / faults: 15276 minor 10 major
14410-01 18:14:31.720  4600  4614 E ActivityManager:   41% 4600/system_server: 18% user + 23% kernel / faults: 18597 minor 309 major
14510-01 18:14:31.720  4600  4614 E ActivityManager:   32% 27420/com.google.android.GoogleCamera: 24% user + 7.8% kernel / faults: 48374 minor 338 major
14610-01 18:14:31.720  4600  4614 E ActivityManager:   16% 130/kswapd0: 0% user + 16% kernel
14710-01 18:14:31.720  4600  4614 E ActivityManager:   15% 283/mmcqd/0: 0% user + 15% kernel
148...
14910-01 18:14:31.720  4600  4614 E ActivityManager:   0.1% 27248/irq/503-synapti: 0%
15010-01 18:14:31.721  4600  4614 I ActivityManager: Killing 30363:com.google.android.apps.plus/u0a206 (adj 0): bg anr</pre></p>
151  </div>
152</div>
153
154<h3 id="find-stack-traces">Finding stack traces</h3>
155<p>You can often find stack traces that correspond to an ANR. Make sure the
156timestamp and PID on the VM traces match the ANR you are investigating, then
157check the main thread of the process. Keep in mind:</p>
158<ul>
159<li>The main thread tells you only what the thread was doing at the time of the
160ANR, which may or may not correspond to the true cause of the ANR. (The stack in
161the bug report may be innocent; something else may have been stuck for a long
162time&#8212;but not quite long enough to ANR&#8212;before becoming unstuck.)
163</li>
164<li>More than one set of stack traces (<code>VM TRACES JUST NOW</code> and
165<code>VM TRACES AT LAST ANR</code>) might exist. Make sure you are viewing the
166correct section.</li>
167</ul>
168
169<div class="toggle-content closed">
170  <p><a href="#" onclick="return toggleContent(this)">
171    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
172    <strong><span class="toggle-content-text">Show example</span>
173    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
174  </a></p>
175
176  <div class="toggle-content-toggleme">
177    <p><pre>------ VM TRACES AT LAST ANR (/data/anr/traces.txt: 2015-10-01 18:14:41) ------
178
179----- pid 30363 at 2015-10-01 18:14:11 -----
180Cmd line: com.google.android.apps.plus
181Build fingerprint: 'google/angler/angler:6.0/MDA89D/2294819:userdebug/dev-keys'
182ABI: 'arm'
183Build type: optimized
184Zygote loaded classes=3978 post zygote classes=27
185Intern table: 45068 strong; 21 weak
186JNI: CheckJNI is off; globals=283 (plus 360 weak)
187Libraries: /system/lib/libandroid.so /system/lib/libcompiler_rt.so /system/lib/libjavacrypto.so /system/lib/libjnigraphics.so /system/lib/libmedia_jni.so /system/lib/libwebviewchromium_loader.so libjavacore.so (7)
188Heap: 29% free, 21MB/30MB; 32251 objects
189Dumping cumulative Gc timings
190Total number of allocations 32251
191Total bytes allocated 21MB
192Total bytes freed 0B
193Free memory 9MB
194Free memory until GC 9MB
195Free memory until OOME 490MB
196Total memory 30MB
197Max memory 512MB
198Zygote space size 1260KB
199Total mutator paused time: 0
200Total time waiting for GC to complete: 0
201Total GC count: 0
202Total GC time: 0
203Total blocking GC count: 0
204Total blocking GC time: 0
205
206suspend all histogram:  Sum: 119.728ms 99% C.I. 0.010ms-107.765ms Avg: 5.442ms Max: 119.562ms
207DALVIK THREADS (12):
208"Signal Catcher" daemon prio=5 tid=2 Runnable
209  | group="system" sCount=0 dsCount=0 obj=0x12c400a0 self=0xef460000
210  | sysTid=30368 nice=0 cgrp=default sched=0/0 handle=0xf4a69930
211  | state=R schedstat=( 9021773 5500523 26 ) utm=0 stm=0 core=1 HZ=100
212  | stack=0xf496d000-0xf496f000 stackSize=1014KB
213  | held mutexes= "mutator lock"(shared held)
214  native: #00 pc 0035a217  /system/lib/libart.so (art::DumpNativeStack(std::__1::basic_ostream&lt;char, std::__1::char_traits&lt;char&gt; &gt;&, int, char const*, art::ArtMethod*, void*)+126)
215  native: #01 pc 0033b03b  /system/lib/libart.so (art::Thread::Dump(std::__1::basic_ostream&lt;char, std::__1::char_traits&lt;char&gt; &gt;&) const+138)
216  native: #02 pc 00344701  /system/lib/libart.so (art::DumpCheckpoint::Run(art::Thread*)+424)
217  native: #03 pc 00345265  /system/lib/libart.so (art::ThreadList::RunCheckpoint(art::Closure*)+200)
218  native: #04 pc 00345769  /system/lib/libart.so (art::ThreadList::Dump(std::__1::basic_ostream&lt;char, std::__1::char_traits&lt;char&gt; &gt;&)+124)
219  native: #05 pc 00345e51  /system/lib/libart.so (art::ThreadList::DumpForSigQuit(std::__1::basic_ostream&lt;char, std::__1::char_traits&lt;char&gt; &gt;&)+312)
220  native: #06 pc 0031f829  /system/lib/libart.so (art::Runtime::DumpForSigQuit(std::__1::basic_ostream&lt;char, std::__1::char_traits&lt;char&gt; &gt;&)+68)
221  native: #07 pc 00326831  /system/lib/libart.so (art::SignalCatcher::HandleSigQuit()+896)
222  native: #08 pc 003270a1  /system/lib/libart.so (art::SignalCatcher::Run(void*)+324)
223  native: #09 pc 0003f813  /system/lib/libc.so (__pthread_start(void*)+30)
224  native: #10 pc 00019f75  /system/lib/libc.so (__start_thread+6)
225  (no managed stack frames)
226
227"main" prio=5 tid=1 Suspended
228  | group="main" sCount=1 dsCount=0 obj=0x747552a0 self=0xf5376500
229  | sysTid=30363 nice=0 cgrp=default sched=0/0 handle=0xf74feb34
230  | state=S schedstat=( 331107086 164153349 851 ) utm=6 stm=27 core=3 HZ=100
231  | stack=0xff00f000-0xff011000 stackSize=8MB
232  | held mutexes=
233  kernel: __switch_to+0x7c/0x88
234  kernel: futex_wait_queue_me+0xd4/0x130
235  kernel: futex_wait+0xf0/0x1f4
236  kernel: do_futex+0xcc/0x8f4
237  kernel: compat_SyS_futex+0xd0/0x14c
238  kernel: cpu_switch_to+0x48/0x4c
239  native: #00 pc 000175e8  /system/lib/libc.so (syscall+28)
240  native: #01 pc 000f5ced  /system/lib/libart.so (art::ConditionVariable::Wait(art::Thread*)+80)
241  native: #02 pc 00335353  /system/lib/libart.so (art::Thread::FullSuspendCheck()+838)
242  native: #03 pc 0011d3a7  /system/lib/libart.so (art::ClassLinker::LoadClassMembers(art::Thread*, art::DexFile const&, unsigned char const*, art::Handle&lt;art::mirror::Class&gt;, art::OatFile::OatClass const*)+746)
243  native: #04 pc 0011d81d  /system/lib/libart.so (art::ClassLinker::LoadClass(art::Thread*, art::DexFile const&, art::DexFile::ClassDef const&, art::Handle&lt;art::mirror::Class&gt;)+88)
244  native: #05 pc 00132059  /system/lib/libart.so (art::ClassLinker::DefineClass(art::Thread*, char const*, unsigned int, art::Handle&lt;art::mirror::ClassLoader&gt;, art::DexFile const&, art::DexFile::ClassDef const&)+320)
245  native: #06 pc 001326c1  /system/lib/libart.so (art::ClassLinker::FindClassInPathClassLoader(art::ScopedObjectAccessAlreadyRunnable&, art::Thread*, char const*, unsigned int, art::Handle&lt;art::mirror::ClassLoader&gt;, art::mirror::Class**)+688)
246  native: #07 pc 002cb1a1  /system/lib/libart.so (art::VMClassLoader_findLoadedClass(_JNIEnv*, _jclass*, _jobject*, _jstring*)+264)
247  native: #08 pc 002847fd  /data/dalvik-cache/arm/system@framework@boot.oat (Java_java_lang_VMClassLoader_findLoadedClass__Ljava_lang_ClassLoader_2Ljava_lang_String_2+112)
248  at java.lang.VMClassLoader.findLoadedClass!(Native method)
249  at java.lang.ClassLoader.findLoadedClass(ClassLoader.java:362)
250  at java.lang.ClassLoader.loadClass(ClassLoader.java:499)
251  at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
252  at android.app.ActivityThread.installProvider(ActivityThread.java:5141)
253  at android.app.ActivityThread.installContentProviders(ActivityThread.java:4748)
254  at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4688)
255  at android.app.ActivityThread.-wrap1(ActivityThread.java:-1)
256  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1405)
257  at android.os.Handler.dispatchMessage(Handler.java:102)
258  at android.os.Looper.loop(Looper.java:148)
259  at android.app.ActivityThread.main(ActivityThread.java:5417)
260  at java.lang.reflect.Method.invoke!(Native method)
261  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
262  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
263
264  ...
265<i>  Stacks for other threads in this process follow</i>
266  ...</pre></p>
267  </div>
268</div>
269
270<h3 id="deadlocks">Finding deadlocks</h3>
271<p>Deadlocks often first appear as ANRs because threads are getting stuck. If
272the deadlock hits the system server, the watchdog will eventually kill it,
273leading to an entry in the log similar to:
274<code>WATCHDOG KILLING SYSTEM PROCESS</code>. From the user perspective, the
275device reboots, although technically this is a runtime restart rather than a
276true reboot.</p>
277
278<ul>
279<li>In a <strong>runtime</strong> restart, the system server dies and is
280restarted; the user sees the device return to the boot animation.</li>
281<li>In a <strong>reboot</strong>, the kernel has crashed; the user sees the
282device return to the Google boot logo.</li>
283</ul>
284
285<p>To find deadlocks, check the VM traces sections for a pattern of thread A
286waiting on something held by thread B, which in turn is waiting on something
287held by thread A.</p>
288
289<div class="toggle-content closed">
290  <p><a href="#" onclick="return toggleContent(this)">
291    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
292    <strong><span class="toggle-content-text">Show example</span>
293    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
294  </a></p>
295
296  <div class="toggle-content-toggleme">
297    <p><pre>"Binder_B" prio=5 tid=73 Blocked
298  | group="main" sCount=1 dsCount=0 obj=0x13faa0a0 self=0x95e24800
299  | sysTid=2016 nice=0 cgrp=default sched=0/0 handle=0x8b68d930
300  | state=S schedstat=( 9351576559 4141431119 16920 ) utm=819 stm=116 core=1 HZ=100
301  | stack=0x8b591000-0x8b593000 stackSize=1014KB
302  | held mutexes=
303  at com.android.server.pm.UserManagerService.exists(UserManagerService.java:387)
304  - waiting to lock &lt;0x025f9b02&gt; (a android.util.ArrayMap) held by thread 20
305  at com.android.server.pm.PackageManagerService.getApplicationInfo(PackageManagerService.java:2848)
306  at com.android.server.AppOpsService.getOpsRawLocked(AppOpsService.java:881)
307  at com.android.server.AppOpsService.getOpsLocked(AppOpsService.java:856)
308  at com.android.server.AppOpsService.noteOperationUnchecked(AppOpsService.java:719)
309  - locked &lt;0x0231885a&gt; (a com.android.server.AppOpsService)
310  at com.android.server.AppOpsService.noteOperation(AppOpsService.java:713)
311  at com.android.server.AppOpsService$2.getMountMode(AppOpsService.java:260)
312  at com.android.server.MountService$MountServiceInternalImpl.getExternalStorageMountMode(MountService.java:3416)
313  at com.android.server.am.ActivityManagerService.startProcessLocked(ActivityManagerService.java:3228)
314  at com.android.server.am.ActivityManagerService.startProcessLocked(ActivityManagerService.java:3170)
315  at com.android.server.am.ActivityManagerService.startProcessLocked(ActivityManagerService.java:3059)
316  at com.android.server.am.BroadcastQueue.processNextBroadcast(BroadcastQueue.java:1070)
317  - locked &lt;0x044d166f&gt; (a com.android.server.am.ActivityManagerService)
318  at com.android.server.am.ActivityManagerService.finishReceiver(ActivityManagerService.java:16950)
319  at android.app.ActivityManagerNative.onTransact(ActivityManagerNative.java:494)
320  at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:2432)
321  at android.os.Binder.execTransact(Binder.java:453)
322...
323  "PackageManager" prio=5 tid=20 Blocked
324  | group="main" sCount=1 dsCount=0 obj=0x1304f4a0 self=0xa7f43900
325  | sysTid=1300 nice=10 cgrp=bg_non_interactive sched=0/0 handle=0x9fcf9930
326  | state=S schedstat=( 26190141996 13612154802 44357 ) utm=2410 stm=209 core=2 HZ=100
327  | stack=0x9fbf7000-0x9fbf9000 stackSize=1038KB
328  | held mutexes=
329  at com.android.server.AppOpsService.noteOperationUnchecked(AppOpsService.java:718)
330  - waiting to lock &lt;0x0231885a&gt; (a com.android.server.AppOpsService) held by thread 73
331  at com.android.server.AppOpsService.noteOperation(AppOpsService.java:713)
332  at com.android.server.AppOpsService$2.getMountMode(AppOpsService.java:260)
333  at com.android.server.AppOpsService$2.hasExternalStorage(AppOpsService.java:273)
334  at com.android.server.MountService$MountServiceInternalImpl.hasExternalStorage(MountService.java:3431)
335  at com.android.server.MountService.getVolumeList(MountService.java:2609)
336  at android.os.storage.StorageManager.getVolumeList(StorageManager.java:880)
337  at android.os.Environment$UserEnvironment.getExternalDirs(Environment.java:83)
338  at android.os.Environment.isExternalStorageEmulated(Environment.java:708)
339  at com.android.server.pm.PackageManagerService.isExternalMediaAvailable(PackageManagerService.java:9327)
340  at com.android.server.pm.PackageManagerService.startCleaningPackages(PackageManagerService.java:9367)
341  - locked &lt;0x025f9b02&gt; (a android.util.ArrayMap)
342  at com.android.server.pm.PackageManagerService$PackageHandler.doHandleMessage(PackageManagerService.java:1320)
343  at com.android.server.pm.PackageManagerService$PackageHandler.handleMessage(PackageManagerService.java:1122)
344  at android.os.Handler.dispatchMessage(Handler.java:102)
345  at android.os.Looper.loop(Looper.java:148)
346  at android.os.HandlerThread.run(HandlerThread.java:61)
347  at com.android.server.ServiceThread.run(ServiceThread.java:46)</pre></p>
348  </div>
349</div>
350
351
352<h2 id="activities">Activities</h2>
353<p>An <a href="http://developer.android.com/guide/components/activities.html">Activity</a>
354is an application component that provides a screen users interact with to do
355something such as dial a number, take a photo, send an email, etc. From a bug
356report perspective, an
357<a href="http://developer.android.com/reference/android/app/Activity.html">activity</a>
358is a single, focused thing a user can do, which makes locating the activity that
359was in focus during a crash very important. Activities (via ActivityManager)
360run processes, so locating all process stops and starts for a given activity can
361also aid troubleshooting.</p>
362
363<h3 id="history-focused-activities">Viewing focused activities</h3>
364<p>To view a history of focused activities, search for
365<code>am_focused_activity</code>.</p>
366
367<div class="toggle-content closed">
368  <p><a href="#" onclick="return toggleContent(this)">
369    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
370    <strong><span class="toggle-content-text">Show example</span>
371    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
372  </a></p>
373
374  <div class="toggle-content-toggleme">
375    <p><pre>grep "am_focused_activity" bugreport-2015-10-01-18-13-48.txt
37610-01 18:10:41.409  4600 14112 I am_focused_activity: [0,com.google.android.GoogleCamera/com.android.camera.CameraActivity]
37710-01 18:11:17.313  4600  5687 I am_focused_activity: [0,com.google.android.googlequicksearchbox/com.google.android.launcher.GEL]
37810-01 18:11:52.747  4600 14113 I am_focused_activity: [0,com.google.android.GoogleCamera/com.android.camera.CameraActivity]
37910-01 18:14:07.762  4600  5687 I am_focused_activity: [0,com.google.android.googlequicksearchbox/com.google.android.launcher.GEL]</pre></p>
380  </div>
381</div>
382
383<h3 id="history-process-starts">Viewing process starts</h3>
384<p>To view a history of process starts, search for <code>Start proc</code>.</p>
385
386<div class="toggle-content closed">
387  <p><a href="#" onclick="return toggleContent(this)">
388    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
389    <strong><span class="toggle-content-text">Show example</span>
390    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
391  </a></p>
392
393  <div class="toggle-content-toggleme">
394    <p><pre>grep "Start proc" bugreport-2015-10-01-18-13-48.txt
39510-01 18:09:15.309  4600  4612 I ActivityManager: Start proc 24533:com.metago.astro/u0a240 for broadcast com.metago.astro/com.inmobi.commons.analytics.androidsdk.IMAdTrackerReceiver
39610-01 18:09:15.687  4600 14112 I ActivityManager: Start proc 24548:com.google.android.apps.fitness/u0a173 for service com.google.android.apps.fitness/.api.services.ActivityUpsamplingService
39710-01 18:09:15.777  4600  6604 I ActivityManager: Start proc 24563:cloudtv.hdwidgets/u0a145 for broadcast cloudtv.hdwidgets/cloudtv.switches.SwitchSystemUpdateReceiver
39810-01 18:09:20.574  4600  6604 I ActivityManager: Start proc 24617:com.wageworks.ezreceipts/u0a111 for broadcast com.wageworks.ezreceipts/.ui.managers.IntentReceiver
399...</pre></p>
400  </div>
401</div>
402
403<h3 id="device-thrashing">Is the device thrashing?</h3>
404<p>To determine if the device is
405<a href="https://en.wikipedia.org/wiki/Thrashing_(computer_science)">thrashing</a>,
406check for an abnormal increase in activity around <code>am_proc_died</code> and
407<code>am_proc_start</code> in a short amount of time.</p>
408
409<div class="toggle-content closed">
410  <p><a href="#" onclick="return toggleContent(this)">
411    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
412    <strong><span class="toggle-content-text">Show example</span>
413    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
414  </a></p>
415
416  <div class="toggle-content-toggleme">
417    <p><pre>grep -e "am_proc_died" -e "am_proc_start" bugreport-2015-10-01-18-13-48.txt
41810-01 18:07:06.494  4600  9696 I am_proc_died: [0,20074,com.android.musicfx]
41910-01 18:07:06.555  4600  6606 I am_proc_died: [0,31166,com.concur.breeze]
42010-01 18:07:06.566  4600 14112 I am_proc_died: [0,18812,com.google.android.apps.fitness]
42110-01 18:07:07.018  4600  7513 I am_proc_start: [0,20361,10113,com.sony.playmemories.mobile,broadcast,com.sony.playmemories.mobile/.service.StartupReceiver]
42210-01 18:07:07.357  4600  4614 I am_proc_start: [0,20381,10056,com.google.android.talk,service,com.google.android.talk/com.google.android.libraries.hangouts.video.CallService]
42310-01 18:07:07.784  4600  4612 I am_proc_start: [0,20402,10190,com.andcreate.app.trafficmonitor:loopback_measure_serivce,service,com.andcreate.app.trafficmonitor/.loopback.LoopbackMeasureService]
42410-01 18:07:10.753  4600  5997 I am_proc_start: [0,20450,10097,com.amazon.mShop.android.shopping,broadcast,com.amazon.mShop.android.shopping/com.amazon.identity.auth.device.storage.LambortishClock$ChangeTimestampsBroadcastReceiver]
42510-01 18:07:15.267  4600  6605 I am_proc_start: [0,20539,10173,com.google.android.apps.fitness,service,com.google.android.apps.fitness/.api.services.ActivityUpsamplingService]
42610-01 18:07:15.985  4600  4612 I am_proc_start: [0,20568,10022,com.android.musicfx,broadcast,com.android.musicfx/.ControlPanelReceiver]
42710-01 18:07:16.315  4600  7512 I am_proc_died: [0,20096,com.google.android.GoogleCamera]</pre></p>
428  </div>
429</div>
430
431<h2 id="memory">Memory</h2>
432<p>Because Android devices often have constrained physical memory, managing
433random-access memory (RAM) is critical. Bug reports contain several indicators
434of low memory as well as a dumpstate that provides a memory snapshot.</p>
435
436<h3 id="low-memory">Identifying low memory</h3>
437<p>Low memory can cause the system to thrash as it kills some processes to free
438memory but continues to start other processes. To view corroborating evidence of
439low memory, check for concentrations of <code>am_proc_died</code> and
440<code>am_proc_start</code> entries in the binary event log.</p>
441
442<p>Low memory can also slow task switching and thwart return attempts (because
443the task the user was trying to return to was killed). If the launcher was
444killed, it restarts when the user touches the home button and logs show the
445launcher reload its content.</p>
446
447<h4 id="historical-low-memory">Viewing historical indicators</h4>
448<p>The <code>am_low_memory</code> entry in the binary event log indicates the
449last cached process has died. After this, the system starts killing services.
450
451<div class="toggle-content closed">
452  <p><a href="#" onclick="return toggleContent(this)">
453    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
454    <strong><span class="toggle-content-text">Show example</span>
455    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
456  </a></p>
457
458  <div class="toggle-content-toggleme">
459    <p><pre>grep "am_low_memory" bugreport-2015-10-01-18-13-48.txt
46010-01 18:11:02.219  4600  7513 I am_low_memory: 41
46110-01 18:12:18.526  4600 14112 I am_low_memory: 39
46210-01 18:12:18.874  4600  7514 I am_low_memory: 38
46310-01 18:12:22.570  4600 14112 I am_low_memory: 40
46410-01 18:12:34.811  4600 20319 I am_low_memory: 43
46510-01 18:12:37.945  4600  6521 I am_low_memory: 43
46610-01 18:12:47.804  4600 14110 I am_low_memory: 43</pre></p>
467  </div>
468</div>
469
470<h4 id="thrashing-indicators">Viewing thrashing indicators</h4>
471<p>Other indicators of system thrashing (paging, direct reclaim, etc.) include
472<code>kswapd</code>, <code>kworker</code>, and <code>mmcqd</code> consuming
473cycles. (Keep in mind the bugreport being gathered can influence thrashing
474indicators.)</p>
475
476<div class="toggle-content closed">
477  <p><a href="#" onclick="return toggleContent(this)">
478    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
479    <strong><span class="toggle-content-text">Show example</span>
480    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
481  </a></p>
482
483  <div class="toggle-content-toggleme">
484    <p><pre>------ CPU INFO (top -n 1 -d 1 -m 30 -t) ------
485
486User 15%, System 54%, IOW 28%, IRQ 0%
487User 82 + Nice 2 + Sys 287 + Idle 1 + IOW 152 + IRQ 0 + SIRQ 5 = 529
488
489  PID   TID PR CPU% S     VSS     RSS PCY UID      Thread          Proc
49015229 15229  0  19% R      0K      0K  fg root     kworker/0:2
49129512 29517  1   7% D 1173524K 101188K  bg u0_a27   Signal Catcher  com.google.android.talk
49224565 24570  3   6% D 2090920K 145168K  fg u0_a22   Signal Catcher  com.google.android.googlequicksearchbox:search
49319525 19525  2   6% R   3476K   1644K  fg shell    top             top
49424957 24962  2   5% R 1706928K 125716K  bg u0_a47   Signal Catcher  com.google.android.GoogleCamera
49519519 19519  3   4% S      0K      0K  fg root     kworker/3:1
496  120   120  0   3% S      0K      0K  fg root     mmcqd/1
49718233 18233  1   3% S      0K      0K  fg root     kworker/1:1
49825589 25594  1   2% D 1270476K  75776K  fg u0_a8    Signal Catcher  com.google.android.gms
49919399 19399  2   1% S      0K      0K  fg root     kworker/2:2
500 1963  1978  1   0% S 1819100K 125136K  fg system   android.fg      system_server
501 1963  1981  3   0% S 1819100K 125136K  fg system   android.display system_server</pre></p>
502  </div>
503</div>
504<p></p>
505
506<p>ANR logs can provide a similar memory snapshot.</p>
507
508<div class="toggle-content closed">
509  <p><a href="#" onclick="return toggleContent(this)">
510    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
511    <strong><span class="toggle-content-text">Show example</span>
512    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
513  </a></p>
514
515  <div class="toggle-content-toggleme">
516    <p><pre>10-03 17:19:59.959  1963  1976 E ActivityManager: ANR in com.google.android.apps.magazines
51710-03 17:19:59.959  1963  1976 E ActivityManager: PID: 18819
51810-03 17:19:59.959  1963  1976 E ActivityManager: Reason: Broadcast of Intent { act=android.net.conn.CONNECTIVITY_CHANGE flg=0x4000010 cmp=com.google.android.apps.magazines/com.google.apps.dots.android.newsstand.appwidget.NewsWidgetProvider (has extras) }
51910-03 17:19:59.959  1963  1976 E ActivityManager: Load: 19.19 / 14.76 / 12.03
52010-03 17:19:59.959  1963  1976 E ActivityManager: CPU usage from 0ms to 11463ms later:
52110-03 17:19:59.959  1963  1976 E ActivityManager:   54% 15229/kworker/0:2: 0% user + 54% kernel
52210-03 17:19:59.959  1963  1976 E ActivityManager:   38% 1963/system_server: 14% user + 23% kernel / faults: 17152 minor 1073 major
52310-03 17:19:59.959  1963  1976 E ActivityManager:   11% 120/mmcqd/1: 0% user + 11% kernel
52410-03 17:19:59.959  1963  1976 E ActivityManager:   10% 2737/com.android.systemui: 4.7% user + 5.6% kernel / faults: 7211 minor 149 major
52510-03 17:19:59.959  1963  1976 E ActivityManager:   0.2% 1451/debuggerd: 0% user + 0.2% kernel / faults: 15211 minor 147 major
52610-03 17:19:59.959  1963  1976 E ActivityManager:   8.7% 6162/com.twofortyfouram.locale: 4% user + 4.7% kernel / faults: 4924 minor 260 major
52710-03 17:19:59.959  1963  1976 E ActivityManager:   6.1% 24565/com.google.android.googlequicksearchbox:search: 2.4% user + 3.7% kernel / faults: 2902 minor 129 major
52810-03 17:19:59.959  1963  1976 E ActivityManager:   6% 55/kswapd0: 0% user + 6% kernel
52910-03 17:19:59.959  1963  1976 E ActivityManager:   4.9% 18819/com.google.android.apps.magazines: 1.5% user + 3.3% kernel / faults: 10129 minor 986 major
53010-03 17:19:59.959  1963  1976 E ActivityManager:   2.8% 18233/kworker/1:1: 0% user + 2.8% kernel
53110-03 17:19:59.959  1963  1976 E ActivityManager:   4.2% 3145/com.android.phone: 2% user + 2.2% kernel / faults: 3005 minor 43 major
53210-03 17:19:59.959  1963  1976 E ActivityManager:   4.2% 8084/com.android.chrome: 2% user + 2.1% kernel / faults: 4798 minor 380 major
53310-03 17:19:59.959  1963  1976 E ActivityManager:   3.4% 182/surfaceflinger: 1.1% user + 2.3% kernel / faults: 842 minor 13 major
53410-03 17:19:59.959  1963  1976 E ActivityManager:   3% 18236/kworker/1:2: 0% user + 3% kernel
53510-03 17:19:59.959  1963  1976 E ActivityManager:   2.9% 19231/com.android.systemui:screenshot: 0.8% user + 2.1% kernel / faults: 6119 minor 348 major
53610-03 17:19:59.959  1963  1976 E ActivityManager:   2.3% 15350/kworker/0:4: 0% user + 2.3% kernel
53710-03 17:19:59.959  1963  1976 E ActivityManager:   2.2% 1454/mediaserver: 0% user + 2.2% kernel / faults: 479 minor 6 major
53810-03 17:19:59.959  1963  1976 E ActivityManager:   2% 16496/com.android.chrome:sandboxed_process10: 0.1% user + 1.8% kernel / faults: 3610 minor 234 major
53910-03 17:19:59.959  1963  1976 E ActivityManager:   1% 3119/com.android.nfc: 0.4% user + 0.5% kernel / faults: 1789 minor 17 major
54010-03 17:19:59.959  1963  1976 E ActivityManager:   1.7% 19337/com.jarettmillard.localeconnectiontype:background: 0.1% user + 1.5% kernel / faults: 7854 minor 439 major
54110-03 17:19:59.959  1963  1976 E ActivityManager:   0.7% 3066/com.google.android.inputmethod.latin: 0.3% user + 0.3% kernel / faults: 1336 minor 7 major
54210-03 17:19:59.959  1963  1976 E ActivityManager:   1% 25589/com.google.android.gms: 0.3% user + 0.6% kernel / faults: 2867 minor 237 major
54310-03 17:19:59.959  1963  1976 E ActivityManager:   0.9% 1460/sensors.qcom: 0.5% user + 0.4% kernel / faults: 262 minor 5 major
54410-03 17:19:59.959  1963  1976 E ActivityManager:   0.8% 3650/mpdecision: 0% user + 0.8% kernel / faults: 160 minor 1 major
54510-03 17:19:59.959  1963  1976 E ActivityManager:   0.1% 3132/com.redbend.vdmc: 0% user + 0% kernel / faults: 1746 minor 5 major</pre></p>
546  </div>
547</div>
548
549<h3 id="memory-snapshot">Getting a memory snapshot</h3>
550<p>The memory snapshot is a dumpstate that lists running Java and native
551processes (for details, refer to
552<a href="https://developer.android.com/tools/debugging/debugging-memory.html#ViewingAllocations">Viewing
553Overall Memory Allocations</a>). Keep in mind the snapshot gives only the state
554at a specific moment in time; the system might have been in better (or worse)
555shape before the snapshot.</p>
556<ul>
557<li>To understand how long a process runs, see
558<a href="#process-runtime">Process runtime</a>.</li>
559<li>To understand why something is currently running, see
560<a href="#why-is-process-running">Why is a process running?</a></li>
561</ul>
562
563<div class="toggle-content closed">
564  <p><a href="#" onclick="return toggleContent(this)">
565    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
566    <strong><span class="toggle-content-text">Show example</span>
567    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
568  </a></p>
569
570  <div class="toggle-content-toggleme">
571    <p><pre>Total PSS by OOM adjustment:
572    86752 kB: Native
573               22645 kB: surfaceflinger (pid 197)
574               18597 kB: mediaserver (pid 204)
575               ...
576   136959 kB: System
577              136959 kB: system (pid 785)
578   220218 kB: Persistent
579              138859 kB: com.android.systemui (pid 947 / activities)
580               39178 kB: com.android.nfc (pid 1636)
581               28313 kB: com.android.phone (pid 1659)
582               13868 kB: com.redbend.vdmc (pid 1646)
583     9534 kB: Persistent Service
584                9534 kB: com.android.bluetooth (pid 23807)
585   178604 kB: Foreground
586              168620 kB: com.google.android.googlequicksearchbox (pid 1675 / activities)
587                9984 kB: com.google.android.apps.maps (pid 13952)
588   188286 kB: Visible
589               85326 kB: com.google.android.wearable.app (pid 1535)
590               38978 kB: com.google.process.gapps (pid 1510)
591               31936 kB: com.google.android.gms.persistent (pid 2072)
592               27950 kB: com.google.android.gms.wearable (pid 1601)
593                4096 kB: com.google.android.googlequicksearchbox:interactor (pid 1550)
594    52948 kB: Perceptible
595               52948 kB: com.google.android.inputmethod.latin (pid 1566)
596   150851 kB: A Services
597               81121 kB: com.google.android.gms (pid 1814)
598               37586 kB: com.google.android.talk (pid 9584)
599               10949 kB: com.google.android.music:main (pid 4019)
600               10727 kB: com.motorola.targetnotif (pid 31071)
601               10468 kB: com.google.android.GoogleCamera (pid 9984)
602    33298 kB: Previous
603               33298 kB: com.android.settings (pid 9673 / activities)
604   165188 kB: B Services
605               49490 kB: com.facebook.katana (pid 15035)
606               22483 kB: com.whatsapp (pid 28694)
607               21308 kB: com.iPass.OpenMobile (pid 5325)
608               19788 kB: com.google.android.apps.googlevoice (pid 23934)
609               17399 kB: com.google.android.googlequicksearchbox:search (pid 30359)
610                9073 kB: com.google.android.apps.youtube.unplugged (pid 21194)
611                7660 kB: com.iPass.OpenMobile:remote (pid 23754)
612                7291 kB: com.pujie.wristwear.pujieblack (pid 24240)
613                7157 kB: com.instagram.android:mqtt (pid 9530)
614                3539 kB: com.qualcomm.qcrilmsgtunnel (pid 16186)
615   204324 kB: Cached
616               43424 kB: com.amazon.mShop.android (pid 13558)
617               22563 kB: com.google.android.apps.magazines (pid 13844)
618               ...
619                4298 kB: com.google.android.apps.enterprise.dmagent (pid 13826)</pre></p>
620  </div>
621</div>
622
623<h2 id="broadcasts">Broadcasts</h2>
624<p>Applications generate broadcasts to send events within the current
625application or to another application. Broadcast receivers subscribe to specific
626messages (via filters), enabling them to both listen and respond to a broadcast.
627Bug reports contain information about sent broadcasts and unsent broadcasts, as
628well as a dumpsys of all receivers listening to a specific broadcast.</p>
629
630<h3 id="historical-broadcasts">Viewing historical broadcasts</h3>
631<p>Historical broadcasts are those that have already been sent, listed in
632reverse chronological order.</p>
633
634<p>The <strong>summary</strong> section is an overview of the last 300
635foreground broadcasts and the last 300 background broadcasts.</p>
636
637<div class="toggle-content closed">
638  <p><a href="#" onclick="return toggleContent(this)">
639    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
640    <strong><span class="toggle-content-text">Show example</span>
641    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
642  </a></p>
643
644  <div class="toggle-content-toggleme">
645    <p><pre> Historical broadcasts summary [foreground]:
646 #0: act=android.intent.action.SCREEN_ON flg=0x50000010
647    +1ms dispatch +90ms finish
648    enq=2015-10-29 17:10:51 disp=2015-10-29 17:10:51 fin=2015-10-29 17:10:51
649  #1: act=android.intent.action.SCREEN_OFF flg=0x50000010
650    0 dispatch +60ms finish
651    enq=2015-10-29 17:10:05 disp=2015-10-29 17:10:05 fin=2015-10-29 17:10:05
652 ...
653 Historical broadcasts summary [background]:
654 ...</pre></p>
655  </div>
656</div>
657<p></p>
658
659<p>The <strong>detail</strong> section contains complete information for the
660last 50 foreground broadcasts and the last 50 background broadcasts, as well as
661the receivers for each broadcast. Receivers that have a:</p>
662<ul>
663<li><code>BroadcastRecord</code> entry are registered at runtime and are sent
664only to already running processes.</li>
665<li><code>ResolveInfo</code> entry are registered through manifest entries. The
666ActivityManager starts the process for each <code>ResolveInfo</code> if it is
667not already running.</li>
668</ul>
669
670<div class="toggle-content closed">
671  <p><a href="#" onclick="return toggleContent(this)">
672    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
673    <strong><span class="toggle-content-text">Show example</span>
674    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
675  </a></p>
676
677  <div class="toggle-content-toggleme">
678    <p><pre>Historical broadcasts [foreground]:
679 ...
680 Historical broadcasts [background]:
681 Historical Broadcast background #0:
682 ...
683 Historical Broadcast background #5:
684    BroadcastRecord{18dbb16 u0 android.intent.action.USER_PRESENT} to user 0
685    Intent { act=android.intent.action.USER_PRESENT flg=0x24000010 }
686    caller=com.android.systemui 2925:com.android.systemui/u0a27 pid=2925 uid=10027
687    enqueueClockTime=2015-10-29 17:10:55 dispatchClockTime=2015-10-29 17:10:55
688    dispatchTime=-2s321ms (0 since enq) finishTime=-2s320ms (+1ms since disp)
689    Receiver #0: BroadcastFilter{8181cc1 u-1 ReceiverList{5d929a8 902 system/1000/u-1 local:eca4dcb}}
690    Receiver #1: BroadcastFilter{6371c97 u-1 ReceiverList{2938b16 902 system/1000/u-1 local:840b831}}
691    ...
692    Receiver #19: BroadcastFilter{93f16b u0 ReceiverList{5c61eba 17016 com.google.android.gm/10079/u0 remote:24083e5}}
693 ...
694 Historical Broadcast background #37:
695    BroadcastRecord{7f6dd6 u0 android.hardware.action.NEW_PICTURE} to user 0
696    Intent { act=android.hardware.action.NEW_PICTURE dat=content://media/external/images/media/6345 flg=0x10 }
697    caller=com.google.android.GoogleCamera 32734:com.google.android.GoogleCamera/u0a53 pid=32734 uid=10053
698    enqueueClockTime=2015-10-29 17:09:48 dispatchClockTime=2015-10-29 17:09:49
699    dispatchTime=-45s720ms (+399ms since enq) finishTime=-45s701ms (+19ms since disp)
700    resultTo=null resultCode=0 resultData=null
701    nextReceiver=4 receiver=null
702    Receiver #0: ResolveInfo{33d2857 com.google.android.gms/com.google.android.libraries.social.mediamonitor.MediaMonitor m=0x608000}
703      priority=0 preferredOrder=0 match=0x608000 specificIndex=-1 isDefault=false
704      ActivityInfo:
705        name=com.google.android.libraries.social.mediamonitor.MediaMonitor
706        packageName=com.google.android.gms
707        enabled=true exported=true processName=com.google.android.gms
708        ...
709    Receiver #1: ResolveInfo{d9edf44 com.google.android.apps.maps/com.google.android.apps.gmm.ugc.clientnotification.StartPhotoTakenNotifierServiceReceiver m=0x608000}
710      priority=0 preferredOrder=0 match=0x608000 specificIndex=-1 isDefault=false
711      ActivityInfo:
712        name=com.google.android.apps.gmm.ugc.clientnotification.StartPhotoTakenNotifierServiceReceiver
713        packageName=com.google.android.apps.maps
714        enabled=true exported=true processName=com.google.android.apps.maps
715        ...
716    Receiver #2: ResolveInfo{743f82d com.google.android.apps.photos/com.google.android.libraries.social.mediamonitor.MediaMonitor m=0x608000}
717      priority=0 preferredOrder=0 match=0x608000 specificIndex=-1 isDefault=false
718      ActivityInfo:
719        name=com.google.android.libraries.social.mediamonitor.MediaMonitor
720        packageName=com.google.android.apps.photos
721        enabled=true exported=true processName=com.google.android.apps.photos
722        ...
723    Receiver #3: ResolveInfo{d5c9162 com.google.android.apps.plus/com.google.android.libraries.social.mediamonitor.MediaMonitor m=0x608000}
724      priority=0 preferredOrder=0 match=0x608000 specificIndex=-1 isDefault=false
725      ActivityInfo:
726        name=com.google.android.libraries.social.mediamonitor.MediaMonitor
727        packageName=com.google.android.apps.plus
728        enabled=true exported=true processName=com.google.android.apps.plus
729        ...</pre></p>
730  </div>
731</div>
732
733<h3 id="active-broadcasts">Viewing active broadcasts</h3>
734<p>Active broadcasts are those that have yet to be sent. A large number in the
735queue means the system can't dispatch the broadcasts fast enough to keep up.</p>
736
737<div class="toggle-content closed">
738  <p><a href="#" onclick="return toggleContent(this)">
739    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
740    <strong><span class="toggle-content-text">Show example</span>
741    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
742  </a></p>
743
744  <div class="toggle-content-toggleme">
745    <p><pre> Active ordered broadcasts [background]:
746 Active Ordered Broadcast background #133: <i>// size of queue</i>
747 ...</pre></p>
748  </div>
749</div>
750
751<h3 id="broadcast-listeners">Viewing broadcast listeners</h3>
752<p>To view a list of receivers listening for a broadcast, check the Receiver
753Resolver Table in the <code>dumpsys activity broadcasts</code>. The following
754example displays all receivers listening for <code>USER_PRESENT</code>.</p>
755
756<div class="toggle-content closed">
757  <p><a href="#" onclick="return toggleContent(this)">
758    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
759    <strong><span class="toggle-content-text">Show example</span>
760    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
761  </a></p>
762
763  <div class="toggle-content-toggleme">
764    <p><pre>-------------------------------------------------------------------------------
765ACTIVITY MANAGER BROADCAST STATE (dumpsys activity broadcasts)
766..
767  Receiver Resolver Table:
768    Full MIME Types:
769    ..
770    Wild MIME Types:
771    ..
772    Schemes:
773    ..
774    Non-Data Actions:
775    ..
776        android.intent.action.USER_PRESENT:
777          BroadcastFilter{8181cc1 u-1 ReceiverList{5d929a8 902 system/1000/u-1 local:eca4dcb}}
778          BroadcastFilter{6371c97 u-1 ReceiverList{2938b16 902 system/1000/u-1 local:840b831}}
779          BroadcastFilter{320c00 u0 ReceiverList{d3a6283 902 system/1000/u0 local:799c532}}
780          BroadcastFilter{e486048 u0 ReceiverList{36fbaeb 902 system/1000/u0 local:5f51e3a}}
781          BroadcastFilter{22b02 u-1 ReceiverList{b3f744d 902 system/1000/u-1 local:de837e4}}
782          BroadcastFilter{3e989ab u0 ReceiverList{f8deffa 2981 com.google.process.gapps/10012/u0 remote:26bd225}}
783          BroadcastFilter{fb56150 u0 ReceiverList{22b7b13 2925 com.android.systemui/10027/u0 remote:c54a602}}
784          BroadcastFilter{63bbb6 u-1 ReceiverList{ba6c751 3484 com.android.nfc/1027/u-1 remote:5c4a478}}
785          BroadcastFilter{95ad20d u0 ReceiverList{d8374a4 3586 com.google.android.googlequicksearchbox/10029/u0 remote:feb3737}}
786          BroadcastFilter{fdef551 u0 ReceiverList{28ca78 3745 com.google.android.gms.persistent/10012/u0 remote:f23afdb}}
787          BroadcastFilter{9830707 u0 ReceiverList{aabd946 3745 com.google.android.gms.persistent/10012/u0 remote:a4da121}}
788          BroadcastFilter{83c43d2 u0 ReceiverList{d422e5d 3745 com.google.android.gms.persistent/10012/u0 remote:f585034}}
789          BroadcastFilter{8890378 u0 ReceiverList{26d2cdb 3745 com.google.android.gms.persistent/10012/u0 remote:dfa61ea}}
790          BroadcastFilter{7bbb7 u0 ReceiverList{214b2b6 3745 com.google.android.gms.persistent/10012/u0 remote:8353a51}}
791          BroadcastFilter{38d3566 u0 ReceiverList{de859c1 3745 com.google.android.gms.persistent/10012/u0 remote:e003aa8}}
792          BroadcastFilter{3435d9f u0 ReceiverList{6e38b3e 3745 com.google.android.gms.persistent/10012/u0 remote:8dd7ff9}}
793          BroadcastFilter{d0a34bb u0 ReceiverList{5091d4a 3745 com.google.android.gms.persistent/10012/u0 remote:d6d22b5}}
794          BroadcastFilter{d43c416 u0 ReceiverList{51a3531 3745 com.google.android.gms.persistent/10012/u0 remote:d0b9dd8}}
795          BroadcastFilter{aabf36d u0 ReceiverList{a88bf84 3745 com.google.android.gms.persistent/10012/u0 remote:a9d6197}}
796          BroadcastFilter{93f16b u0 ReceiverList{5c61eba 17016 com.google.android.gm/10079/u0 remote:24083e5}}
797          BroadcastFilter{68f794e u0 ReceiverList{4cb1c49 947 com.google.android.googlequicksearchbox:search/10029/u0 remote:251d250}}
798      ..
799    MIME Typed Actions:</pre></p>
800  </div>
801</div>
802
803<h2 id="monitor contention">Monitor Contention</h2>
804<p>Monitor contention logging can sometimes indicate actual monitor contention,
805but most often indicates the system is so loaded that everything has slowed down.
806You might see long monitor events logged by ART in system or event log.</p>
807
808<p>In the system log:</p>
809<p><pre>10-01 18:12:44.343 29761 29914 W art     : Long monitor contention event with owner method=void android.database.sqlite.SQLiteClosable.acquireReference() from SQLiteClosable.java:52 waiters=0 for 3.914s</pre></p>
810
811<p>In the event log:</p>
812<p><pre>10-01 18:12:44.364 29761 29914 I dvm_lock_sample: [com.google.android.youtube,0,pool-3-thread-9,3914,ScheduledTaskMaster.java,138,SQLiteClosable.java,52,100]</pre></p>
813
814<h2 id="background-compilation">Background Compilation</h2>
815<p>Compilation can be expensive and load the device.</p>
816
817<div class="toggle-content closed">
818  <p><a href="#" onclick="return toggleContent(this)">
819    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
820    <strong><span class="toggle-content-text">Show example</span>
821    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
822  </a></p>
823
824  <div class="toggle-content-toggleme">
825    <p><pre>09-14 06:27:05.670  2508  2587 E ActivityManager: CPU usage from 0ms to 5857ms later:
82609-14 06:27:05.670  2508  2587 E ActivityManager:   84% 5708/dex2oat: 81% user + 2.3% kernel / faults: 3731 minor 1 major
82709-14 06:27:05.670  2508  2587 E ActivityManager:   73% 2508/system_server: 21% user + 51% kernel / faults: 10019 minor 28 major
82809-14 06:27:05.670  2508  2587 E ActivityManager:   1% 3935/com.android.phone: 0.3% user + 0.6% kernel / faults: 2684 minor 2 major</pre></p>
829  </div>
830</div>
831<p></p>
832
833<p>Compilation might occur in the background when Google Play store updates are
834downloading. In this case, messages from the Google Play store app
835(<code>finsky</code>) and <code>installd</code> appear prior to
836<code>dex2oat</code> messages.</p>
837
838<div class="toggle-content closed">
839  <p><a href="#" onclick="return toggleContent(this)">
840    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
841    <strong><span class="toggle-content-text">Show example</span>
842    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
843  </a></p>
844
845  <div class="toggle-content-toggleme">
846    <p><pre>10-07 08:42:33.725 11051 11051 D Finsky  : [1] InstallerTask.advanceState: Prepare to patch com.garmin.android.apps.virb (com.garmin.android.apps.virb) from content://downloads/my_downloads/3602 format 2
84710-07 08:42:33.752   495   495 I installd: free_cache(48637657) avail 15111192576
84884910-07 08:42:39.998  2497  2567 I PackageManager.DexOptimizer: Running dexopt (dex2oat) on: /data/app/vmdl436577137.tmp/base.apk pkg=com.garmin.android.apps.virb isa=arm vmSafeMode=false debuggable=false oatDir = /data/app/vmdl436577137.tmp/oat bootComplete=true
850…</pre></p>
851  </div>
852</div>
853<p></p>
854
855<p>Compilation might also occur in the background when an application is loading
856a dex file that has not yet been compiled. In this case, you won't see
857<code>finsky</code> or <code>installd</code> logging.</p>
858
859<div class="toggle-content closed">
860  <p><a href="#" onclick="return toggleContent(this)">
861    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
862    <strong><span class="toggle-content-text">Show example</span>
863    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
864  </a></p>
865
866  <div class="toggle-content-toggleme">
867    <p><pre>09-14 07:29:20.433 15736 15736 I dex2oat : /system/bin/dex2oat -j4 --dex-file=/data/user/0/com.facebook.katana/app_secondary_program_dex/program-72cef82b591768306676e10161c886b58b34315a308602be.dex.jar --oat-file=/data/user/0/com.facebook.katana/app_secondary_program_dex_opt/program-72cef82b591768306676e10161c886b58b34315a308602be.dex.dex
868...
86909-14 07:29:25.102 15736 15736 I dex2oat : dex2oat took 4.669s (threads: 4) arena alloc=7MB java alloc=3MB native alloc=29MB free=4MB</pre></p>
870  </div>
871</div>
872
873<h2 id="narrative">Narrative</h2>
874<p>Establishing the narrative of a problem (how it started, what happened, how
875the system reacted) requires a solid timeline of events. You can use the
876information in the bug report to sync timelines across multiple logs and
877determine the exact timestamp of the bug report.</p>
878
879<h3 id="timelines">Syncing timelines</h3>
880<p>A bugreport reflects multiple parallel timelines: system log, event log,
881kernel log, and multiple specialized timelines for broadcasts, battery stats,
882etc. Unfortunately, timelines are often reported using different time bases.</p>
883
884<p>The system and event log timestamps are in the same timezone as the user (as
885are most other timestamps). For example, when user taps the home button, the
886system log reports:</p>
887<p><pre>10-03 17:19:52.939  1963  2071 I ActivityManager: START u0 {act=android.intent.action.MAIN cat=[android.intent.category.HOME] flg=0x10200000 cmp=com.google.android.googlequicksearchbox/com.google.android.launcher.GEL (has extras)} from uid 1000 on display 0</pre></p>
888
889<p>For the same action, the event log reports:</p>
890<p><pre>10-03 17:19:54.279  1963  2071 I am_focused_activity: [0,com.google.android.googlequicksearchbox/com.google.android.launcher.GEL]</pre></p>
891
892<p>Kernel (<code>dmesg</code>) logs use a different time base, tagging log items
893with seconds since bootloader completes. To register this timescale to other
894timescales, search for <em>suspend exit</em> and <em>suspend entry</em> messages:</p>
895<p><pre>&lt;6&gt;[201640.779997] PM: suspend exit 2015-10-03 19:11:06.646094058 UTC
896897&lt;6&gt;[201644.854315] PM: suspend entry 2015-10-03 19:11:10.720416452 UTC</pre></p>
898
899<p>Because kernel logs might not include time while in suspend, you should
900piecewise register the log between the suspend entry and exit messages.
901Additionally, kernel logs use UTC timezone and must be adjusted to the user
902timezone.</p>
903
904<h3 id="time-of-bugreport">Identifying bugreport time</h3>
905<p>To determine when a bugreport was taken, first check the system log (Logcat)
906for the <code>dumpstate: begin</code>:</p>
907<p><pre>10-03 17:19:54.322 19398 19398 I dumpstate: begin</pre></p>
908
909<p>Next, check the kernel log (<code>dmesg</code>) timestamps for the <code>Starting service
910'bugreport'</code> message:</p>
911<p><pre>&lt;5&gt;[207064.285315] init: Starting service 'bugreport'...</pre></p>
912
913<p>Work backwards to correlate the two events, keeping in mind the caveats
914mentioned in <a href="#timelines">Syncing timelines</a>. While there's a lot
915happening after the bugreport is initiated, most activity isn't very useful as
916the act of taking the bugreport loads the system substantially.</p>
917
918<h2 id="power">Power</h2>
919
920<p>The event log contains screen power status, where 0 is screen off, 1 is
921screen on, and 2 is for keyguard done.</p>
922
923<div class="toggle-content closed">
924  <p><a href="#" onclick="return toggleContent(this)">
925    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
926    <strong><span class="toggle-content-text">Show example</span>
927    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
928  </a></p>
929
930  <div class="toggle-content-toggleme">
931    <p><pre>grep screen_toggled bugreport-2015-10-18-16-52-22.txt
93210-18 15:05:04.383   992   992 I screen_toggled: 1
93310-18 15:05:07.010   992   992 I screen_toggled: 0
93410-18 15:23:15.063   992   992 I screen_toggled: 1
93510-18 15:23:25.684   992   992 I screen_toggled: 0
93610-18 15:36:31.623   992   992 I screen_toggled: 1
93710-18 15:36:37.660  3283  3283 I screen_toggled: 2</pre></p>
938  </div>
939</div>
940
941<h2 id="packages">Packages</h2>
942<p>The DUMP OF SERVICE package contains application versions (and other useful
943info).</p>
944
945<div class="toggle-content closed">
946  <p><a href="#" onclick="return toggleContent(this)">
947    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
948    <strong><span class="toggle-content-text">Show example</span>
949    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
950  </a></p>
951
952  <div class="toggle-content-toggleme">
953    <p><pre>...
954Packages:
955...
956 Package [com.google.android.gms] (3cf534b):
957   userId=10013
958   sharedUser=SharedUserSetting{98f3d28 com.google.uid.shared/10013}
959   pkg=Package{b8f6a41 com.google.android.gms}
960   codePath=/system/priv-app/PrebuiltGmsCore
961   resourcePath=/system/priv-app/PrebuiltGmsCore
962   legacyNativeLibraryDir=/system/priv-app/PrebuiltGmsCore/lib
963   primaryCpuAbi=arm64-v8a
964   secondaryCpuAbi=armeabi-v7a
965   versionCode=8186448 targetSdk=23
966   versionName=8.1.86 (2287566-448)
967   splits=[base]
968   applicationInfo=ApplicationInfo{5158507 com.google.android.gms}
969   flags=[ SYSTEM HAS_CODE ALLOW_CLEAR_USER_DATA ]
970   privateFlags=[ PRIVILEGED ]
971   dataDir=/data/user/0/com.google.android.gms
972   supportsScreens=[small, medium, large, xlarge, resizeable, anyDensity]
973   libraries:
974     com.google.android.gms
975   usesOptionalLibraries:
976     com.android.location.provider
977     com.google.android.ble
978     com.android.media.remotedisplay
979   usesLibraryFiles:
980     /system/framework/com.android.media.remotedisplay.jar
981     /system/framework/com.android.location.provider.jar
982   timeStamp=2015-10-14 15:17:56
983   firstInstallTime=2015-09-22 14:08:35
984   lastUpdateTime=2015-10-14 15:17:56
985   signatures=PackageSignatures{db63be6 [1af63d8]}
986   installPermissionsFixed=true installStatus=1
987   pkgFlags=[ SYSTEM HAS_CODE ALLOW_CLEAR_USER_DATA ]
988   declared permissions:
989     com.google.android.gms.permission.INTERNAL_BROADCAST: prot=signature, INSTALLED
990     ...
991     com.google.android.gms.permission.CAR_VENDOR_EXTENSION: prot=dangerous, INSTALLED
992   User 0:  installed=true hidden=false stopped=false notLaunched=false enabled=0
993     disabledComponents:
994       com.google.android.gms.icing.service.PowerConnectedReceiver
995       ...
996       com.google.android.gms.icing.proxy.AppsMonitor
997     enabledComponents:
998       com.google.android.gms.mdm.receivers.GmsRegisteredReceiver
999       ...
1000       com.google.android.gms.subscribedfeeds.SyncService</pre></p>
1001  </div>
1002</div>
1003
1004<h2 id="processes">Processes</h2>
1005<p>Bug reports contain a huge amount of data for processes, including start and
1006stop time, runtime length, associated services, <code>oom_adj</code> score, etc.
1007For details on how Android manages processes, refer to
1008<a href="http://developer.android.com/guide/components/processes-and-threads.html">Processes
1009and Threads</a>.</p>
1010
1011<h3 id="process-runtime">Determining process runtime</h3>
1012<p>The <code>procstats</code> section contains complete statistics on how long
1013processes and associated services have been running. For a quick, human-readable
1014summary, search for <code>AGGREGATED OVER</code> to view data from either the
1015last three or 24 hours, then search for <code>Summary:</code> to view the list
1016of processes, how long those processes have run at various priorities, and their
1017RAM usage formatted as min-average-max PSS/min-average-max USS.</p>
1018
1019<div class="toggle-content closed">
1020  <p><a href="#" onclick="return toggleContent(this)">
1021    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
1022    <strong><span class="toggle-content-text">Show example</span>
1023    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
1024  </a></p>
1025
1026  <div class="toggle-content-toggleme">
1027    <p><pre>-------------------------------------------------------------------------------
1028DUMP OF SERVICE processinfo:
1029-------------------------------------------------------------------------------
1030DUMP OF SERVICE procstats:
1031COMMITTED STATS FROM 2015-10-19-23-54-56 (checked in):
1032...
1033COMMITTED STATS FROM 2015-10-20-03-00-00 (checked in):
1034...
1035CURRENT STATS:
1036...
1037AGGREGATED OVER LAST 24 HOURS:
1038System memory usage:
1039...
1040Per-Package Stats:
1041...
1042Summary:
1043...
1044  * com.google.android.gms.persistent / u0a13 / v8186448:
1045           TOTAL: 100% (21MB-27MB-40MB/20MB-24MB-38MB over 597)
1046             Top: 51% (22MB-26MB-38MB/21MB-24MB-36MB over 383)
1047          Imp Fg: 49% (21MB-27MB-40MB/20MB-25MB-38MB over 214)
10481049          Start time: 2015-10-19 09:14:37
1050  Total elapsed time: +1d0h22m7s390ms (partial) libart.so
1051
1052AGGREGATED OVER LAST 3 HOURS:
1053System memory usage:
1054...
1055Per-Package Stats:
1056...
1057Summary:
1058  * com.google.android.gms.persistent / u0a13 / v8186448:
1059           TOTAL: 100% (23MB-27MB-32MB/21MB-25MB-29MB over 111)
1060             Top: 61% (23MB-26MB-31MB/21MB-24MB-28MB over 67)
1061          Imp Fg: 39% (23MB-28MB-32MB/21MB-26MB-29MB over 44)
1062...
1063          Start time: 2015-10-20 06:49:24
1064  Total elapsed time: +2h46m59s736ms (partial) libart.so</pre></p>
1065  </div>
1066</div>
1067
1068<h3 id="why-is-process-running">Why is a process running?</h3>
1069<p>The <code>dumpsys activity processes</code> section lists all currently
1070running processes ordered by <code>oom_adj</code> score (Android indicates
1071process importance by assigning the process an <code>oom_adj</code> value, which
1072can be dynamically updated by ActivityManager). The output is similar to that of
1073a <a href="#memory-snapshot">memory snapshot</a> but includes additional
1074information about what is causing the process to run. In the example below,
1075the bolded entries indicate the <code>gms.persistent</code> process is running
1076at <code>vis</code> (visible) priority because the system process is bound to
1077its <code>NetworkLocationService</code>.</p>
1078
1079<div class="toggle-content closed">
1080  <p><a href="#" onclick="return toggleContent(this)">
1081    <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
1082    <strong><span class="toggle-content-text">Show example</span>
1083    <span class="toggle-content-text" style="display:none;">Hide example</span></strong>
1084  </a></p>
1085
1086  <div class="toggle-content-toggleme">
1087    <p><pre>-------------------------------------------------------------------------------
1088ACTIVITY MANAGER RUNNING PROCESSES (dumpsys activity processes)
1089...
1090Process LRU list (sorted by oom_adj, 34 total, non-act at 14, non-svc at 14):
1091    PERS #33: sys   F/ /P  trm: 0 902:system/1000 (fixed)
1092    PERS #32: pers  F/ /P  trm: 0 2925:com.android.systemui/u0a27 (fixed)
1093    PERS #31: pers  F/ /P  trm: 0 3477:com.quicinc.cne.CNEService/1000 (fixed)
1094    PERS #30: pers  F/ /P  trm: 0 3484:com.android.nfc/1027 (fixed)
1095    PERS #29: pers  F/ /P  trm: 0 3502:com.qualcomm.qti.rcsbootstraputil/1001 (fixed)
1096    PERS #28: pers  F/ /P  trm: 0 3534:com.qualcomm.qti.rcsimsbootstraputil/1001 (fixed)
1097    PERS #27: pers  F/ /P  trm: 0 3553:com.android.phone/1001 (fixed)
1098    Proc #25: psvc  F/ /IF trm: 0 4951:com.android.bluetooth/1002 (service)
1099        com.android.bluetooth/.hfp.HeadsetService&lt;=Proc{902:system/1000}
1100    Proc # 0: fore  F/A/T  trm: 0 3586:com.google.android.googlequicksearchbox/u0a29 (top-activity)
1101  Proc #26: vis   F/ /SB trm: 0 3374:com.google.android.googlequicksearchbox:interactor/u0a29 (service)
1102        com.google.android.googlequicksearchbox/com.google.android.voiceinteraction.GsaVoiceInteractionService&lt;=Proc{902:system/1000}
1103<b>    Proc # 5: vis   F/ /T  trm: 0 3745:com.google.android.gms.persistent/u0a12 (service)</b>
1104<b>        com.google.android.gms/com.google.android.location.network.NetworkLocationService&lt;=Proc{902:system/1000}</b>
1105    Proc # 3: vis   F/ /SB trm: 0 3279:com.google.android.gms/u0a12 (service)
1106        com.google.android.gms/.icing.service.IndexService&lt;=Proc{947:com.google.android.googlequicksearchbox:search/u0a29}
1107    Proc # 2: vis   F/ /T  trm: 0 947:com.google.android.googlequicksearchbox:search/u0a29 (service)
1108        com.google.android.googlequicksearchbox/com.google.android.sidekick.main.remoteservice.GoogleNowRemoteService&lt;=Proc{3586:com.google.android.googlequicksearchbox/u0a29}
1109    Proc # 1: vis   F/ /T  trm: 0 2981:com.google.process.gapps/u0a12 (service)
1110        com.google.android.gms/.tapandpay.hce.service.TpHceService&lt;=Proc{3484:com.android.nfc/1027}
1111    Proc #11: prcp  B/ /IB trm: 0 3392:com.google.android.inputmethod.latin/u0a64 (service)
1112        com.google.android.inputmethod.latin/com.android.inputmethod.latin.LatinIME&lt;=Proc{902:system/1000}
1113    Proc #24: svc   B/ /S  trm: 0 27071:com.google.android.music:main/u0a67 (started-services)
1114    Proc #22: svc   B/ /S  trm: 0 853:com.qualcomm.qcrilmsgtunnel/1001 (started-services)
1115    Proc # 4: prev  B/ /LA trm: 0 32734:com.google.android.GoogleCamera/u0a53 (previous)
1116    Proc #23: svcb  B/ /S  trm: 0 671:com.qualcomm.telephony/1000 (started-services)
1117    Proc #20: cch   B/ /CE trm: 0 27659:com.android.providers.calendar/u0a2 (provider)
1118        com.android.providers.calendar/.CalendarProvider2&lt;=Proc{27697:com.google.android.calendar/u0a40}
1119    Proc #13: cch   B/ /CE trm: 0 653:com.google.android.gms.wearable/u0a12 (cch-empty)
1120    Proc #10: cch   B/ /S  trm: 0 4067:com.google.android.talk/u0a62 (cch-started-ui-services)
1121    Proc # 7: cch   B/ /S  trm: 0 18868:com.google.corp.huddle.android/u0a95 (cch-started-ui-services)
1122    Proc # 6: cch   B/ /CA trm: 0 27697:com.google.android.calendar/u0a40 (cch-act)
1123    Proc # 8: cch+1 B/ /CA trm: 0 25675:com.google.android.apps.genie.geniewidget/u0a81 (cch-act)
1124    Proc #16: cch+2 B/ /CE trm: 0 1272:com.google.android.keep/u0a106 (cch-empty)
1125    Proc #15: cch+2 B/ /CE trm: 0 885:android.process.media/u0a9 (cch-empty)
1126    Proc #14: cch+2 B/ /CE trm: 0 15146:android.process.acore/u0a3 (cch-empty)
1127    Proc # 9: cch+3 B/ /CA trm: 0 17016:com.google.android.gm/u0a79 (cch-act)
1128    Proc #19: cch+4 B/ /CE trm: 0 973:com.google.android.apps.maps/u0a66 (cch-empty)
1129    Proc #18: cch+4 B/ /CE trm: 0 1091:com.google.android.apps.photos/u0a71 (cch-empty)
1130    Proc #17: cch+4 B/ /CE trm: 0 1141:com.google.android.apps.plus/u0a74 (cch-empty)
1131    Proc #12: cch+5 B/ /CA trm: 0 22299:com.google.android.apps.dogfood/u0a105 (cch-act)
1132    Proc #21: cch+6 B/ /CE trm: 0 995:com.google.android.partnersetup/u0a18 (cch-empty)></pre></p>
1133  </div>
1134</div>
1135</body>
1136</html>