1page.title=BufferQueue and gralloc 2@jd:body 3 4<!-- 5 Copyright 2014 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18--> 19<div id="qv-wrapper"> 20 <div id="qv"> 21 <h2>In this document</h2> 22 <ol id="auto-toc"> 23 </ol> 24 </div> 25</div> 26 27<p>Understanding the Android graphics system starts behind the scenes with 28BufferQueue and the gralloc HAL.</p> 29 30<p>The BufferQueue class is at the heart of everything graphical in Android. Its 31role is simple: Connect something that generates buffers of graphical data (the 32<em>producer</em>) to something that accepts the data for display or further 33processing (the <em>consumer</em>). Nearly everything that moves buffers of 34graphical data through the system relies on BufferQueue.</p> 35 36<p>The gralloc memory allocator performs buffer allocations and is 37implemented through a vendor-specific HAL interface (see 38<code>hardware/libhardware/include/hardware/gralloc.h</code>). The 39<code>alloc()</code> function takes expected arguments (width, height, pixel 40format) as well as a set of usage flags (detailed below).</p> 41 42<h2 id="BufferQueue">BufferQueue producers and consumers</h2> 43 44<p>Basic usage is straightforward: The producer requests a free buffer 45(<code>dequeueBuffer()</code>), specifying a set of characteristics including 46width, height, pixel format, and usage flags. The producer populates the buffer 47and returns it to the queue (<code>queueBuffer()</code>). Later, the consumer 48acquires the buffer (<code>acquireBuffer()</code>) and makes use of the buffer 49contents. When the consumer is done, it returns the buffer to the queue 50(<code>releaseBuffer()</code>).</p> 51 52<p>Recent Android devices support the <em>sync framework</em>, which enables the 53system to do nifty things when combined with hardware components that can 54manipulate graphics data asynchronously. For example, a producer can submit a 55series of OpenGL ES drawing commands and then enqueue the output buffer before 56rendering completes. The buffer is accompanied by a fence that signals when the 57contents are ready. A second fence accompanies the buffer when it is returned 58to the free list, so the consumer can release the buffer while the contents are 59still in use. This approach improves latency and throughput as the buffers 60move through the system.</p> 61 62<p>Some characteristics of the queue, such as the maximum number of buffers it 63can hold, are determined jointly by the producer and the consumer. However, the 64BufferQueue is responsible for allocating buffers as it needs them. Buffers are 65retained unless the characteristics change; for example, if the producer 66requests buffers with a different size, old buffers are freed and new buffers 67are allocated on demand.</p> 68 69<p>Producers and consumers can live in different processes. Currently, the 70consumer always creates and owns the data structure. In older versions of 71Android, only the producer side was binderized (i.e. producer could be in a 72remote process but consumer had to live in the process where the queue was 73created). Android 4.4 and later releases moved toward a more general 74implementation.</p> 75 76<p>Buffer contents are never copied by BufferQueue (moving that much data around 77would be very inefficient). Instead, buffers are always passed by handle.</p> 78 79<h2 id="gralloc_HAL">gralloc HAL usage flags</h2> 80 81<p>The gralloc allocator is not just another way to allocate memory on the 82native heap; in some situations, the allocated memory may not be cache-coherent 83or could be totally inaccessible from user space. The nature of the allocation 84is determined by the usage flags, which include attributes such as:</p> 85 86<ul> 87<li>How often the memory will be accessed from software (CPU)</li> 88<li>How often the memory will be accessed from hardware (GPU)</li> 89<li>Whether the memory will be used as an OpenGL ES (GLES) texture</li> 90<li>Whether the memory will be used by a video encoder</li> 91</ul> 92 93<p>For example, if your format specifies RGBA 8888 pixels, and you indicate the 94buffer will be accessed from software (meaning your application will touch 95pixels directly) then the allocator must create a buffer with 4 bytes per pixel 96in R-G-B-A order. If instead, you say the buffer will be only accessed from 97hardware and as a GLES texture, the allocator can do anything the GLES driver 98wants—BGRA ordering, non-linear swizzled layouts, alternative color 99formats, etc. Allowing the hardware to use its preferred format can improve 100performance.</p> 101 102<p>Some values cannot be combined on certain platforms. For example, the video 103encoder flag may require YUV pixels, so adding software access and specifying 104RGBA 8888 would fail.</p> 105 106<p>The handle returned by the gralloc allocator can be passed between processes 107through Binder.</p> 108 109<h2 id=tracking>Tracking BufferQueue with systrace</h2> 110 111<p>To really understand how graphics buffers move around, use systrace. The 112system-level graphics code is well instrumented, as is much of the relevant app 113framework code.</p> 114 115<p>A full description of how to use systrace effectively would fill a rather 116long document. Start by enabling the <code>gfx</code>, <code>view</code>, and 117<code>sched</code> tags. You'll also see BufferQueues in the trace. If you've 118used systrace before, you've probably seen them but maybe weren't sure what they 119were. As an example, if you grab a trace while 120<a href="https://github.com/google/grafika">Grafika's</a> "Play video 121(SurfaceView)" is running, the row labeled <em>SurfaceView</em> tells you how 122many buffers were queued up at any given time.</p> 123 124<p>The value increments while the app is active—triggering the rendering 125of frames by the MediaCodec decoder—and decrements while SurfaceFlinger is 126doing work, consuming buffers. When showing video at 30fps, the queue's value 127varies from 0 to 1 because the ~60fps display can easily keep up with the 128source. (Notice also that SurfaceFlinger only wakes when there's work to 129be done, not 60 times per second. The system tries very hard to avoid work and 130will disable VSYNC entirely if nothing is updating the screen.)</p> 131 132<p>If you switch to Grafika's "Play video (TextureView)" and grab a new trace, 133you'll see a row labeled 134com.android.grafika/com.android.grafika.PlayMovieActivity. This is the main UI 135layer, which is just another BufferQueue. Because TextureView renders into the 136UI layer (rather than a separate layer), you'll see all of the video-driven 137updates here.</p> 138 139<p>For more information about the systrace tool, refer to <a 140href="http://developer.android.com/tools/help/systrace.html">Systrace 141documentation</a>.</p> 142