1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 2<html lang="en"> 3<head> 4 <meta http-equiv="content-type" content="text/html; charset=utf-8"> 5 <title>GL Dispatch in Mesa</title> 6 <link rel="stylesheet" type="text/css" href="mesa.css"> 7</head> 8<body> 9 10<div class="header"> 11 <h1>The Mesa 3D Graphics Library</h1> 12</div> 13 14<iframe src="contents.html"></iframe> 15<div class="content"> 16 17<h1>GL Dispatch in Mesa</h1> 18 19<p>Several factors combine to make efficient dispatch of OpenGL functions 20fairly complicated. This document attempts to explain some of the issues 21and introduce the reader to Mesa's implementation. Readers already familiar 22with the issues around GL dispatch can safely skip ahead to the <a 23href="#overview">overview of Mesa's implementation</a>.</p> 24 25<h2>1. Complexity of GL Dispatch</h2> 26 27<p>Every GL application has at least one object called a GL <em>context</em>. 28This object, which is an implicit parameter to every GL function, stores all 29of the GL related state for the application. Every texture, every buffer 30object, every enable, and much, much more is stored in the context. Since 31an application can have more than one context, the context to be used is 32selected by a window-system dependent function such as 33<tt>glXMakeContextCurrent</tt>.</p> 34 35<p>In environments that implement OpenGL with X-Windows using GLX, every GL 36function, including the pointers returned by <tt>glXGetProcAddress</tt>, are 37<em>context independent</em>. This means that no matter what context is 38currently active, the same <tt>glVertex3fv</tt> function is used.</p> 39 40<p>This creates the first bit of dispatch complexity. An application can 41have two GL contexts. One context is a direct rendering context where 42function calls are routed directly to a driver loaded within the 43application's address space. The other context is an indirect rendering 44context where function calls are converted to GLX protocol and sent to a 45server. The same <tt>glVertex3fv</tt> has to do the right thing depending 46on which context is current.</p> 47 48<p>Highly optimized drivers or GLX protocol implementations may want to 49change the behavior of GL functions depending on current state. For 50example, <tt>glFogCoordf</tt> may operate differently depending on whether 51or not fog is enabled.</p> 52 53<p>In multi-threaded environments, it is possible for each thread to have a 54different GL context current. This means that poor old <tt>glVertex3fv</tt> 55has to know which GL context is current in the thread where it is being 56called.</p> 57 58<h2 id="overview">2. Overview of Mesa's Implementation</h2> 59 60<p>Mesa uses two per-thread pointers. The first pointer stores the address 61of the context current in the thread, and the second pointer stores the 62address of the <em>dispatch table</em> associated with that context. The 63dispatch table stores pointers to functions that actually implement 64specific GL functions. Each time a new context is made current in a thread, 65these pointers a updated.</p> 66 67<p>The implementation of functions such as <tt>glVertex3fv</tt> becomes 68conceptually simple:</p> 69 70<ul> 71<li>Fetch the current dispatch table pointer.</li> 72<li>Fetch the pointer to the real <tt>glVertex3fv</tt> function from the 73table.</li> 74<li>Call the real function.</li> 75</ul> 76 77<p>This can be implemented in just a few lines of C code. The file 78<tt>src/mesa/glapi/glapitemp.h</tt> contains code very similar to this.</p> 79 80<blockquote> 81<table border="1"> 82<tr><td><pre> 83void glVertex3f(GLfloat x, GLfloat y, GLfloat z) 84{ 85 const struct _glapi_table * const dispatch = GET_DISPATCH(); 86 87 (*dispatch->Vertex3f)(x, y, z); 88}</pre></td></tr> 89<tr><td>Sample dispatch function</td></tr></table> 90</blockquote> 91 92<p>The problem with this simple implementation is the large amount of 93overhead that it adds to every GL function call.</p> 94 95<p>In a multithreaded environment, a naive implementation of 96<tt>GET_DISPATCH</tt> involves a call to <tt>pthread_getspecific</tt> or a 97similar function. Mesa provides a wrapper function called 98<tt>_glapi_get_dispatch</tt> that is used by default.</p> 99 100<h2>3. Optimizations</h2> 101 102<p>A number of optimizations have been made over the years to diminish the 103performance hit imposed by GL dispatch. This section describes these 104optimizations. The benefits of each optimization and the situations where 105each can or cannot be used are listed.</p> 106 107<h3>3.1. Dual dispatch table pointers</h3> 108 109<p>The vast majority of OpenGL applications use the API in a single threaded 110manner. That is, the application has only one thread that makes calls into 111the GL. In these cases, not only do the calls to 112<tt>pthread_getspecific</tt> hurt performance, but they are completely 113unnecessary! It is possible to detect this common case and avoid these 114calls.</p> 115 116<p>Each time a new dispatch table is set, Mesa examines and records the ID 117of the executing thread. If the same thread ID is always seen, Mesa knows 118that the application is, from OpenGL's point of view, single threaded.</p> 119 120<p>As long as an application is single threaded, Mesa stores a pointer to 121the dispatch table in a global variable called <tt>_glapi_Dispatch</tt>. 122The pointer is also stored in a per-thread location via 123<tt>pthread_setspecific</tt>. When Mesa detects that an application has 124become multithreaded, <tt>NULL</tt> is stored in <tt>_glapi_Dispatch</tt>.</p> 125 126<p>Using this simple mechanism the dispatch functions can detect the 127multithreaded case by comparing <tt>_glapi_Dispatch</tt> to <tt>NULL</tt>. 128The resulting implementation of <tt>GET_DISPATCH</tt> is slightly more 129complex, but it avoids the expensive <tt>pthread_getspecific</tt> call in 130the common case.</p> 131 132<blockquote> 133<table border="1"> 134<tr><td><pre> 135#define GET_DISPATCH() \ 136 (_glapi_Dispatch != NULL) \ 137 ? _glapi_Dispatch : pthread_getspecific(&_glapi_Dispatch_key) 138</pre></td></tr> 139<tr><td>Improved <tt>GET_DISPATCH</tt> Implementation</td></tr></table> 140</blockquote> 141 142<h3>3.2. ELF TLS</h3> 143 144<p>Starting with the 2.4.20 Linux kernel, each thread is allocated an area 145of per-thread, global storage. Variables can be put in this area using some 146extensions to GCC. By storing the dispatch table pointer in this area, the 147expensive call to <tt>pthread_getspecific</tt> and the test of 148<tt>_glapi_Dispatch</tt> can be avoided.</p> 149 150<p>The dispatch table pointer is stored in a new variable called 151<tt>_glapi_tls_Dispatch</tt>. A new variable name is used so that a single 152libGL can implement both interfaces. This allows the libGL to operate with 153direct rendering drivers that use either interface. Once the pointer is 154properly declared, <tt>GET_DISPACH</tt> becomes a simple variable 155reference.</p> 156 157<blockquote> 158<table border="1"> 159<tr><td><pre> 160extern __thread struct _glapi_table *_glapi_tls_Dispatch 161 __attribute__((tls_model("initial-exec"))); 162 163#define GET_DISPATCH() _glapi_tls_Dispatch 164</pre></td></tr> 165<tr><td>TLS <tt>GET_DISPATCH</tt> Implementation</td></tr></table> 166</blockquote> 167 168<p>Use of this path is controlled by the preprocessor define 169<tt>GLX_USE_TLS</tt>. Any platform capable of using TLS should use this as 170the default dispatch method.</p> 171 172<h3>3.3. Assembly Language Dispatch Stubs</h3> 173 174<p>Many platforms has difficulty properly optimizing the tail-call in the 175dispatch stubs. Platforms like x86 that pass parameters on the stack seem 176to have even more difficulty optimizing these routines. All of the dispatch 177routines are very short, and it is trivial to create optimal assembly 178language versions. The amount of optimization provided by using assembly 179stubs varies from platform to platform and application to application. 180However, by using the assembly stubs, many platforms can use an additional 181space optimization (see <a href="#fixedsize">below</a>).</p> 182 183<p>The biggest hurdle to creating assembly stubs is handling the various 184ways that the dispatch table pointer can be accessed. There are four 185different methods that can be used:</p> 186 187<ol> 188<li>Using <tt>_glapi_Dispatch</tt> directly in builds for non-multithreaded 189environments.</li> 190<li>Using <tt>_glapi_Dispatch</tt> and <tt>_glapi_get_dispatch</tt> in 191multithreaded environments.</li> 192<li>Using <tt>_glapi_Dispatch</tt> and <tt>pthread_getspecific</tt> in 193multithreaded environments.</li> 194<li>Using <tt>_glapi_tls_Dispatch</tt> directly in TLS enabled 195multithreaded environments.</li> 196</ol> 197 198<p>People wishing to implement assembly stubs for new platforms should focus 199on #4 if the new platform supports TLS. Otherwise, implement #2 followed by 200#3. Environments that do not support multithreading are uncommon and not 201terribly relevant.</p> 202 203<p>Selection of the dispatch table pointer access method is controlled by a 204few preprocessor defines.</p> 205 206<ul> 207<li>If <tt>GLX_USE_TLS</tt> is defined, method #3 is used.</li> 208<li>If <tt>HAVE_PTHREAD</tt> is defined, method #2 is used.</li> 209<li>If none of the preceding are defined, method #1 is used.</li> 210</ul> 211 212<p>Two different techniques are used to handle the various different cases. 213On x86 and SPARC, a macro called <tt>GL_STUB</tt> is used. In the preamble 214of the assembly source file different implementations of the macro are 215selected based on the defined preprocessor variables. The assembly code 216then consists of a series of invocations of the macros such as: 217 218<blockquote> 219<table border="1"> 220<tr><td><pre> 221GL_STUB(Color3fv, _gloffset_Color3fv) 222</pre></td></tr> 223<tr><td>SPARC Assembly Implementation of <tt>glColor3fv</tt></td></tr></table> 224</blockquote> 225 226<p>The benefit of this technique is that changes to the calling pattern 227(i.e., addition of a new dispatch table pointer access method) require fewer 228changed lines in the assembly code.</p> 229 230<p>However, this technique can only be used on platforms where the function 231implementation does not change based on the parameters passed to the 232function. For example, since x86 passes all parameters on the stack, no 233additional code is needed to save and restore function parameters around a 234call to <tt>pthread_getspecific</tt>. Since x86-64 passes parameters in 235registers, varying amounts of code needs to be inserted around the call to 236<tt>pthread_getspecific</tt> to save and restore the GL function's 237parameters.</p> 238 239<p>The other technique, used by platforms like x86-64 that cannot use the 240first technique, is to insert <tt>#ifdef</tt> within the assembly 241implementation of each function. This makes the assembly file considerably 242larger (e.g., 29,332 lines for <tt>glapi_x86-64.S</tt> versus 1,155 lines for 243<tt>glapi_x86.S</tt>) and causes simple changes to the function 244implementation to generate many lines of diffs. Since the assembly files 245are typically generated by scripts (see <a href="#autogen">below</a>), this 246isn't a significant problem.</p> 247 248<p>Once a new assembly file is created, it must be inserted in the build 249system. There are two steps to this. The file must first be added to 250<tt>src/mesa/sources</tt>. That gets the file built and linked. The second 251step is to add the correct <tt>#ifdef</tt> magic to 252<tt>src/mesa/glapi/glapi_dispatch.c</tt> to prevent the C version of the 253dispatch functions from being built.</p> 254 255<h3 id="fixedsize">3.4. Fixed-Length Dispatch Stubs</h3> 256 257<p>To implement <tt>glXGetProcAddress</tt>, Mesa stores a table that 258associates function names with pointers to those functions. This table is 259stored in <tt>src/mesa/glapi/glprocs.h</tt>. For different reasons on 260different platforms, storing all of those pointers is inefficient. On most 261platforms, including all known platforms that support TLS, we can avoid this 262added overhead.</p> 263 264<p>If the assembly stubs are all the same size, the pointer need not be 265stored for every function. The location of the function can instead be 266calculated by multiplying the size of the dispatch stub by the offset of the 267function in the table. This value is then added to the address of the first 268dispatch stub.</p> 269 270<p>This path is activated by adding the correct <tt>#ifdef</tt> magic to 271<tt>src/mesa/glapi/glapi.c</tt> just before <tt>glprocs.h</tt> is 272included.</p> 273 274<h2 id="autogen">4. Automatic Generation of Dispatch Stubs</h2> 275 276</div> 277</body> 278</html> 279