Name ARB_get_program_binary Name Strings GL_ARB_get_program_binary Contributors Acorn Pooley Aske Simon Christensen Bruce Merry David Garcia Eric Werness Georg Kolling Greg Roth Jason Green Jeff Bolz Jeremy Sandmel Joey Blankenship Jon Leech Mark Callow Pat Brown Robert Simpson Tom Olson Contact Benj Lipchak, APPLE (lipchak 'at' apple.com) Greg Roth, NVIDIA (groth 'at' nvidia.com) Piers Daniell, NVIDIA (pdaniell 'at' nvidia.com) Notice Copyright (c) 2010-2016 The Khronos Group Inc. Copyright terms at http://www.khronos.org/registry/speccopyright.html Specification Update Policy Khronos-approved extension specifications are updated in response to issues and bugs prioritized by the Khronos OpenGL Working Group. For extensions which have been promoted to a core Specification, fixes will first appear in the latest version of that core Specification, and will eventually be backported to the extension document. This policy is described in more detail at https://www.khronos.org/registry/OpenGL/docs/update_policy.php Status Complete. Approved by the ARB on June 9, 2010. Approved by the Khronos Board of Promoters on July 23, 2010. Version Last Modified Date: January 11, 2019 Revision: #10 Number ARB Extension #96 Dependencies OpenGL 3.0 is required. Written based on the wording of the OpenGL 3.2 compatibility specification. Overview This extension introduces new commands to retrieve and set the binary representation of a program object. GetProgramBinary allows an application to cache compiled and linked programs to avoid compiling and linking when used again. This may even allow the GL itself to act as an offline compiler. The resulting program binary can be reloaded into the GL via ProgramBinary. This is a very useful path for applications that wish to remain portable by shipping pure GLSL source shaders, yet would like to avoid the cost of compiling their shaders at runtime. Instead an application can supply its GLSL source shaders during first application run, or even during installation. The application then compiles and links its shaders and reads back the program binaries. On subsequent runs, only the program binaries need be supplied. ProgramBinary may also accept binaries in vendor-specific formats produced by specialized offline compilation tools. This extension does not add any such formats, but allows for them in further extensions. Though the level of optimization may not be identical -- the offline shader compiler may have the luxury of more aggressive optimization at its disposal -- program binaries generated online by the GL are interchangeable with those generated offline by an SDK tool. IP Status No known IP claims. New Procedures and Functions void GetProgramBinary(uint program, sizei bufSize, sizei *length, enum *binaryFormat, void *binary); void ProgramBinary(uint program, enum binaryFormat, const void *binary, sizei length); void ProgramParameteri(uint program, enum pname, int value); New Tokens Accepted by the parameter of ProgramParameteri and GetProgramiv: PROGRAM_BINARY_RETRIEVABLE_HINT 0x8257 Accepted by the parameter of GetProgramiv: PROGRAM_BINARY_LENGTH 0x8741 Accepted by the parameter of GetBooleanv, GetIntegerv, GetInteger64v, GetFloatv and GetDoublev: NUM_PROGRAM_BINARY_FORMATS 0x87FE PROGRAM_BINARY_FORMATS 0x87FF Additions to Chapter 2 of the OpenGL 3.2 Specification (OpenGL Operation) Update section 2.14 "Vertex Shaders", insert before the last sentence of the third paragraph: "... which generates executable code from all the compiled shader objects attached to the program. Alternatively, pre-compiled program binary code may be directly loaded into a program object. When a linked program object is used..." Add the following paragraphs above the description of DeleteProgram, p. 86: To set a program object parameter, call void ProgramParameteri(uint program, enum pname, int value) identifies which parameter to set for . holds the value being set. If is PROGRAM_BINARY_RETRIEVABLE_HINT, must be TRUE or FALSE, and indicates whether a program binary is likely to be retrieved later, as described for ProgramBinary in section 2.14.3. This hint will not take effect until the next time LinkProgram or ProgramBinary has been called successfully. Add section 2.14.3, Program Binaries "The command void GetProgramBinary(uint program, sizei bufSize, sizei *length, enum *binaryFormat, void *binary); returns a binary representation of the program object's compiled and linked executable source, henceforth referred to as its program binary. The maximum number of bytes that may be written into is specified by . If is less than the number of bytes in the program binary, then an INVALID_OPERATION error is thrown. Otherwise, the actual number of bytes written into is returned in and its format is returned in . If is NULL, then no length is returned. The number of bytes in the program binary can be queried by calling GetProgramiv with PROGRAM_BINARY_LENGTH. When a program object's LINK_STATUS is FALSE, its program binary length is zero, and a call to GetProgramBinary will generate an INVALID_OPERATION error. The command void ProgramBinary(uint program, enum binaryFormat, const void *binary, sizei length); loads a program object with a program binary previously returned from GetProgramBinary. This is useful for future instantiations of the GL to avoid online compilation, while still using OpenGL Shading Language source shaders as a portable initial format. and must be those returned by a previous call to GetProgramBinary, and must be the length of the program binary as returned by GetProgramBinary or GetProgramiv with PROGRAM_BINARY_LENGTH. Loading the program binary will fail, setting the LINK_STATUS of to FALSE, if these conditions are not met. Loading a program binary may also fail if the implementation determines that there has been a change in hardware or software configuration from when the program binary was produced such as having been compiled with an incompatible or outdated version of the compiler. In this case the application should fall back to providing the original OpenGL Shading Language source shaders, and perhaps again retrieve the program binary for future use. A program object's program binary is replaced by calls to LinkProgram or ProgramBinary. Where linking success or failure is concerned, ProgramBinary can be considered to perform an implicit linking operation. LinkProgram and ProgramBinary both set the program object's LINK_STATUS to TRUE or FALSE, as queried with GetProgramiv, to reflect success or failure and update the information log, queried with GetProgramInfoLog, to provide details about warnings or errors. A successful call to ProgramBinary will reset all uniform variables to their initial values. The initial value is either the value of the variable's initializer as specified in the original shader source, or 0 if no initializer was present. Additionally, all vertex shader input and fragment shader output assignments that were in effect when the program was linked before saving are restored when ProgramBinary is called. If ProgramBinary fails to load a binary, no error is generated, but any information about a previous link or load of that program object is lost. Thus, a failed load does not restore the old state of . The failure does not alter other program state not affected by linking such as the attached shaders, and the vertex attribute and fragment data location bindings as set by BindAttribLocation and BindFragDataLocation. Queries of values NUM_PROGRAM_BINARY_FORMATS and PROGRAM_BINARY_FORMATS return the number of program binary formats and the list of program binary format values supported by an implementation. The returned by GetProgramBinary must be present in this list. Any program binary retrieved using GetProgramBinary and submitted using ProgramBinary under the same configuration must be successful. Any programs loaded successfully by ProgramBinary must be run properly with any legal GL state vector. If a GL implementation needs to recompile or otherwise modify program executables based on GL state outside the program, GetProgramBinary is required to save enough information to allow such recompilation. To indicate that a program binary is likely to be retrieved, ProgramParameteri should be called with PROGRAM_BINARY_RETRIEVABLE_HINT and TRUE. This setting will not be in effect until the next time LinkProgram or ProgramBinary has been called successfully. Additionally, GetProgramBinary calls may be deferred until after using the program with all non-program state vectors that it is likely to encounter. Such deferral may allow implementations to save additional information in the program binary that would minimize recompilation in future uses of the program binary." Modify Section 2.14.8, Required State Add the following bullet to the state required per program object: * boolean holding the hint to the retrievability of the program binary, initially FALSE. Additions to Chapter 6 of the OpenGL 3.2 Specification (State and State Requests) Modify section 6.1.16, Shader and Program Objects Add to the end of the description of GetProgramiv, p. 385: If is PROGRAM_BINARY_RETRIEVABLE_HINT, the current value of whether the binary retrieval hint is enabled for is returned. GLX Protocol None. Errors An INVALID_VALUE error is generated if the argument to GetProgramBinary, ProgramBinary, or ProgramParameteri is not the name of a program object previously created with CreateProgram. An INVALID_ENUM error is generated if the argument to ProgramParameteri is not PROGRAM_BINARY_RETRIEVABLE_HINT. An INVALID_VALUE error is generated if the argument to ProgramParameteri is not TRUE or FALSE. An INVALID_OPERATION error is generated if GetProgramBinary is called when the program object, , does not contain a valid program binary as reflected by its LINK_STATUS state; if is not big enough to contain the entire program binary; or if the value of NUM_PROGRAM_BINARY_FORMATS is zero. New State (table 6.42, Program Object State) add the following: Get Value Type Get Command Initial Value Description Section ------------- ---- ----------- ------------- ----------- ------- PROGRAM_BINARY_LENGTH Z+ GetProgramiv 0 Length of program binary 2.14.3 PROGRAM_BINARY_RETRIEVABLE_HINT Z GetProgramiv FALSE Retriavable binary hint enabled 2.14.3 - Z1 GetProgramBinary N/A Binary representation of program 2.14.3 (table 6.51, Implementation Dependent Values) add the following: Get Value Type Get Command Minimum Value Description Section ------------- ---- ----------- ------------- ----------- ------- PROGRAM_BINARY_FORMATS 0* x Z GetIntegerv N/A Enumerated program binary formats 2.14.3 NUM_PROGRAM_BINARY_FORMATS Z GetIntegerv 0 Number of program binary formats 2.14.3 Sample Usage void retrieveProgramBinary(const GLchar* vsSource, const GLchar* fsSource, const char* myBinaryFileName, GLenum* binaryFormat) { GLuint newFS, newVS; GLuint newProgram; const GLchar* sources[1]; GLint success; GLint binaryLength; void* binary; FILE* outfile; // // Create new shader/program objects and attach them together. // newVS = glCreateShader(GL_VERTEX_SHADER); newFS = glCreateShader(GL_FRAGMENT_SHADER); newProgram = glCreateProgram(); glAttachShader(newProgram, newVS); glAttachShader(newProgram, newFS); // // Supply GLSL source shaders, compile, and link them // sources[0] = vsSource; glShaderSource(newVS, 1, sources, NULL); glCompileShader(newVS); sources[0] = fsSource; glShaderSource(newFS, 1, sources, NULL); glCompileShader(newFS); glProgramParameteri(newProgram, GL_PROGRAM_BINARY_RETRIEVABLE_HINT, GL_TRUE); glLinkProgram(newProgram); glGetProgramiv(newProgram, GL_LINK_STATUS, &success); if (!success) { // // Fallback to simpler source shaders? Take my toys and go home? // } glUseProgram(newProgram); // // Perform rendering and state changes likely to be encountered. // DoRendering(newProgram); // // Retrieve the binary from the program object // glGetProgramiv(newProgram, GL_PROGRAM_BINARY_LENGTH, &binaryLength); binary = (void*)malloc(binaryLength); glGetProgramBinary(newProgram, binaryLength, NULL, binaryFormat, binary); // // Cache the program binary for future runs // outfile = fopen(myBinaryFileName, "wb"); fwrite(binary, binaryLength, 1, outfile); fclose(outfile); free(binary); // // Clean up // glDeleteShader(newVS); glDeleteShader(newFS); glDeleteProgram(newProgram); } void loadProgramBinary(const char* myBinaryFileName, GLenum binaryFormat, GLuint progObj) { GLint binaryLength; void* binary; GLint success; FILE* infile; // // Read the program binary // infile = fopen(myBinaryFileName, "rb"); fseek(infile, 0, SEEK_END); binaryLength = (GLint)ftell(infile); binary = (void*)malloc(binaryLength); fseek(infile, 0, SEEK_SET); fread(binary, binaryLength, 1, infile); fclose(infile); // // Load the binary into the program object -- no need to link! // glProgramBinary(progObj, binaryFormat, binary, binaryLength); free(binary); glGetProgramiv(progObj, GL_LINK_STATUS, &success); if (!success) { // // Something must have changed since the program binaries // were cached away. Fallback to source shader loading path, // and then retrieve and cache new program binaries once again. // } } Issues 1. Do we need to consider state dependencies when using this extension? RESOLVED: A little. A program binary retrieved with GetProgramBinary can be expected to work regardless of the current GL state in effect at the time it was retrieved with GetProgramBinary, loaded with ProgramBinary, installed as part of render state with UseProgram, or used for drawing with DrawArrays or DrawElements. However, in many cases, internal state dependencies affect the way source shaders are compiled. State considerations may indicate whether optimizations are available or if workarounds are necessary. In consideration of these issues, users of this extension should be advised that querying the program binary after the full range of state configurations have been seen may allow the implementation to include optimized versions and avoid recompiles that may affect performance. 2. How are shader objects involved, if at all? RESOLVED: Shader objects play no part in the specification of program binaries. (See also Issue 3.) The program binary retrieved by GetProgramBinary is the one installed during the most recent call to LinkProgram or ProgramBinary, i.e. the one which would go into effect if we were to call UseProgram. Attaching different shader objects after the most recent call to LinkProgram is inconsequential. 3. Should we throw an error as a programming aid if there are shader objects attached to the program object when ProgramBinary is called? RESOLVED: No, they are irrelevant but harmless, and GL precedent is to throw errors on bad state combinations, not on harmless ones. Besides, the programmer should discover pretty quickly that they're getting the wrong shader, if they accidentally called ProgramBinary instead of LinkProgram. Also, an app may intentionally leave the attachments in place if it for some reason is switching back and forth between loading a program object with program binaries, and loading it with compiled GLSL shaders. 4. Where are the binary formats defined and described? RESOLVED: This extension provides a common infrastructure for retrieving and loading program binaries. A vendor extension must also be present in order to define one or more binary formats, thereby populating the list of PROGRAM_BINARY_FORMATS. The returned by GetProgramBinary is always one of the binary formats in this list. If ProgramBinary is called with a not in this list, the implementation will throw an INVALID_ENUM error. The beauty of this extension, however, is that an application does not need to be aware of the vendor extension on any given implementation. It only needs to retrieve a program binary with an anonymous and resupply that same when loading the program binary. 5. Under what conditions might a call to ProgramBinary fail? RESOLVED: Even if a program binary is successfully retrieved with GetProgramBinary and then in a future run the program binary is resupplied with ProgramBinary, and all of the parameters are correct, the program binary load may still fail. This can happen if there has been a change to the hardware or software on the system, such as a hardware upgrade or driver update. In this case the PROGRAM_BINARY_FORMATS list may no longer contain the binary format associated with the cached program binary, and INVALID_ENUM will be thrown if the cached program binary format is passed into ProgramBinary anyway. Even if the cached program binary format is still valid, ProgramBinary may still fail to load the cached binary. This is the driver's way of signaling to the app that it needs to recompile and recache its program binaries because there has been some important change to the online compiler, such as a bug fix or a significant new optimization. 6. Can BindAttribLocation be called after ProgramBinary to remap an attribute location used by the program binary? RESOLVED: No. BindAttribLocation only affects the result of a subsequent call to LinkProgram. LinkProgram operates on the attached shader objects and replaces any program binary loaded prior to LinkProgram. So there is no mechanism to remap an attribute location after loading a program binary. However, an application is free to remap an attribute location prior to retrieving the program binary. By calling BindAttribLocation followed by LinkProgram, an application can remap the attribute location. If this is followed by a call to GetProgramBinary, the retrieved program binary will include the desired attribute location assignment. 7. How does this spec differ from the OES_get_program_binary and why? To better facilitate state-dependent optimizations and workarounds, a hint is provided to ensure that the implementation includes any state-based variants it might encounter. These dependencies also encourage the programmer to retrieve binary programs late in the execution, after relevant state changes have already occurred. There are other areas where language was clarified, but this is meant to be consistent with the intent of the original specification and not to alter previously established behavior. 8. How does ProgramBinary interact with uniform values, including shader-specified defaults? RESOLVED: All uniforms specified in the binary are reset to their shader- specified defaults, or to zero if they aren't specified, when the program binary is loaded. The spec language has been updated to specify this behavior. 9. Should ProgramBinary be expected to load a previously saved program binary even if non-program state has changed enough such that a recompile is needed by the GL implementation. RESOLVED: Yes. Given the same configuration, for example hardware, driver version, etc., the GL implementation should save as much data as it needs with the program binary such that if a recompile is necessary it has all the data it needs to do this. Only if the configuration changes, it is acceptable to fail ProgramBinary. The application may use the PROGRAM_BINARY_RETRIEVABLE_HINT hint to indicate to the GL implementation that this program will likely be saved with GetProgramBinary at some point. This will give the GL implementation the opportunity to track any state changes made to the program before being saved such that when it is loaded again a recompile can be avoided. Revision History 10 01/11/2019 Add an error for ProgramBinary if there are no binary formats (Bug 16155). 09 07/05/2016 Fix missing GL_ prefix in example code. 08 10/08/2013 Change GLvoid -> void (Bug 10412). 07 06/14/2012 Clean up descriptions of "failed binary", add errors, and move HINT description back to ProgramParameteri. 06 07/19/2010 Fixed a teenie typo with OES left on ProgramBinaryOES in issue #3. 05 05/26/2010 Modifications based on feedback from Pat Brown and the ARB f2f. 04 05/17/2010 Restore the old "int" type for the length parameter to ProgramBinary since matching the ratified GLES is more important in this case. 03 05/10/2010 Various minor cleanup based on feedback from Bruce. 02 04/09/2010 Converted to ARB version for inclusion in OpenGL 4.1. 01 01/11/2010 Initial EXT version.