1g------------------------------------------------------------------------- 2drawElements Quality Program Test Specification 3----------------------------------------------- 4 5Copyright 2014 The Android Open Source Project 6 7Licensed under the Apache License, Version 2.0 (the "License"); 8you may not use this file except in compliance with the License. 9You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13Unless required by applicable law or agreed to in writing, software 14distributed under the License is distributed on an "AS IS" BASIS, 15WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16See the License for the specific language governing permissions and 17limitations under the License. 18------------------------------------------------------------------------- 19 Buffer upload performance tests 20 21Tests: 22 + dEQP-GLES3.performance.buffer.data_upload.* 23 24Includes: 25 + Memcpy performance tests for reference values 26 + Performance of glBufferData and glBufferSubData 27 - STREAM, STATIC and DYNAMIC buffer usages 28 - Target buffer in different states: 29 * BufferData: 30 ~ Just generated buffer 31 ~ Generated buffer with unspecified contents (bufferData(..., NULL)) 32 ~ Generated buffer with specified contents 33 ~ Buffer (with the same size) that has been used in drawing earlier 34 ~ Buffer (with a larger size) that has been used in drawing earlier 35 * BufferSubData: 36 ~ Respecify whole buffer contents 37 ~ Respecify half of the buffer 38 ~ Invalidate buffer contents manually (bufferData(..., NULL)) before 39 calling bufferSubData 40 + Buffer mapping performance 41 - STREAM, STATIC and DYNAMIC buffer usages 42 - Full and partial (half of the buffer) mapping 43 - Different map bits: 44 * Write 45 * Write-Read 46 * Write-InvalidateRange 47 * Write-InvalidateBuffer 48 * Write-Unsynchronized 49 * Write-Unsynchronized-InvalidateBuffer 50 * Write-FlushExplicit 51 - Target buffer in different states: 52 * Generated buffer with unspecified contents (bufferData(..., NULL)) 53 * Generated buffer with specified contents 54 * A buffer (with the same size) that has been used in drawing earlier 55 - Different mapping & flushing strategies: 56 * Map whole buffer and flush whole mapped area (in one call) 57 * Map buffer partially (half) and flush whole mapped area (in one call) 58 * Map whole buffer and flush buffer partially (half) (in one call) 59 * Map whole buffer and flush whole mapped area in 4 non-interleaving 60 flush calls 61 + Buffer upload performance after a draw call using the target buffer 62 - STREAM, STATIC and DYNAMIC buffer usages 63 - BufferData, BufferSubData, MapBufferRange methods 64 + Uploaded buffer rendering performance tests 65 66Excludes: 67 + Exhaustive testing of all combinations of usages, buffer states, and methods 68 + Complex buffer usage before upload (e.g transform feedback, copyBufferSubData) 69 + Sustained upload performance tests 70 71Description: 72 73Upload performance tests approximate upload speed by testing time usage of different 74buffer upload functions with multiple different buffer sizes. The time usage of these 75functions is expected to be of form t(x) = b * x + c, where x is the buffer size, b 76is the linear cost of the function and c is the constant cost of the function. Tests 77find suitable parameters b and c to fit the sampled values and then report these 78parameters. The test result of an upload test is the median transfer rate of the test 79samples unless stated otherwise. 80 81Additionally, test cases calculate "sample linearity" and "sample temporal stability" 82metrics. Sample linearity describes the curvature of the sampled values and it is 83calculated by comparing the line fitted to the first half of the samples (the half with 84smaller buffer sizes) to the line fitted to the second half of the samples. A straight 85line would result in 100% linearity. A linearity value less than 90% usually implies 86either large errors in sampling (noise), insufficient timer resolution, or non-linearity 87of the function. 88 89Sample temporal stability describes the stability of the function during the sampling. 90Samples are split to two categories: early and late samples. All early samples are tested 91before any late sample. A line is fitted to the early samples and it is compared to a 92line fitted to the late samples. If the samples get similar values in the beginning of 93the test and in the end of the test, the temporal stability will be 100%. A value less 94than 90% usually implies that test samples are not independent from other samples, or 95that there were large errors in sampling (noise). 96 97(Note that all logged data sizes are MiBs (mebibytes) even if they are marked with unit 98symbol "MB".) 99 100reference.* tests test memcpy with different sizes. The transfer rate may be useful as a 101reference value. "small_buffers" test case tests memcpy performance with transfer size 102in range 0 - 256KiB. The result is the median transfer rate of the test samples. 103"large_buffers" test case measures the transfer rate with the transfer size in range 104256KiB - 16 MiB. The result is the approximated transfer rate of an infinitely large 105transfer size calculated form the fitted line. (i.e c = 0). 106 107function_call.* tests measure the time taken by the corresponding command or the series of 108commands with different buffer sizes. The result is the median transfer rate of the test 109samples. 110 111modify_after_use.* tests are similar to the function_call.* test but they target a buffer 112that has been used in a draw operation just before the measured function call. "_repeated" 113variants repeat the "specify buffer contents, draw buffer" sequence a few times before 114the test target function is called. 115 116render_after_upload.* tests measure rendering time of a buffer in different scenarios. The 117tests measure time from rendering command issuance (glDraw* call) to finished rendering 118(return from glReadPixels call). The purpose of the tests is to measure the data transfer 119performance from the observation of a buffer upload completion (a return from a 120glBuffer(Sub)Data / glUnmap) to observation of finished rendering. This combined rate of 121data transfer and rendering is in these tests referred to as "processing rate". Note that 122"processing time" does not include the upload time. All tests and test samples have the 123same number of visible fragments but may have different number of vertices. The tests are 124NOT rendering performance tests and should not be treated as such. 125 126Each case's draw function, uploaded buffer type, and upload method is encoded to the case 127name. For example "draw_elements_upload_indices_with_buffer_data" would specify index 128(GL_ELEMENT_ARRAY_BUFFER) buffer contents with glBufferData, and issue render with 129glDrawElements call. 130 131render_after_upload.reference.draw.* tests produce baseline results for basic functions. 132"read_pixels" case measures the constant cost (in microseconds) of a single glReadPixels 133call with identical arguments as in other rendering tests. The result might used for 134excluding ReadPixels' constant cost from the total constant costs in the other test 135results. "draw_arrays" and "draw_elements" tests measure the processing rate in the case 136where there is no buffer uploading required. As such, they are expected to result in the 137highest processing rates. The results should be interpreted as a reference values for 138render_after_upload.upload_and_draw.* group tests. 139 140render_after_upload.reference.draw_upload_draw.* cases first render from an existing 141buffer, upload another buffer, and then render using the just uploaded buffer. The 142processing rate (excluding the upload time) is measured. The results are baseline 143reference results for render_after_upload.draw_modify_draw.* group tests. 144 145render_after_upload.upload_unrelated_and_draw.* tests upload an unrelated buffer and 146then render to the viewport. The rendering does not depend on the uploaded buffer. The 147unrelated upload size is equal to the rendering workload size. 148 149render_after_upload.upload_and_draw.* cases upload a buffer and then render using it. In 150used_buffer.* group cases, the buffer has been used in rendering before upload. In 151new_buffer.* group cases, the uploaded buffer is generated just before uploading. In this 152case if the upload method is not glBufferData, glBufferData is called with NULL data 153argument before using the actual upload method for specifying the buffer contents. In 154cases in test case groups "*_and_unrelated_upload", another buffer is uploaded after the 155target buffer upload with glBufferData. The rendering does not depend on this second 156upload. The unrelated upload size is equal to the rendering workload size. 157 158render_after_upload.draw_modify_draw.* cases draw from a buffer, respecify its contents, 159and draw again using the buffer. Cases are identical to 160render_after_upload.reference.draw_upload_draw.* cases with the exception that the upload 161target buffer is used in rendering just before issuing upload commands. 162 163render_after_upload.upload_wait_draw.* cases upload a buffer, wait a certain number of 164frames (buffer swaps), and then render using the buffer. The number of frames required 165for the render times to stabilize is measured and reported as a test result. For example, 166result value of 0 would indicate that rendering is just as fast just after upload than 167N frames later. A value of 1 would mean that performance of rendering from the buffer 168is different during the frame it was uploaded in, but does not change after one frame 169swap. 170 171Since the frame times during the "waiting" period may be significantly lower than in the 172upload frame or in the frame of the final rendering, in order to keep frame times 173consistent, the test code performs dummy calculations (busy waits) for 8 milliseconds in 174each frame. Also during the "dummy" frames, a simple grid pattern is drawn to the viewport. 175