1# Performance benchmarks 2 3This document lists TensorFlow Lite performance benchmarks when running well 4known models on some Android and iOS devices. 5 6These performance benchmark numbers were generated with the 7[Android TFLite benchmark binary](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark) 8and the [iOS benchmark app](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark/ios). 9 10# Android performance benchmarks 11 12For Android benchmarks, the CPU affinity is set to use big cores on the device to 13reduce variance (see [details](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#reducing-variance-between-runs-on-android)). 14 15It assumes that models were download and unzipped to the 16`/data/local/tmp/tflite_models` directory. The benchmark binary is built 17using [these instructions](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#on-android) 18and assumed in the `/data/local/tmp` directory. 19 20To run the benchmark: 21 22``` 23adb shell /data/local/tmp/benchmark_model \ 24 --num_threads=4 \ 25 --graph=/data/local/tmp/tflite_models/${GRAPH} \ 26 --warmup_runs=1 \ 27 --num_runs=50 28``` 29 30To run with nnapi delegate, please set --use_nnapi=true. To run with gpu 31delegate, please set --use_gpu=true. 32 33The performance values below are measured on Android 10. 34 35<table> 36 <thead> 37 <tr> 38 <th>Model Name</th> 39 <th>Device </th> 40 <th>CPU, 4 threads</th> 41 <th>GPU</th> 42 <th>NNAPI</th> 43 </tr> 44 </thead> 45 <tr> 46 <td rowspan = 2> 47 <a href="https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz">Mobilenet_1.0_224(float)</a> 48 </td> 49 <td>Pixel 3 </td> 50 <td>23.9 ms</td> 51 <td>6.45 ms</td> 52 <td>13.8 ms</td> 53 </tr> 54 <tr> 55 <td>Pixel 4 </td> 56 <td>14.0 ms</td> 57 <td>9.0 ms</td> 58 <td>14.8 ms</td> 59 </tr> 60 <tr> 61 <td rowspan = 2> 62 <a href="https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz">Mobilenet_1.0_224 (quant)</a> 63 </td> 64 <td>Pixel 3 </td> 65 <td>13.4 ms</td> 66 <td>--- </td> 67 <td>6.0 ms</td> 68 </tr> 69 <tr> 70 <td>Pixel 4 </td> 71 <td>5.0 ms</td> 72 <td>--- </td> 73 <td>3.2 ms</td> 74 </tr> 75 <tr> 76 <td rowspan = 2> 77 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/nasnet_mobile_2018_04_27.tgz">NASNet mobile</a> 78 </td> 79 <td>Pixel 3 </td> 80 <td>56 ms</td> 81 <td>--- </td> 82 <td>102 ms</td> 83 </tr> 84 <tr> 85 <td>Pixel 4 </td> 86 <td>34.5 ms</td> 87 <td>--- </td> 88 <td>99.0 ms</td> 89 </tr> 90 <tr> 91 <td rowspan = 2> 92 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/squeezenet_2018_04_27.tgz">SqueezeNet</a> 93 </td> 94 <td>Pixel 3 </td> 95 <td>35.8 ms</td> 96 <td>9.5 ms </td> 97 <td>18.5 ms</td> 98 </tr> 99 <tr> 100 <td>Pixel 4 </td> 101 <td>23.9 ms</td> 102 <td>11.1 ms</td> 103 <td>19.0 ms</td> 104 </tr> 105 <tr> 106 <td rowspan = 2> 107 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_resnet_v2_2018_04_27.tgz">Inception_ResNet_V2</a> 108 </td> 109 <td>Pixel 3 </td> 110 <td>422 ms</td> 111 <td>99.8 ms </td> 112 <td>201 ms</td> 113 </tr> 114 <tr> 115 <td>Pixel 4 </td> 116 <td>272.6 ms</td> 117 <td>87.2 ms</td> 118 <td>171.1 ms</td> 119 </tr> 120 <tr> 121 <td rowspan = 2> 122 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_v4_2018_04_27.tgz">Inception_V4</a> 123 </td> 124 <td>Pixel 3 </td> 125 <td>486 ms</td> 126 <td>93 ms </td> 127 <td>292 ms</td> 128 </tr> 129 <tr> 130 <td>Pixel 4 </td> 131 <td>324.1 ms</td> 132 <td>97.6 ms</td> 133 <td>186.9 ms</td> 134 </tr> 135 136 </table> 137 138# iOS benchmarks 139 140To run iOS benchmarks, the 141[benchmark app](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark/ios) 142was modified to include the appropriate model and `benchmark_params.json` was 143modified to set `num_threads` to 2. For GPU delegate, `"use_gpu" : "1"` and 144`"gpu_wait_type" : "aggressive"` options were also added to 145`benchmark_params.json`. 146 147<table> 148 <thead> 149 <tr> 150 <th>Model Name</th> 151 <th>Device </th> 152 <th>CPU, 2 threads</th> 153 <th>GPU</th> 154 </tr> 155 </thead> 156 <tr> 157 <td> 158 <a href="https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz">Mobilenet_1.0_224(float)</a> 159 </td> 160 <td>iPhone XS </td> 161 <td>14.8 ms</td> 162 <td>3.4 ms</td> 163 </tr> 164 <tr> 165 <td> 166 <a href="https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz)">Mobilenet_1.0_224 (quant)</a> 167 </td> 168 <td>iPhone XS </td> 169 <td>11 ms</td> 170 <td>---</td> 171 </tr> 172 <tr> 173 <td> 174 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/nasnet_mobile_2018_04_27.tgz">NASNet mobile</a> 175 </td> 176 <td>iPhone XS </td> 177 <td>30.4 ms</td> 178 <td>---</td> 179 </tr> 180 <tr> 181 <td> 182 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/squeezenet_2018_04_27.tgz">SqueezeNet</a> 183 </td> 184 <td>iPhone XS </td> 185 <td>21.1 ms</td> 186 <td>15.5 ms</td> 187 </tr> 188 <tr> 189 <td> 190 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_resnet_v2_2018_04_27.tgz">Inception_ResNet_V2</a> 191 </td> 192 <td>iPhone XS </td> 193 <td>261.1 ms</td> 194 <td>45.7 ms</td> 195 </tr> 196 <tr> 197 <td> 198 <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_v4_2018_04_27.tgz">Inception_V4</a> 199 </td> 200 <td>iPhone XS </td> 201 <td>309 ms</td> 202 <td>54.4 ms</td> 203 </tr> 204 </table> 205