1{ 2 "cells": [ 3 { 4 "cell_type": "markdown", 5 "id": "g_nWetWWd_ns", 6 "metadata": { 7 "id": "g_nWetWWd_ns" 8 }, 9 "source": [ 10 "##### Copyright 2021 The TensorFlow Authors." 11 ] 12 }, 13 { 14 "cell_type": "code", 15 "execution_count": null, 16 "id": "2pHVBk_seED1", 17 "metadata": { 18 "cellView": "form", 19 "id": "2pHVBk_seED1" 20 }, 21 "outputs": [], 22 "source": [ 23 "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 24 "# you may not use this file except in compliance with the License.\n", 25 "# You may obtain a copy of the License at\n", 26 "#\n", 27 "# https://www.apache.org/licenses/LICENSE-2.0\n", 28 "#\n", 29 "# Unless required by applicable law or agreed to in writing, software\n", 30 "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 31 "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 32 "# See the License for the specific language governing permissions and\n", 33 "# limitations under the License." 34 ] 35 }, 36 { 37 "cell_type": "markdown", 38 "id": "M7vSdG6sAIQn", 39 "metadata": { 40 "id": "M7vSdG6sAIQn" 41 }, 42 "source": [ 43 "# Signatures in TensorFlow Lite" 44 ] 45 }, 46 { 47 "cell_type": "markdown", 48 "id": "fwc5GKHBASdc", 49 "metadata": { 50 "id": "fwc5GKHBASdc" 51 }, 52 "source": [ 53 "\u003ctable class=\"tfo-notebook-buttons\" align=\"left\"\u003e\n", 54 " \u003ctd\u003e\n", 55 " \u003ca target=\"_blank\" href=\"https://www.tensorflow.org/lite/guide/signatures\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" /\u003eView on TensorFlow.org\u003c/a\u003e\n", 56 " \u003c/td\u003e\n", 57 " \u003ctd\u003e\n", 58 " \u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/guide/signatures.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /\u003eRun in Google Colab\u003c/a\u003e\n", 59 " \u003c/td\u003e\n", 60 " \u003ctd\u003e\n", 61 " \u003ca target=\"_blank\" href=\"https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/guide/signatures.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /\u003eView source on GitHub\u003c/a\u003e\n", 62 " \u003c/td\u003e\n", 63 " \u003ctd\u003e\n", 64 " \u003ca href=\"https://storage.googleapis.com/tensorflow_docs/tensorflow/tensorflow/lite/g3doc/guide/signatures.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/download_logo_32px.png\" /\u003eDownload notebook\u003c/a\u003e\n", 65 " \u003c/td\u003e\n", 66 "\u003c/table\u003e" 67 ] 68 }, 69 { 70 "cell_type": "markdown", 71 "id": "9ee074e4", 72 "metadata": { 73 "id": "9ee074e4" 74 }, 75 "source": [ 76 "TensorFlow Lite supports converting TensorFlow model's input/output\n", 77 "specifications to TensorFlow Lite models. The input/output specifications are\n", 78 "called \"signatures\". Signatures can be specified when building a SavedModel or\n", 79 "creating concrete functions.\n", 80 "\n", 81 "Signatures in TensorFlow Lite provide the following features:\n", 82 "\n", 83 "* They specify inputs and outputs of the converted TensorFlow Lite model by\n", 84 " respecting the TensorFlow model's signatures.\n", 85 "* Allow a single TensorFlow Lite model to support multiple entry points.\n", 86 "\n", 87 "The signature is composed of three pieces:\n", 88 "\n", 89 "* Inputs: Map for inputs from input name in the signature to an input tensor.\n", 90 "* Outputs: Map for output mapping from output name in signature to an output\n", 91 " tensor.\n", 92 "* Signature Key: Name that identifies an entry point of the graph.\n", 93 "\n" 94 ] 95 }, 96 { 97 "cell_type": "markdown", 98 "id": "UaWdLA3fQDK2", 99 "metadata": { 100 "id": "UaWdLA3fQDK2" 101 }, 102 "source": [ 103 "## Setup" 104 ] 105 }, 106 { 107 "cell_type": "code", 108 "execution_count": null, 109 "id": "9j4MGqyKQEo4", 110 "metadata": { 111 "id": "9j4MGqyKQEo4" 112 }, 113 "outputs": [], 114 "source": [ 115 "import tensorflow as tf" 116 ] 117 }, 118 { 119 "cell_type": "markdown", 120 "id": "FN2N6hPEP-Ay", 121 "metadata": { 122 "id": "FN2N6hPEP-Ay" 123 }, 124 "source": [ 125 "## Example model\n", 126 "\n", 127 "Let's say we have two tasks, e.g., encoding and decoding, as a TensorFlow model:" 128 ] 129 }, 130 { 131 "cell_type": "code", 132 "execution_count": null, 133 "id": "d8577c80", 134 "metadata": { 135 "id": "d8577c80" 136 }, 137 "outputs": [], 138 "source": [ 139 "class Model(tf.Module):\n", 140 "\n", 141 " @tf.function(input_signature=[tf.TensorSpec(shape=[None], dtype=tf.float32)])\n", 142 " def encode(self, x):\n", 143 " result = tf.strings.as_string(x)\n", 144 " return {\n", 145 " \"encoded_result\": result\n", 146 " }\n", 147 "\n", 148 " @tf.function(input_signature=[tf.TensorSpec(shape=[None], dtype=tf.string)])\n", 149 " def decode(self, x):\n", 150 " result = tf.strings.to_number(x)\n", 151 " return {\n", 152 " \"decoded_result\": result\n", 153 " }" 154 ] 155 }, 156 { 157 "cell_type": "markdown", 158 "id": "9c814c6e", 159 "metadata": { 160 "id": "9c814c6e" 161 }, 162 "source": [ 163 "In the signature wise, the above TensorFlow model can be summarized as follows:\n", 164 "\n", 165 "* Signature\n", 166 "\n", 167 " - Key: encode\n", 168 " - Inputs: {\"x\"}\n", 169 " - Output: {\"encoded_result\"}\n", 170 "\n", 171 "* Signature\n", 172 "\n", 173 " - Key: decode\n", 174 " - Inputs: {\"x\"}\n", 175 " - Output: {\"decoded_result\"}" 176 ] 177 }, 178 { 179 "cell_type": "markdown", 180 "id": "c4099f20", 181 "metadata": { 182 "id": "c4099f20" 183 }, 184 "source": [ 185 "## Convert a model with Signatures\n", 186 "\n", 187 "TensorFlow Lite converter APIs will bring the above signature information into\n", 188 "the converted TensorFlow Lite model.\n", 189 "\n", 190 "This conversion functionality is available on all the converter APIs starting\n", 191 "from TensorFlow version 2.7.0. See example usages.\n" 192 ] 193 }, 194 { 195 "cell_type": "markdown", 196 "id": "Qv0WwFQkQgnO", 197 "metadata": { 198 "id": "Qv0WwFQkQgnO" 199 }, 200 "source": [ 201 "\n", 202 "### From Saved Model" 203 ] 204 }, 205 { 206 "cell_type": "code", 207 "execution_count": null, 208 "id": "96c8fc79", 209 "metadata": { 210 "id": "96c8fc79" 211 }, 212 "outputs": [], 213 "source": [ 214 "model = Model()\n", 215 "\n", 216 "# Save the model\n", 217 "SAVED_MODEL_PATH = 'content/saved_models/coding'\n", 218 "\n", 219 "tf.saved_model.save(\n", 220 " model, SAVED_MODEL_PATH,\n", 221 " signatures={\n", 222 " 'encode': model.encode.get_concrete_function(),\n", 223 " 'decode': model.decode.get_concrete_function()\n", 224 " })\n", 225 "\n", 226 "# Convert the saved model using TFLiteConverter\n", 227 "converter = tf.lite.TFLiteConverter.from_saved_model(SAVED_MODEL_PATH)\n", 228 "converter.target_spec.supported_ops = [\n", 229 " tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.\n", 230 " tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.\n", 231 "]\n", 232 "tflite_model = converter.convert()\n", 233 "\n", 234 "# Print the signatures from the converted model\n", 235 "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n", 236 "signatures = interpreter.get_signature_list()\n", 237 "print(signatures)" 238 ] 239 }, 240 { 241 "cell_type": "markdown", 242 "id": "5baa9f17", 243 "metadata": { 244 "id": "5baa9f17" 245 }, 246 "source": [ 247 "### From Keras Model" 248 ] 249 }, 250 { 251 "cell_type": "code", 252 "execution_count": null, 253 "id": "71f29229", 254 "metadata": { 255 "id": "71f29229" 256 }, 257 "outputs": [], 258 "source": [ 259 "# Generate a Keras model.\n", 260 "keras_model = tf.keras.Sequential(\n", 261 " [\n", 262 " tf.keras.layers.Dense(2, input_dim=4, activation='relu', name='x'),\n", 263 " tf.keras.layers.Dense(1, activation='relu', name='output'),\n", 264 " ]\n", 265 ")\n", 266 "\n", 267 "# Convert the keras model using TFLiteConverter.\n", 268 "# Keras model converter API uses the default signature automatically.\n", 269 "converter = tf.lite.TFLiteConverter.from_keras_model(keras_model)\n", 270 "tflite_model = converter.convert()\n", 271 "\n", 272 "# Print the signatures from the converted model\n", 273 "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n", 274 "\n", 275 "signatures = interpreter.get_signature_list()\n", 276 "print(signatures)" 277 ] 278 }, 279 { 280 "cell_type": "markdown", 281 "id": "e4d30f85", 282 "metadata": { 283 "id": "e4d30f85" 284 }, 285 "source": [ 286 "### From Concrete Functions" 287 ] 288 }, 289 { 290 "cell_type": "code", 291 "execution_count": null, 292 "id": "c9e8a742", 293 "metadata": { 294 "id": "c9e8a742" 295 }, 296 "outputs": [], 297 "source": [ 298 "model = Model()\n", 299 "\n", 300 "# Convert the concrete functions using TFLiteConverter\n", 301 "converter = tf.lite.TFLiteConverter.from_concrete_functions(\n", 302 " [model.encode.get_concrete_function(),\n", 303 " model.decode.get_concrete_function()], model)\n", 304 "converter.target_spec.supported_ops = [\n", 305 " tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.\n", 306 " tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.\n", 307 "]\n", 308 "tflite_model = converter.convert()\n", 309 "\n", 310 "# Print the signatures from the converted model\n", 311 "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n", 312 "signatures = interpreter.get_signature_list()\n", 313 "print(signatures)" 314 ] 315 }, 316 { 317 "cell_type": "markdown", 318 "id": "b5e85934", 319 "metadata": { 320 "id": "b5e85934" 321 }, 322 "source": [ 323 "## Run Signatures\n", 324 "\n", 325 "TensorFlow inference APIs support the signature-based executions:\n", 326 "\n", 327 "* Accessing the input/output tensors through the names of the inputs and\n", 328 " outputs, specified by the signature.\n", 329 "* Running each entry point of the graph separately, identified by the\n", 330 " signature key.\n", 331 "* Support for the SavedModel's initialization procedure.\n", 332 "\n", 333 "Java, C++ and Python language bindings are currently available. See example the\n", 334 "below sections.\n" 335 ] 336 }, 337 { 338 "cell_type": "markdown", 339 "id": "ZRBMFciMQmiB", 340 "metadata": { 341 "id": "ZRBMFciMQmiB" 342 }, 343 "source": [ 344 "\n", 345 "### Java" 346 ] 347 }, 348 { 349 "cell_type": "markdown", 350 "id": "04c5a4fc", 351 "metadata": { 352 "id": "04c5a4fc" 353 }, 354 "source": [ 355 "```\n", 356 "try (Interpreter interpreter = new Interpreter(file_of_tensorflowlite_model)) {\n", 357 " // Run encoding signature.\n", 358 " Map\u003cString, Object\u003e inputs = new HashMap\u003c\u003e();\n", 359 " inputs.put(\"x\", input);\n", 360 " Map\u003cString, Object\u003e outputs = new HashMap\u003c\u003e();\n", 361 " outputs.put(\"encoded_result\", encoded_result);\n", 362 " interpreter.runSignature(inputs, outputs, \"encode\");\n", 363 "\n", 364 " // Run decoding signature.\n", 365 " Map\u003cString, Object\u003e inputs = new HashMap\u003c\u003e();\n", 366 " inputs.put(\"x\", encoded_result);\n", 367 " Map\u003cString, Object\u003e outputs = new HashMap\u003c\u003e();\n", 368 " outputs.put(\"decoded_result\", decoded_result);\n", 369 " interpreter.runSignature(inputs, outputs, \"decode\");\n", 370 "}\n", 371 "```" 372 ] 373 }, 374 { 375 "cell_type": "markdown", 376 "id": "5ba86c64", 377 "metadata": { 378 "id": "5ba86c64" 379 }, 380 "source": [ 381 "### C++" 382 ] 383 }, 384 { 385 "cell_type": "markdown", 386 "id": "397ad6fd", 387 "metadata": { 388 "id": "397ad6fd" 389 }, 390 "source": [ 391 "```\n", 392 "SignatureRunner* encode_runner =\n", 393 " interpreter-\u003eGetSignatureRunner(\"encode\");\n", 394 "encode_runner-\u003eResizeInputTensor(\"x\", {100});\n", 395 "encode_runner-\u003eAllocateTensors();\n", 396 "\n", 397 "TfLiteTensor* input_tensor = encode_runner-\u003einput_tensor(\"x\");\n", 398 "float* input = input_tensor-\u003edata.f;\n", 399 "// Fill `input`.\n", 400 "\n", 401 "encode_runner-\u003eInvoke();\n", 402 "\n", 403 "const TfLiteTensor* output_tensor = encode_runner-\u003eoutput_tensor(\n", 404 " \"encoded_result\");\n", 405 "float* output = output_tensor-\u003edata.f;\n", 406 "// Access `output`.\n", 407 "```" 408 ] 409 }, 410 { 411 "cell_type": "markdown", 412 "id": "0f4c6ad4", 413 "metadata": { 414 "id": "0f4c6ad4" 415 }, 416 "source": [ 417 "### Python" 418 ] 419 }, 420 { 421 "cell_type": "code", 422 "execution_count": null, 423 "id": "ab7b1963", 424 "metadata": { 425 "id": "ab7b1963" 426 }, 427 "outputs": [], 428 "source": [ 429 "# Load the TFLite model in TFLite Interpreter\n", 430 "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n", 431 "\n", 432 "# Print the signatures from the converted model\n", 433 "signatures = interpreter.get_signature_list()\n", 434 "print('Signature:', signatures)\n", 435 "\n", 436 "# encode and decode are callable with input as arguments.\n", 437 "encode = interpreter.get_signature_runner('encode')\n", 438 "decode = interpreter.get_signature_runner('decode')\n", 439 "\n", 440 "# 'encoded' and 'decoded' are dictionaries with all outputs from the inference.\n", 441 "input = tf.constant([1, 2, 3], dtype=tf.float32)\n", 442 "print('Input:', input)\n", 443 "encoded = encode(x=input)\n", 444 "print('Encoded result:', encoded)\n", 445 "decoded = decode(x=encoded['encoded_result'])\n", 446 "print('Decoded result:', decoded)" 447 ] 448 }, 449 { 450 "cell_type": "markdown", 451 "id": "81b42e5b", 452 "metadata": { 453 "id": "81b42e5b" 454 }, 455 "source": [ 456 "## Known limitations\n", 457 "\n", 458 "* As TFLite interpreter does not gurantee thread safety, the signature runners\n", 459 " from the same interpreter won't be executed concurrently.\n", 460 "* Support for C/iOS/Swift is not available yet.\n", 461 "\n" 462 ] 463 }, 464 { 465 "cell_type": "markdown", 466 "id": "3032Iof6QqmJ", 467 "metadata": { 468 "id": "3032Iof6QqmJ" 469 }, 470 "source": [ 471 "## Updates\n", 472 "\n", 473 "* Version 2.7\n", 474 " - The multiple signature feature is implemented.\n", 475 " - All the converter APIs from version two generate signature-enabled\n", 476 " TensorFlow Lite models.\n", 477 "* Version 2.5\n", 478 " - Signature feature is available through the `from_saved_model` converter\n", 479 " API." 480 ] 481 } 482 ], 483 "metadata": { 484 "colab": { 485 "collapsed_sections": [], 486 "name": "Signatures in TensorFlow Lite", 487 "provenance": [], 488 "toc_visible": true 489 }, 490 "id": "a1b42e5b", 491 "kernelspec": { 492 "display_name": "Python 3", 493 "name": "python3" 494 }, 495 "language_info": { 496 "name": "python" 497 } 498 }, 499 "nbformat": 4, 500 "nbformat_minor": 5 501} 502
