README.md
1# Using TensorRT in TensorFlow (TF-TRT)
2
3This module provides necessary bindings and introduces `TRTEngineOp` operator
4that wraps a subgraph in TensorRT. This module is under active development.
5
6## Installing TF-TRT
7
8Currently TensorFlow nightly builds include TF-TRT by default, which means you
9don't need to install TF-TRT separately. You can pull the latest TF containers
10from docker hub or install the latest TF pip package to get access to the latest
11TF-TRT.
12
13If you want to use TF-TRT on NVIDIA Jetson platform, you can find the download
14links for the relevant TensorFlow pip packages here:
15https://docs.nvidia.com/deeplearning/dgx/index.html#installing-frameworks-for-jetson
16
17## Installing TensorRT
18
19In order to make use of TF-TRT, you will need a local installation of TensorRT.
20Installation instructions for compatibility with TensorFlow are provided on the
21[TensorFlow GPU support](https://www.tensorflow.org/install/gpu) guide.
22
23## Examples
24
25You can find example scripts for running inference on deep learning models in
26this repository: https://github.com/tensorflow/tensorrt
27
28We have used these examples to verify the accuracy and performance of TF-TRT.
29For more information see
30[Verified Models](https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#verified-models).
31
32## Documentation
33
34[TF-TRT documentation](https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html)
35gives an overview of the supported functionalities, provides tutorials and
36verified models, explains best practices with troubleshooting guides.
37
38## Tests
39
40TF-TRT includes both Python tests and C++ unit tests. Most of Python tests are
41located in the test directory and they can be executed using `bazel test` or
42directly with the Python command. Most of the C++ unit tests are used to test
43the conversion functions that convert each TF op to a number of TensorRT layers.
44
45## Compilation
46
47In order to compile the module, you need to have a local TensorRT installation
48(libnvinfer.so and respective include files). During the configuration step,
49TensorRT should be enabled and installation path should be set. If installed
50through package managers (deb,rpm), configure script should find the necessary
51components from the system automatically. If installed from tar packages, user
52has to set path to location where the library is installed during configuration.
53
54```shell
55bazel build --config=cuda --config=opt //tensorflow/tools/pip_package:build_pip_package
56bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/
57```
58
59