1# TensorFlow Android Camera Demo 2 3DEPRECATED: These examples are deprecated. 4 5This folder contains an example application utilizing TensorFlow for Android 6devices. 7 8## Description 9 10The demos in this folder are designed to give straightforward samples of using 11TensorFlow in mobile applications. 12 13Inference is done using the [TensorFlow Android Inference 14Interface](../../tools/android/inference_interface), which may be built 15separately if you want a standalone library to drop into your existing 16application. Object tracking and efficient YUV -> RGB conversion are handled by 17`libtensorflow_demo.so`. 18 19A device running Android 5.0 (API 21) or higher is required to run the demo due 20to the use of the camera2 API, although the native libraries themselves can run 21on API >= 14 devices. 22 23## Current samples: 24 251. [TF Classify](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/android/test/src/org/tensorflow/demo/ClassifierActivity.java): 26 Uses the [Google Inception](https://arxiv.org/abs/1409.4842) 27 model to classify camera frames in real-time, displaying the top results 28 in an overlay on the camera image. 292. [TF Detect](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/android/test/src/org/tensorflow/demo/DetectorActivity.java): 30 Demonstrates an SSD-Mobilenet model trained using the 31 [Tensorflow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection/) 32 introduced in [Speed/accuracy trade-offs for modern convolutional object detectors](https://arxiv.org/abs/1611.10012) to 33 localize and track objects (from 80 categories) in the camera preview 34 in real-time. 353. [TF Stylize](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/android/test/src/org/tensorflow/demo/StylizeActivity.java): 36 Uses a model based on [A Learned Representation For Artistic 37 Style](https://arxiv.org/abs/1610.07629) to restyle the camera preview 38 image to that of a number of different artists. 394. [TF 40 Speech](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/android/test/src/org/tensorflow/demo/SpeechActivity.java): 41 Runs a simple speech recognition model built by the [audio training 42 tutorial](https://www.tensorflow.org/versions/master/tutorials/audio_recognition). Listens 43 for a small set of words, and highlights them in the UI when they are 44 recognized. 45 46<img src="sample_images/classify1.jpg" width="30%"><img src="sample_images/stylize1.jpg" width="30%"><img src="sample_images/detect1.jpg" width="30%"> 47 48## Prebuilt Components: 49 50The fastest path to trying the demo is to download the [prebuilt demo APK](https://storage.googleapis.com/download.tensorflow.org/deps/tflite/TfLiteCameraDemo.apk). 51 52Also available are precompiled native libraries, and a jcenter package that you 53may simply drop into your own applications. See 54[tensorflow/tools/android/inference_interface/README.md](../../tools/android/inference_interface/README.md) 55for more details. 56 57## Running the Demo 58 59Once the app is installed it can be started via the "TF Classify", "TF Detect", 60"TF Stylize", and "TF Speech" icons, which have the orange TensorFlow logo as 61their icon. 62 63While running the activities, pressing the volume keys on your device will 64toggle debug visualizations on/off, rendering additional info to the screen that 65may be useful for development purposes. 66 67## Building in Android Studio using the TensorFlow AAR from JCenter 68 69The simplest way to compile the demo app yourself, and try out changes to the 70project code is to use AndroidStudio. Simply set this `android` directory as the 71project root. 72 73Then edit the `build.gradle` file and change the value of `nativeBuildSystem` to 74`'none'` so that the project is built in the simplest way possible: 75 76```None 77def nativeBuildSystem = 'none' 78``` 79 80While this project includes full build integration for TensorFlow, this setting 81disables it, and uses the TensorFlow Inference Interface package from JCenter. 82 83Note: Currently, in this build mode, YUV -> RGB is done using a less efficient 84Java implementation, and object tracking is not available in the "TF Detect" 85activity. 86 87For any project that does not include custom low level TensorFlow code, this is 88likely sufficient. 89 90For details on how to include this JCenter package in your own project see 91[tensorflow/tools/android/inference_interface/README.md](../../tools/android/inference_interface/README.md) 92 93## Building the Demo with TensorFlow from Source 94 95Pick your preferred approach below. At the moment, we have full support for 96Bazel, and partial support for gradle and Android Studio. 97 98As a first step for all build types, clone the TensorFlow repo with: 99 100``` 101git clone --recurse-submodules https://github.com/tensorflow/tensorflow.git 102``` 103 104Note that `--recurse-submodules` is necessary to prevent some issues with 105protobuf compilation. 106 107### Bazel 108 109NOTE: Bazel does not currently support building for Android on Windows. In the 110meantime we suggest that Windows users download the 111[prebuilt demo APK](https://storage.googleapis.com/download.tensorflow.org/deps/tflite/TfLiteCameraDemo.apk) 112instead. 113 114##### Install Bazel and Android Prerequisites 115 116Bazel is the primary build system for TensorFlow. To build with Bazel, it and 117the Android NDK and SDK must be installed on your system. 118 1191. Install the latest version of Bazel as per the instructions [on the Bazel 120 website](https://bazel.build/versions/master/docs/install.html). 1212. The Android NDK is required to build the native (C/C++) TensorFlow code. The 122 current recommended version is 14b, which may be found 123 [here](https://developer.android.com/ndk/downloads/older_releases.html#ndk-14b-downloads). 1243. The Android SDK and build tools may be obtained 125 [here](https://developer.android.com/tools/revisions/build-tools.html), or 126 alternatively as part of [Android 127 Studio](https://developer.android.com/studio/index.html). Build tools API >= 128 23 is required to build the TF Android demo (though it will run on API >= 21 129 devices). 130 131##### Edit WORKSPACE 132 133NOTE: As long as you have the SDK and NDK installed, the `./configure` script 134will create these rules for you. Answer "Yes" when the script asks to 135automatically configure the `./WORKSPACE`. 136 137The Android entries in 138[`<workspace_root>/WORKSPACE`](../../../WORKSPACE#L19-L36) must be uncommented 139with the paths filled in appropriately depending on where you installed the NDK 140and SDK. Otherwise an error such as: "The external label 141'//external:android/sdk' is not bound to anything" will be reported. 142 143Also edit the API levels for the SDK in WORKSPACE to the highest level you have 144installed in your SDK. This must be >= 23 (this is completely independent of the 145API level of the demo, which is defined in AndroidManifest.xml). The NDK API 146level may remain at 14. 147 148##### Install Model Files (optional) 149 150The TensorFlow `GraphDef`s that contain the model definitions and weights are 151not packaged in the repo because of their size. They are downloaded 152automatically and packaged with the APK by Bazel via a new_http_archive defined 153in `WORKSPACE` during the build process, and by Gradle via 154download-models.gradle. 155 156**Optional**: If you wish to place the models in your assets manually, remove 157all of the `model_files` entries from the `assets` list in `tensorflow_demo` 158found in the [`BUILD`](BUILD#L92) file. Then download and extract the archives 159yourself to the `assets` directory in the source tree: 160 161```bash 162BASE_URL=https://storage.googleapis.com/download.tensorflow.org/models 163for MODEL_ZIP in inception5h.zip ssd_mobilenet_v1_android_export.zip stylize_v1.zip 164do 165 curl -L ${BASE_URL}/${MODEL_ZIP} -o /tmp/${MODEL_ZIP} 166 unzip /tmp/${MODEL_ZIP} -d tensorflow/tools/android/test/assets/ 167done 168``` 169 170This will extract the models and their associated metadata files to the local 171assets/ directory. 172 173If you are using Gradle, make sure to remove download-models.gradle reference 174from build.gradle after your manually download models; otherwise gradle might 175download models again and overwrite your models. 176 177##### Build 178 179After editing your WORKSPACE file to update the SDK/NDK configuration, you may 180build the APK. Run this from your workspace root: 181 182```bash 183bazel build --cxxopt='--std=c++11' -c opt //tensorflow/tools/android/test:tensorflow_demo 184``` 185 186##### Install 187 188Make sure that adb debugging is enabled on your Android 5.0 (API 21) or later 189device, then after building use the following command from your workspace root 190to install the APK: 191 192```bash 193adb install -r bazel-bin/tensorflow/tools/android/test/tensorflow_demo.apk 194``` 195 196### Android Studio with Bazel 197 198Android Studio may be used to build the demo in conjunction with Bazel. First, 199make sure that you can build with Bazel following the above directions. Then, 200look at [build.gradle](build.gradle) and make sure that the path to Bazel 201matches that of your system. 202 203At this point you can add the tensorflow/tools/android/test directory as a new 204Android Studio project. Click through installing all the Gradle extensions it 205requests, and you should be able to have Android Studio build the demo like any 206other application (it will call out to Bazel to build the native code with the 207NDK). 208 209
