1# Pose estimation 2 3<img src="../images/pose.png" class="attempt-right" /> 4 5## Get started 6 7_PoseNet_ is a vision model that can be used to estimate the pose of a person in 8an image or video by estimating where key body joints are. 9 10<a class="button button-primary" href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/gpu/multi_person_mobilenet_v1_075_float.tflite">Download 11starter model</a> 12 13Android and iOS end-to-end tutorials are coming soon. In the meantime, if you 14want to experiment this on a web browser, check out the 15<a href="https://github.com/tensorflow/tfjs-models/tree/master/posenet">TensorFlow.js 16GitHub repository</a>. 17 18## How it works 19 20Pose estimation refers to computer vision techniques that detect human figures 21in images and videos, so that one could determine, for example, where someone’s 22elbow shows up in an image. 23 24To be clear, this technology is not recognizing who is in an image. The 25algorithm is simply estimating where key body joints are. 26 27The key points detected are indexed by "Part ID", with a confidence score 28between 0.0 and 1.0, 1.0 being the highest. 29 30<table style="width: 30%;"> 31 <thead> 32 <tr> 33 <th>Id</th> 34 <th>Part</th> 35 </tr> 36 </thead> 37 <tbody> 38 <tr> 39 <td>0</td> 40 <td>nose</td> 41 </tr> 42 <tr> 43 <td>1</td> 44 <td>leftEye</td> 45 </tr> 46 <tr> 47 <td>2</td> 48 <td>rightEye</td> 49 </tr> 50 <tr> 51 <td>3</td> 52 <td>leftEar</td> 53 </tr> 54 <tr> 55 <td>4</td> 56 <td>rightEar</td> 57 </tr> 58 <tr> 59 <td>5</td> 60 <td>leftShoulder</td> 61 </tr> 62 <tr> 63 <td>6</td> 64 <td>rightShoulder</td> 65 </tr> 66 <tr> 67 <td>7</td> 68 <td>leftElbow</td> 69 </tr> 70 <tr> 71 <td>8</td> 72 <td>rightElbow</td> 73 </tr> 74 <tr> 75 <td>9</td> 76 <td>leftWrist</td> 77 </tr> 78 <tr> 79 <td>10</td> 80 <td>rightWrist</td> 81 </tr> 82 <tr> 83 <td>11</td> 84 <td>leftHip</td> 85 </tr> 86 <tr> 87 <td>12</td> 88 <td>rightHip</td> 89 </tr> 90 <tr> 91 <td>13</td> 92 <td>leftKnee</td> 93 </tr> 94 <tr> 95 <td>14</td> 96 <td>rightKnee</td> 97 </tr> 98 <tr> 99 <td>15</td> 100 <td>leftAnkle</td> 101 </tr> 102 <tr> 103 <td>16</td> 104 <td>rightAnkle</td> 105 </tr> 106 </tbody> 107</table> 108 109## Example output 110 111<img alt="Animation showing pose estimation" src="https://www.tensorflow.org/images/models/pose_estimation.gif" /> 112 113## How it performs 114 115Performance varies based on your device and output stride (heatmaps and offset 116vectors). The PoseNet model is image size invariant, which means it can predict 117pose positions in the same scale as the original image regardless of whether the 118image is downscaled. This means PoseNet can be configured to have a higher 119accuracy at the expense of performance. 120 121The output stride determines how much we’re scaling down the output relative to 122the input image size. It affects the size of the layers and the model outputs. 123The higher the output stride, the smaller the resolution of layers in the 124network and the outputs, and correspondingly their accuracy. In this 125implementation, the output stride can have values of 8, 16, or 32. In other 126words, an output stride of 32 will result in the fastest performance but lowest 127accuracy, while 8 will result in the highest accuracy but slowest performance. 128We recommend starting with 16. 129 130The following image shows how the output stride determines how much we’re 131scaling down the output relative to the input image size. A higher output stride 132is faster but results in lower accuracy. 133 134<img alt="Output stride and heatmap resolution" src="../images/output_stride.png" > 135 136## Read more about pose estimation 137 138<ul> 139 <li><a href="https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5">Blog post: Real-time Human Pose Estimation in the Browser with TensorFlow.js</a></li> 140 <li><a href="https://github.com/tensorflow/tfjs-models/tree/master/posenet">TF.js GitHub: Pose Detection in the Browser: PoseNet Model</a></li> 141</ul> 142 143### Use cases 144 145<ul> 146 <li><a href="https://vimeo.com/128375543">‘PomPom Mirror’</a></li> 147 <li><a href="https://youtu.be/I5__9hq-yas">Amazing Art Installation Turns You Into A Bird | Chris Milk "The Treachery of Sanctuary"</a></li> 148 <li><a href="https://vimeo.com/34824490">Puppet Parade - Interactive Kinect Puppets</a></li> 149 <li><a href="https://vimeo.com/2892576">Messa di Voce (Performance), Excerpts</a></li> 150 <li><a href="https://www.instagram.com/p/BbkKLiegrTR/">Augmented reality</a></li> 151 <li><a href="https://www.instagram.com/p/Bg1EgOihgyh/">Interactive animation</a></li> 152 <li><a href="https://www.runnersneed.com/expert-advice/gear-guides/gait-analysis.html">Gait analysis</a></li> 153</ul> 154
