1AKAZE and ORB planar tracking {#tutorial_akaze_tracking} 2============================= 3 4Introduction 5------------ 6 7In this tutorial we will compare *AKAZE* and *ORB* local features using them to find matches between 8video frames and track object movements. 9 10The algorithm is as follows: 11 12- Detect and describe keypoints on the first frame, manually set object boundaries 13- For every next frame: 14 -# Detect and describe keypoints 15 -# Match them using bruteforce matcher 16 -# Estimate homography transformation using RANSAC 17 -# Filter inliers from all the matches 18 -# Apply homography transformation to the bounding box to find the object 19 -# Draw bounding box and inliers, compute inlier ratio as evaluation metric 20 21![](images/frame.png) 22 23Data 24---- 25 26To do the tracking we need a video and object position on the first frame. 27 28You can download our example video and data from 29[here](https://docs.google.com/file/d/0B72G7D4snftJandBb0taLVJHMFk). 30 31To run the code you have to specify input and output video path and object bounding box. 32@code{.none} 33./planar_tracking blais.mp4 result.avi blais_bb.xml.gz 34@endcode 35 36Source Code 37----------- 38 39@include cpp/tutorial_code/features2D/AKAZE_tracking/planar_tracking.cpp 40 41Explanation 42----------- 43 44### Tracker class 45 46This class implements algorithm described abobve using given feature detector and descriptor 47matcher. 48 49- **Setting up the first frame** 50 @code{.cpp} 51 void Tracker::setFirstFrame(const Mat frame, vector<Point2f> bb, string title, Stats& stats) 52 { 53 first_frame = frame.clone(); 54 (*detector)(first_frame, noArray(), first_kp, first_desc); 55 stats.keypoints = (int)first_kp.size(); 56 drawBoundingBox(first_frame, bb); 57 putText(first_frame, title, Point(0, 60), FONT_HERSHEY_PLAIN, 5, Scalar::all(0), 4); 58 object_bb = bb; 59 } 60 @endcode 61 We compute and store keypoints and descriptors from the first frame and prepare it for the 62 output. 63 64 We need to save number of detected keypoints to make sure both detectors locate roughly the same 65 number of those. 66 67- **Processing frames** 68 69 -# Locate keypoints and compute descriptors 70 @code{.cpp} 71 (*detector)(frame, noArray(), kp, desc); 72 @endcode 73 74 To find matches between frames we have to locate the keypoints first. 75 76 In this tutorial detectors are set up to find about 1000 keypoints on each frame. 77 78 -# Use 2-nn matcher to find correspondences 79 @code{.cpp} 80 matcher->knnMatch(first_desc, desc, matches, 2); 81 for(unsigned i = 0; i < matches.size(); i++) { 82 if(matches[i][0].distance < nn_match_ratio * matches[i][1].distance) { 83 matched1.push_back(first_kp[matches[i][0].queryIdx]); 84 matched2.push_back( kp[matches[i][0].trainIdx]); 85 } 86 } 87 @endcode 88 If the closest match is *nn_match_ratio* closer than the second closest one, then it's a 89 match. 90 91 -# Use *RANSAC* to estimate homography transformation 92 @code{.cpp} 93 homography = findHomography(Points(matched1), Points(matched2), 94 RANSAC, ransac_thresh, inlier_mask); 95 @endcode 96 If there are at least 4 matches we can use random sample consensus to estimate image 97 transformation. 98 99 -# Save the inliers 100 @code{.cpp} 101 for(unsigned i = 0; i < matched1.size(); i++) { 102 if(inlier_mask.at<uchar>(i)) { 103 int new_i = static_cast<int>(inliers1.size()); 104 inliers1.push_back(matched1[i]); 105 inliers2.push_back(matched2[i]); 106 inlier_matches.push_back(DMatch(new_i, new_i, 0)); 107 } 108 } 109 @endcode 110 Since *findHomography* computes the inliers we only have to save the chosen points and 111 matches. 112 113 -# Project object bounding box 114 @code{.cpp} 115 perspectiveTransform(object_bb, new_bb, homography); 116 @endcode 117 118 If there is a reasonable number of inliers we can use estimated transformation to locate the 119 object. 120 121Results 122------- 123 124You can watch the resulting [video on youtube](http://www.youtube.com/watch?v=LWY-w8AGGhE). 125 126*AKAZE* statistics: 127@code{.none} 128Matches 626 129Inliers 410 130Inlier ratio 0.58 131Keypoints 1117 132@endcode 133 134*ORB* statistics: 135@code{.none} 136Matches 504 137Inliers 319 138Inlier ratio 0.56 139Keypoints 1112 140@endcode 141