• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1AKAZE and ORB planar tracking {#tutorial_akaze_tracking}
2=============================
3
4Introduction
5------------
6
7In this tutorial we will compare *AKAZE* and *ORB* local features using them to find matches between
8video frames and track object movements.
9
10The algorithm is as follows:
11
12-   Detect and describe keypoints on the first frame, manually set object boundaries
13-   For every next frame:
14    -#  Detect and describe keypoints
15    -#  Match them using bruteforce matcher
16    -#  Estimate homography transformation using RANSAC
17    -#  Filter inliers from all the matches
18    -#  Apply homography transformation to the bounding box to find the object
19    -#  Draw bounding box and inliers, compute inlier ratio as evaluation metric
20
21![](images/frame.png)
22
23Data
24----
25
26To do the tracking we need a video and object position on the first frame.
27
28You can download our example video and data from
29[here](https://docs.google.com/file/d/0B72G7D4snftJandBb0taLVJHMFk).
30
31To run the code you have to specify input and output video path and object bounding box.
32@code{.none}
33./planar_tracking blais.mp4 result.avi blais_bb.xml.gz
34@endcode
35
36Source Code
37-----------
38
39@include cpp/tutorial_code/features2D/AKAZE_tracking/planar_tracking.cpp
40
41Explanation
42-----------
43
44### Tracker class
45
46This class implements algorithm described abobve using given feature detector and descriptor
47matcher.
48
49-   **Setting up the first frame**
50    @code{.cpp}
51    void Tracker::setFirstFrame(const Mat frame, vector<Point2f> bb, string title, Stats& stats)
52    {
53        first_frame = frame.clone();
54        (*detector)(first_frame, noArray(), first_kp, first_desc);
55        stats.keypoints = (int)first_kp.size();
56        drawBoundingBox(first_frame, bb);
57        putText(first_frame, title, Point(0, 60), FONT_HERSHEY_PLAIN, 5, Scalar::all(0), 4);
58        object_bb = bb;
59    }
60    @endcode
61    We compute and store keypoints and descriptors from the first frame and prepare it for the
62    output.
63
64    We need to save number of detected keypoints to make sure both detectors locate roughly the same
65    number of those.
66
67-   **Processing frames**
68
69    -#  Locate keypoints and compute descriptors
70        @code{.cpp}
71        (*detector)(frame, noArray(), kp, desc);
72        @endcode
73
74        To find matches between frames we have to locate the keypoints first.
75
76        In this tutorial detectors are set up to find about 1000 keypoints on each frame.
77
78    -#  Use 2-nn matcher to find correspondences
79        @code{.cpp}
80        matcher->knnMatch(first_desc, desc, matches, 2);
81        for(unsigned i = 0; i < matches.size(); i++) {
82            if(matches[i][0].distance < nn_match_ratio * matches[i][1].distance) {
83                matched1.push_back(first_kp[matches[i][0].queryIdx]);
84                matched2.push_back(      kp[matches[i][0].trainIdx]);
85            }
86        }
87        @endcode
88        If the closest match is *nn_match_ratio* closer than the second closest one, then it's a
89        match.
90
91    -#  Use *RANSAC* to estimate homography transformation
92        @code{.cpp}
93        homography = findHomography(Points(matched1), Points(matched2),
94                                    RANSAC, ransac_thresh, inlier_mask);
95        @endcode
96        If there are at least 4 matches we can use random sample consensus to estimate image
97        transformation.
98
99    -#  Save the inliers
100        @code{.cpp}
101        for(unsigned i = 0; i < matched1.size(); i++) {
102            if(inlier_mask.at<uchar>(i)) {
103                int new_i = static_cast<int>(inliers1.size());
104                inliers1.push_back(matched1[i]);
105                inliers2.push_back(matched2[i]);
106                inlier_matches.push_back(DMatch(new_i, new_i, 0));
107            }
108        }
109        @endcode
110        Since *findHomography* computes the inliers we only have to save the chosen points and
111        matches.
112
113    -#  Project object bounding box
114        @code{.cpp}
115        perspectiveTransform(object_bb, new_bb, homography);
116        @endcode
117
118        If there is a reasonable number of inliers we can use estimated transformation to locate the
119        object.
120
121Results
122-------
123
124You can watch the resulting [video on youtube](http://www.youtube.com/watch?v=LWY-w8AGGhE).
125
126*AKAZE* statistics:
127@code{.none}
128Matches      626
129Inliers      410
130Inlier ratio 0.58
131Keypoints    1117
132@endcode
133
134*ORB* statistics:
135@code{.none}
136Matches      504
137Inliers      319
138Inlier ratio 0.56
139Keypoints    1112
140@endcode
141