3D feature-based tracker for multiple object tracking

This paper presents a 3D feature-based tracker for tracking multiple moving objects using a computer-controlled binocular head. Our tracker operates in two phases: an initialization phase and a tracking phase. In the initialization phase, correspondence between 2D features in the first stereo image pair is determined reliably using the epipolar line constraint and mutually-supported consistency. In the tracking phase, the feedback loop is established by first predicting new 3D feature locations with Kalman filters (KF) and then projecting them onto the 2D images to guide the extraction of 2D features in the new image pair. Here, we propose a RANSAC (RAN-dom SAmple Consensus)-based clustering method for motion segmentation and estimation using the principle of rigid body consensus, which states that all the extracted 3D features on a rigid body should have the same 3D motion. This new method leads to a feature-clustering algorithm which provides a systematic method for managing splitting, merging, appearance and disappearance of multiple moving rigid objects-including articulated objects, such as robot manipulators. Using the motion estimates obtained with the RANSAC-based method as the measurements for the KFs, we are able to use linear KFs for predictive visual tracking instead of the commonly-used extended Kalman filters (EKF). Experiments have shown that our tracking system does give good results and can serve as a robust 3D feature tracker for the active binocular vision system we are developing.