Multi-object tracking-by-detection using multi-camera systems

Automatic tracking of objects in sequences of complex scenes is one of most important and challenging topics in modern computer vision research. It is applied to numerous application scenarios, for instance visual surveillance, medical image analysis, and microscopy videos in biological research. Most of the traditional tracking approaches are only single-view-based and hence are facing problems like 2D occlusions and object ambiguities. Expanding the observed scene using several cameras can improve the robustness and reliability of the system as well as provide 3D world coordinates. This thesis develops and analyzes approaches for automatic multi-object tracking from detections using multi-camera systems. First, two solutions for online tracking based on particle filters are proposed. Second, we present a robust offline tracking framework to iteratively optimize object trajectories over the entire sequence in 3D space. The whole process is formulated into two sequential maximum-a-posteriori problems solved by shortest path searching in two graphs. In the first stage, reconstructed detections are mapped into a directed acyclic graph to gain multiple short tracklets. In the second stage, full tracks are generated from a graph consisting of tracklets as nodes. Features extracted from 3D detections and tracklets are used for calculating edge weights in the corresponding graphs. This results in an accurate linking of reconstructions and tracklets, while providing a flexible framework which can be easily adapted to various application scenarios based on the edge weighting functions. The proposed approaches are validated both qualitatively and quantitatively on a wide range of real-world datasets. The accuracy and suitability of the approaches are demonstrated by the superior performance compared with other algorithms. The tracked 3D trajectories of the markers in the X-ray recordings of beating sheep hearts are already successfully used for medical analysis. Furthermore, we show the enhancement of cell phenotype clustering performance in microscopy videos by using tracked cell trajectories, which reveals the robustness and practicability of our approach as well.