End-to-end Active Object Tracking via Reinforcement Learning

Wenhan Luo*¹ Peng Sun*¹ Fangwei Zhong² Wei Liu¹ Tong Zhang¹ Yizhou Wang²

¹Tencent AI Lab ²Peking University

*Indicate equal contribtuions

Abstract

We study active object tracking, where a tracker takes as input the visual observation (i.e., frame sequence) and produces the camera control signal (e.g., move forward, turn left, etc.). Conventional methods tackle the tracking and the camera control separately, which is challenging to tune jointly. It also incurs many human efforts for labeling and many expensive trial-and-errors in real-world. To address these issues, we propose, in this paper, an end-to-end solution via deep reinforcement learning, where a ConvNet-LSTM function approximator is adopted for the direct frame-to-action prediction. We further propose an environment augmentation technique and a customized reward function, which are crucial for a successful training. The tracker trained in simulators (ViZDoom, Unreal Engine) shows good generalization in the case of unseen object moving path, unseen object appearance, unseen background and distracting object. It can restore tracking when occasionally losing the target. With the experiments over the VOT dataset, we also find that the tracking ability, obtained solely from simulators, can potentially transfer to real-world scenarios.

Code

We use vanilla A3C code from OpenAI to train the tracker and interface code to interact with virtual environment including ViZDoom and Unreal. For ViZDoom, we develop gym_tvizdoom, and for Unreal we use gym_unrealcv.

The link to gym_tvizdoom is here.

The link to gym_unrealcv is here.

The training code is here.

Virtual Environment

We employ virtual environments including ViZDoom and Unreal Engine to train our agent as an active tracker.

Paper

The download link is here.

Citation

@inproceedings{luo2018end,

title={End-to-end Active Object Tracking via Reinforcement Learning},

author={Luo, Wenhan and Sun, Peng and Zhong, Fangwei and Liu, Wei and Zhang, Tong and Wang, Yizhou},

booktitle={International Conference on Machine Learning},

year={2018}

}