Real-time Discriminative Background Subtraction

Brief Description

We examine the problem of segmenting foreground objects in live video when background scene textures change over time.
We formulate background subtraction as minimizing a penalized instantaneous risk functional--- yielding a local on-line discriminative algorithm that can quickly adapt to temporal changes. We analyze the algorithm's convergence, discuss its robustness to non-stationarity, and provide an efficient non-linear extension via sparse kernels. To accommodate interactions among neighboring pixels, a global algorithm is then derived that explicitly distinguishes objects versus background via maximum a posteriori inference in a Markov random field (implemented via graph cuts). By exploiting the parallel nature of the proposed algorithms, we develop an implementation that can run efficiently on parallel graphics processors, suitable for real-time video analysis >=60 fps on a mid-range GPU.

From left to right:: three typical scenarios that are characterized by these non-stationary background distributions: drifting, jumping and multi-modal switching.

We explicitly connect the problem of background subtraction to work in on-line learning and novelty detection, which possess a rich literature and well-studied theoretical principles. In addition, we present a series discriminative on-line learning algorithms based on kernels (called SILK and SILK-GC) that provide a principled method for modeling the spatial-temporal
characteristics of the background subtraction problem. our algorithm is designed to work with GPUs, yielding a processing speed of 60-120 frames per second (FPS).