Real-time foreground segmentation via range and color imaging

This paper describes a real-time method for foreground/background segmentation of a color video sequence based primarily on range data of a time-of-flight sensor. This method uses depth information of a TOF-sensor paired with a high resolution color video camera to efficiently segment foreground from background in a two-step process. First a trimap is produced using only range data: areas are located in each frame that have a high probability of being background or foreground, respectively. Pixels which cannot be definitively classified as foreground or background, typically about 1-2% of the frame, are assigned alpha-matte values using a cross bilateral filtering, applied directly to an estimate of the alpha-matte.