An online approach for multi-sprite generation based on camera parameters estimation

We propose in this work to synthesize online the visual content of a complex video into multiple sprites, while detecting on-the-fly each sprite's limits. For each received frame, physically meaningful camera rotation angles and focal lengths are firstly estimated from frame-to-sprite homography. These physical parameters are then used by a thresholding-based assignment stage to confirm whether the frame can be added without distortions on one of the sprites that have been built up to that instant. In this case, the selected sprite will be updated online accordingly by applying direct frame-to-sprite warping and advanced median blending of only the newly discovered background content. Else, a new sprite must be defined while selecting the current frame as a reference frame. Extensive experiments accomplished on challenging videos demonstrate that the proposed approach can process online a long video while preventing accumulation of alignment errors. © 2011 Springer-Verlag London Limited.