RESEARCH:EDITING

 

Computational Videography

Videoshop: A New Framework for Video Editing

    Hongcheng Wang, Ning Xu, Ramesh Raskar and Narendra Ahuja

 

 

INTRODUCTION


Computational videography is a new field of producing a novel or better output video streams from sets of input streams using a computer. With easy access to video camcorders, computational
videography becomes more and more important in many applications. However, the capacity to video based rendering has not kept pace with our ability to capture it. I developed a state-of-the-art computational videography system, i.e., Videoshop, which mainly focused on object-level video editing [1] and high dynamic range video generation utilizing a split-aperture camera [2]. The method used in the system preserves important local perceptual cues and temporal consistency of video sequences while avoiding traditional problems such as aliasing, ghosting/haloing and flickering.

 

 

PUBLICATIONS


1. Hongcheng Wang, Ramesh Raskar and Narendra Ahuja, Seamless Video Editing, IEEE, International Conference on Pattern Recognition (ICPR), 2004.

Abstract: This paper presents a new framework for seamless video editing in the gradient domain. The spatio-temporal gradient fields of target videos are modified or mixed to generate a new gradient field, which is usually not integrable. We propose a 3D video integration algorithm, which finds a potential function, whose gradient field is closest to the resulting gradient field in the sense of least squares. The video is reconstructed by solving a 3D Poisson equation. We use a fast and accurate 3D discrete Poisson solver using diagonal multigrids. A set of gradient operators are defined for user interaction. The resulting video has temporal coherency and no artifacts. We evaluate our algorithm using a variety of examples.

Full Text:   PDF (108KB)  

BibTex:

@inproceedings{1021105,
 author = {Hongcheng Wang and Ramesh Raskar and Narendra Ahuja},
 title = {Seamless Video Editing},
 booktitle = {ICPR '04: Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 3},
 year = {2004},
 isbn = {0-7695-2128-2},
 pages = {858--861},
 doi = {http://dx.doi.org/10.1109/ICPR.2004.806},
 publisher = {IEEE Computer Society},
 address = {Washington, DC, USA},
 }

 

2. Hongcheng Wang, Ramesh Raskar and Narendra Ahuja,  High Dynamic Range Video Using Split Aperture Camera, IEEE 6th Workshop on Omnidirectional Vision, Camera Networks and Non-classical Cameras (OMNIVIS, in conjunction with ICCV'05), Beijing, China, Oct. 2005

Abstract: We present a new approach to display High Dynamic Range (HDR) video using gradient based high dynamic range compression. To obtain HDR video, we utilize the split aperture camera. We apply a spatio-temporal gradient based video integration algorithm for fast and accurate integration of the three input HDR videos into a low dynamic range video, which is suitable for display. The spatio-temporal video integration generates videos with temporal coherency and without artifacts. In order to improve the computational speed, we propose using a diagonal multigrid algorithm to solve the Poisson equation. We show experimental results on a variety of dynamic scenes.

Full Text:   PDF (1.7MB)  

BibTex:

@inproceedings{wang05,
 author = {
Hongcheng Wang and Ramesh Raskar and Narendra Ahuja},
 title = {High Dynamic Range Video Using Split Aperture Camera},
 booktitle = {IEEE 6
th Workshop on Omnidirectional Vision, Camera Networks and Non-classical Cameras },
 year = {2005},
 publisher = {IEEE Computer Society},
 address = {Washington, DC, USA},
 }

 

3.  Hongcheng Wang, Ning Xu, Ramesh Raskar and Narendra Ahuja, Videoshop: A New Framework for Video Editing in Spatio-Temporal Gradient Domain, IEEE, Video Proceedings, International Conference on Computer Vision and Pattern Recognition (CVPR), 2005

Full Text:   PDF (50KB)  

BibTex:

@inproceedings{1069243,
 author = {Hongcheng Wang and Ning Xu and Ramesh Raskar and Narendra Ahuja},
 title = {Videoshop: A New Framework for Spatio-Temporal Video Editing in Gradient Domain},
 booktitle = {CVPR '05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2},
 year = {2005},
 isbn = {0-7695-2372-2},
 pages = {1201},
 doi = {http://dx.doi.org/10.1109/CVPR.2005.369},
 publisher = {IEEE Computer Society},
 address = {Washington, DC, USA},
 }

 

DEMO


  • DEMO VIDEO: Demo.avi (DivX 5.01 compressed, ~70MB)

  • APPLICATIONS:

  1. Digital Video Face Replacement and Painting: Digital face replacement and painting involves replacing the face of a person in a target image using the face of another person in a source video. The shape, expression and motion of the face in the resulting video will be the same as in the source video, but the color and appearance of the face is the same as in the target face.

 

  1. Graph-Cut Based Video Compositing:
    To obtain the regions of interest for video compositing, we use a 3D graph cut algorithm to find the minimum cut between two sequences, and then use gradients of one video sequence on one side of the cut while using gradients of the other video sequence on the other side.

 

  1. Moving Object Insertion:
    Object insertion addresses the problem of seamlessly importing a moving object from a source video sequence to a new background with little change in the color of the original object.

  1. Compositing Two Video Sequences:
    Compositing two video sequences using user given masks.
 
  1. 3D Shape Editing:
    We illustrate a different method within our framework in 3D surface editing and compositing since the depth map can be treated like an intensity image. The left two columns are the original 3D model, and the right two columns are synthesized models.
 
  1. High Dynamic Range Video Compression: The right figure shows the results. The right bottom image is the output image, and the other three images are input.
     
 

     

ACKNOWLEDGEMENTS


  1. The Computational Science and Engineering (CSE) Fellowship support from UIUC during 2004-2005 is greatly acknowledged.
  2. The preliminary results is due to the support of summer internship from Mitsubishi Electric Research Laboratories (MERL).
  3. The demo video was first shown in Beckman Institute Open House 2005 with a topic of "Verify Video Evidence - Can it be trusted?".

 


Updated: Jan.1, 2006