Image Interpolation [ CVPR'99, CVPR'00, TPAMI'02, TCSVT'03 ]

Email: Maxime { dot } Lhuillier { at } free { dot } fr
Address: Institut Pascal, UMR 6602 CNRS/UBP/IFMA,
Campus Universitaire des Cézeaux,
63171 Aubière Cedex - France.
Tel: +33 (0)4 73 40 75 93
Fax: +33 (0)4 73 40 72 62

Last modification on June 14th 2007.
Go back to my HOME PAGE.

About it

This work is a part of my PhD and includes the topics "Image Matching" and "Image-Based Rendering". The difficulty of the problem is to obtain a fully automatic method in a general context, especially by handling occlusions and low textured areas.
First, two images of a scene are taken at two different locations. Second, matching between pixels of the two images is done: apply a standard method to match a sparse set of interest points in images and propagate the matching to a maximum of pixels using matching constraints (quasi-dense propagation). This step may include the fundamental matrix estimation and a second quasi-dense propagation with the epipolar constraint if the scene is rigid (but this is not an obligation). Third, Joint View Triangulation is build: a constrained Delaunay triangulation in each image with many constraints. The constraints include matching relations between vertices of different triangulations, triangulation edges due to luminance edges in images, and special vertices to extrapolate depth in occluded areas. Fourth, the image interpolation results are obtained by warping all image triangles in a back to front-like order and adequate weights.

Examples

For each examples, the intermediates results of the method are given:
- initial and sparse matches of interest points in red (with their numbers)
- the depth map corresponding to the quasi-dense propagation (black: slow displacement in image, gray: large displacement, white: no match)
- Joint View Triangulation. Red edges are matched between images, blues are not. Yellow and Green crosses are virtual vertices (TCSVT'03) and are used to extrapolate depth in occluded areas. Each triangulation vertex (except virtual vertex) is matched with a vertex of the other triangulation.

Other examples (face morphing) are given in my home page.