Maxime Lhuillier's home page

Email: Maxime { dot } Lhuillier { at } free { dot } fr
Address: Institut Pascal, UMR 6602 CNRS/UBP/IFMA,
Campus Universitaire des Cézeaux (4 av. Blaise Pascal),
63178 Aubière Cedex (TSA 60026, CS 60026) - France.
Tel: +33 (0)4 73 40 75 93
Fax: +33 (0)4 73 40 72 62


Links

Demos (updated on October 6th 2017)
PhD Students (updated on September 17th 2018)
Publications (updated on September 17th 2018)
Paragliding and paramotor (updated on August 22th 2015)

Welcome !

I'm a researcher at the institut Pascal (chargé de recherche CNRS, HDR) working on Computer Vision. My research topics are (in the chronological order on the whole) More details are given in the publications and demos below. For all topics, the main difficulty is to obtain fully automatic methods involving low cost hardware (1 camera+1 PC) most of the time, ground imagery, and uncontrolled environments (neither beacons/turntable/friendly background nor highly finished light control nor well selected viewpoints using UAV).


Demos


Immersive 3D modeling and 360 video using helmet-held gopro cameras [ 3DV'15, CVIU'13, ICPR'14, CVIU'17, ...]


Here are two 3D models reconstructed from videos which can be viewed interactively using sketchfab. The videos are taken by biking 2.5km in a city, or by walking during 4 minutes in a forest.


The following video shows a surface reconstructed from a 900m long video sequence taken by walking in a town.



Other 360 video examples are here.

Manifold Surface of an Environment from a Sparse Cloud of Reconstructed Image Points [ CVIU'13, ICPR'14 ... ]


The following video shows a surface reconstructed from a 3.1km long video sequence taken by biking in a city using a helmet-held catadioptric camera.


The following video shows a surface reconstructed from a 2.5km long video sequence taken in a city using a car-fixed multi-camera (Ladybug).


The three following surfaces are reconstructed from catadioptric images and can be viewed interactively using Sketchfab.


Surface reconstruction of complete environment using a catadioptric camera [ CVIU'08, MIRAGE'11, 3DIMPVT'12 ]






If you encounter problems to play the movies or the 3D models below, you can use ffplay or vrmlview.

3D reconstruction from a catadioptric camera [ OMNIVIS'05, CVPR'07, CVPR'08, CVIU'08 ]



Multi-view surface reconstructions of objects [ ICCV'03, TPAMI'05 ]



Face morphing [ ICPR'00, TPAMI'02 ]

Thanks to B. Achermann (University of Bern, Switzerland) for providing the input images. These faces are ordered in the alphabetical order and automatically morphed.




Image interpolations [ CVPR'99, CVPR'00, TPAMI'02, TCSVT'03 ]




PhD Students

Thanh-Tin Nguyen did a Ph.D thesis (2014-2017) about the self-calibration of several consumer cameras including synchronization and rolling shutter. Now Thant-Tin is working at Xvisio Technology (firm on VSLAM/AI).

Vadim Litvinov did a Ph.D thesis (2011-2014) about the incremental surface reconstruction using a calibrated multi-camera and sparse structure-from-motion data. Now Vadim is working at Machine Zone (firm on mobile gaming).

Shuda Yu did a Ph.D thesis (2010-2013) about the surface reconstruction of environments using a catadioptric camera and sparse structure-from-motion data. Now Shuda is working at SuperD (firm on glasses-free 3d technologies).

Alexandre Eudes did a Ph.D thesis (2007-2010) about the Odometry for driving help in urban areas. Now Alexandre is working at ONERA.

Etienne Mouragnon did a Ph.D thesis (2004-2007) about the incremental estimation of the geometry of a video. Now, Etienne is working at AIGL Developpement (firm on photogrammetry).

Eric Royer did (a Master and) a Ph.D thesis (2003-2006) about the localization of vehicles in urban environments. Now, Eric is lecturer at the university in Clermont Ferrand.


Complete List of Publications

Journals

CVIU'17, Self-calibration of omnidirectional multi-cameras including synchronization and rolling shutter,PDF (also available at the CVIU web site),
T.T. Nguyen and M. Lhuillier,
Computer Vision and Image Understanding, 162:166-184, 2017.

JMIV'17, Overview of Shelling for 2-Manifold Surface Reconstruction based on 3D Delaunay Triangulation,PDF (also available at the JMIV web site),
M. Lhuillier,
Journal of Mathematical Imaging and Vision, 59(2):318-340, 2017.

JMIV'15, 2-Manifold Tests for 3D Delaunay Triangulation-based Surface Reconstruction,PDF (also available at the JMIV web site),
M. Lhuillier,
Journal of Mathematical Imaging and Vision, 51(1):98-105, 2015.

CVIU'13, Manifold Surface Reconstruction of an Environment from Sparse Structure-from-Motion Data,PDF (also available at the CVIU web site),
M. Lhuillier and S. Yu,
Computer Vision and Image Understanding, 117(11):1628-1644, 2013.

PAMI'12, Incremental Fusion of Structure-from-Motion and GPS using Constrained Bundle Adjustment,PDF (also available at the PAMI web site),
M. Lhuillier,
IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12), 2012.

IJCV'11, A Generic Error Model and its Application to Automatic 3D Modeling of Scenes using a Catadioptric Camera,PDF (also available at the IJCV web site),
M. Lhuillier,
International Journal of Computer Vision, 91(2):175-199.

TOG'09, Image-based Street-side City Modeling, PDF,
J. Xiao, T. Fang, P. Zhao, M. Lhuillier and L. Quan,
ACM Transaction on Graphics, 28(5), 2009 (also in proceedings of SIGGRAPH ASIA'09).

IVC'09, Generic and Real-Time Structure from Motion using Local Bundle Adjustment, PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Image and Vision Computing, 27:1178-1193, 2009.

CVIU'08, Automatic Scene Structure and Camera Motion using a Catadioptric System, PDF,
M. Lhuillier,
Computer Vision and Image Understanding, 109(2):186-203, 2008.

IJCV'07/IJRR'07, Monocular vision for mobile robot localization and autonomous navigation, PDF,
E. Royer, M. Lhuillier, M. Dhome and J.M. Lavest,
International Journal of Computer Vision, 74(3), (special joint issue on vision and robotics, with the International Journal of Robotics Research).

TS'06, Localisation par vision monoculaire pour la navigation autonome, PDF,
E. Royer, M. Lhuillier, M. Dhome and J.M. Lavest,
Traitement du Signal, 23(1), 2006.

TPAMI'05, A Quasi-Dense Approach to Surface Reconstruction from Uncalibrated Images, PDF, (These files include large figures and appendix.)
M. Lhuillier and L. Quan,
IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3):418-433, 2005.

TCSVT'03, Image-Based Rendering by Joint View Triangulation, PDF,
M. Lhuillier and L. Quan,
IEEE Transactions on Circuits and Systems for Video Technology, 13(11):1051-1063, 2003.

TPAMI'02, Match Propagation for Image-Based Modeling and Rendering, PDF,
M. Lhuillier and L. Quan,
IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(8):1140-1146, 2002.

Conferences

3DRW'18, Improving thin structures in surface reconstruction from sparse point cloud, PDF,
M. Lhuillier,
Workshop on 3D reconstruction in the wild, in conjunction with ECCV 2018, Munich, Germany, September 2018. (Also to appear in Lecture Notes in Computer Science.)

RFIAP'18, Ajustement de faisceaux multi-caméra intégrant l'estimation de la synchronisation et du rolling-shutter, PDF
T.T. Nguyen and M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes, image, apprentissage et perception, Marne-la-Vallée, France, June 2018.

BMVC'16, Adding Synchronization and Rolling Shutter in Multi-Camera Bundle Adjustment, PDF (supp. mat.)
T.T. Nguyen and M. Lhuillier,
Proceedings of the 27th British Machine Vision Conference, York, United Kingdom, September 2016.

RFIA'16, Synchronisation et auto-étalonnage de plusieurs caméras fixées sur un casque, PDF
T.T. Nguyen and M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Clermont Ferrand, France, June 2016.

3DV'15, Synchronization and Self-Calibration for Helmet-Held Consumer Cameras, Applications to Immersive 3D Modeling and 360 Video, PDF
M. Lhuillier and T.T. Nguyen,
Proceedings of the International Conference on 3D Vision, Lyon, France, October 2015.

ICPR'14, Incremental Solid Modeling from Sparse Structure-from-Motion Data with Improved Visual Artifacts Removal, PDF
V. Litvinov and M. Lhuillier,
Proceedings of the IAPR International Conference on Pattern Recognition, Stockholm, Sweden, August 2014.

RFIA'14, Estimation incrémentale de surface à partir d'un nuage de points épars reconstruit à partir d'images omnidirectionnelles, PDF
V. Litvinov and M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Rouen, France, July 2014.

BMVC'13, Incremental Solid Modeling from Sparse and Omnidirectional Structure-from-Motion Data, PDF
V. Litvinov and M. Lhuillier,
Proceedings of the 22th British Machine Vision Conference, Bristol, United Kingdom, September 2013.

IC3D'12, 2-Manifold Reconstruction from Sparse Visual Features, PDF,
V. Litvinov, S. Yu and M. Lhuillier,
Proceedings of the International Conference on 3D Imaging, Liège, Belgium, December 2012.

ICPR'12, Genus refinement of a manifold surface reconstructed by sculpting the 3d-Delaunay triangulation of Structure-from-Motion points, PDF,
S. Yu and M. Lhuillier,
Proceedings of the IAPR International Conference on Pattern Recognition, Tsukuba, Japan, November 2012.

3DIMPVT'12, Incremental Reconstruction of Manifold Surface from Sparse Visual Mapping, PDF,
S. Yu and M. Lhuillier,
Proceedings of the second joint 3DIM/3DPVT Conference (3D Imaging, Modeling, Processing, Vizualization and Transmission), Zurich, Switzerland, October 2012.

RFIA'12, Fusion GPS-vision à l'aide d'ajustement de faisceaux contraints, PDF,
M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Lyon, France, January 2012.

RFIA'12, Modèles 3d à partir d'un nuage épars de points et une caméra catadioptrique, PDF,
S. Yu and M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Lyon, France, January 2012.

MIRAGE'11, Surface Reconstruction of Scenes using a Catadioptric Camera, PDF,
S. Yu and M. Lhuillier,
Lecture Notes in Computer Science, Volume 6930, Springer Verlag, 2011 (also in the conference on Computer Vision/Computer Craphics Collaboration Techniques and Applications, October 2011).

CVPR'11, Fusion of GPS and Structure-from-Motion using Constrained Bundle Adjustments, PDF,
M. Lhuillier,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, Colorado, June 2011.

PCV'10, Environment Modeling from Images taken by a Low Cost Camera, PDF,
M. Lhuillier,
International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XXXVIII part 3A, 2010 (also in proceedings of PCV'10).

BMVC'10, Weighted Local Bundle Adjustment and Application to Odometry and Visual SLAM Fusion, PDF,
A. Eudes, S. Naudet-Collette, M. Lhuillier and M. Dhome,
Proceedings of the 21th British Machine Vision Conference, Aberystwyth, United Kingdom, September 2010.

ICPR'10, Fast Odometry Integration in Local Bundle Adjustment-based Visual SLAM, PDF,
A. Eudes, M. Lhuillier, S. Naudet-Collette and M. Dhome,
Proceedings of the IAPR International Conference on Pattern Recognition, Istambul, Turkey, August 2010.

RFIA'10, Propagations d'erreur pour l'ajustement de faisceaux local, PDF,
A. Eudes, M. Lhuillier, S. Naudet-Collette,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Caen, France, January 2010.

CVPR'09, Error Propagations for Local Bundle Adjustment, PDF,
A. Eudes and M. Lhuillier,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida, June 2009.

CVPR'08, Toward Automatic 3D Modeling of Scenes using a Generic Camera Model, PDF,
M. Lhuillier,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska, June 2008.

RFIA'08, Vers la modélisation 3D flexible avec une caméra catadioptrique, PDF,
M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Amiens, France, January 2008.

RFIA'08, Reconstruction 3D générique et temps réel, PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Amiens, France, January 2008.

BMVC'07, Generic and Real Time Structure from Motion, PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the 18th British Machine Vision Conference, Warwick, United Kingdom, September 2007.

CVPR'07, Toward Flexible 3D Modeling using a Catadioptric Camera, PDF,
M. Lhuillier,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA, June 2007.

ICPR'06, Monocular Vision Based SLAM for Mobile Robots, PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the IAPR International Conference on Pattern Recognition, Hong Kong, China, August 2006.

ICPR'06, Effective and Generic Structure from Motion using Angular Error, PDF,
M. Lhuillier,
Proceedings of the IAPR International Conference on Pattern Recognition, Hong Kong, China, August 2006.

CVPR'06, Real-Time Localization and 3D Reconstruction, PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New-York, USA, June 2006.

ICRA'06, Uncertainty ellipsoids calculations for complex 3D reconstruction, PDF,
M. Lhuillier and M. Perriollat,
Proceedings of the IEEE International Conference on Robotics and Automation, Orlando, USA, May 2006.

ICRA'06, 3D Reconstruction of complex structures with bundle adjustment: an incremental approach, PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the IEEE International Conference on Robotics and Automation, Orlando, USA, May 2006.

RFIA'06, Reconstruction 3D automatique avec une caméra catadioptrique, PDF,
M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Tours, France, January 2006.

RFIA'06, Localisation par vision monoculaire pour la navigation autonome d'un robot mobile, PDF,
E. Royer, M. Lhuillier, M. Dhome and J.M. Lavest,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Tours, France, January 2006.

OMNIVIS'05, Automatic Structure and Motion using a Catadioptric Camera, PDF,
M. Lhuillier,
Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-classical cameras, Beijing, China, October 2005.

BENCOS'05, Performance Evaluation of a Localization System Relying on Monocular Vision and Natural Landmarks, PDF,
E. Royer, M. Lhuillier, M. Dhome and J.M. Lavest,
Proceedings of the ISPRS Workshop BenCOS (Towards Benchmarking Automated Calibration, Orientation and Surface Reconstruction from Images), Beijing, China, October 2005.

IROS'05, Outdoor autonomous navigation using monocular vision, PDF,
E. Royer, J. Bom, M. Dhome, B. Thuilot, M. Lhuillier and F. Marmoiton,
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Alberta, Canada, August 2005.

CVPR'05, Localization in urban environments: monocular vision compared to a differential GPS sensor, PDF,
E. Royer, M. Lhuillier, M. Dhome and T. Chateau,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, June 2005.

GIAE'04, Recent Methods for Reconstructing Surfaces from Multiple Images, ,
G. Zeng, M. Lhuillier and L. Quan,
Lecture Notes in Computer Science, Volume 3519, Springer Verlag, 2005 (revised selected papers of GIAE/IWMM'2004).

BMVC'04, Towards an alternative GPS sensor in dense urban environment from visual memory, PDF,
E. Royer, M. Lhuillier, M. Dhome and T. Chateau,
Proceedings of the 15th British Machine Vision Conference, London, United Kingdom, September 2004.

ECCV'04, Surface Reconstruction by Propagating 3D Stereo Data in Multiple 2D Images, PDF,
G. Zeng, S. Paris, L. Quan and M. Lhuillier,
Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic, Volume 1, pages 163-174, May 2004.
Also in Lecture Notes in Computer Science 3021, Springer Verlag.

ACCV'04, Fast Segmentation-based dense stereo from quasi-dense matching, PDF,
Y. Wei, M. Lhuillier and L. Quan,
Proceedings of the Asian Conference on Computer Vision, Jeju Island, Korea, January 2004.

RFIA'04, Reconstruction quasi-dense et modèles 3D à partir d'une séquence d'images, PDF,
M. Lhuillier and L. Quan,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle, Toulouse, France, January 2004.

ICCV'03, Surface Reconstruction by Integrating 3D and 2D Data of Multiple Views, PDF,
M. Lhuillier and L. Quan,
Proceedings of the 9th International Conference on Computer Vision, Nice, France, October 2003.

ICPR'02, Structure from Motion from Three Affine Views, PDF,
L. Quan and M. Lhuillier,
Proceedings of the IAPR International Conference on Pattern Recognition, Quebec, Volume 4, August 2002.

ECCV'02, Quasi-Dense Reconstruction from Image Sequence, PDF,
M. Lhuillier and L. Quan,
Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, Volume 2, pages 125-139, May 2002.
Also in Lecture Notes in Computer Science 2351, Springer Verlag.

ICCV'01, Concentric Mosaics, Planar Motions and 1D Cameras,
L. Quan, L. Lu, H. Shum and M. Lhuillier,
Proceedings of the 8th International Conference on Computer Vision, Vancouver, Canada, 2001.

CVPR'01, Relief Mosaics by Joint View Triangulation, PDF,
M. Lhuillier and L. Quan,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, December 2001.

ICPR'00, Robust Dense Matching Using Local and Global Geometric Constraints, PDF,
M. Lhuillier and L. Quan,
Proceedings of the IAPR International Conference on Pattern Recognition, Barcelone, Spain, Volume 2, pages 968-972, September 2000.
ICPR'2000 Piero Zamperoni Best Student Paper Award.

CVPR'00, Edge-Constrained Joint View Triangulation for Image Interpolation, PDF,
M. Lhuillier and L. Quan,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, USA, Volume 2, pages 218-224, June 2000.

RFIA'00, Appariement dense robuste a l'aide de contraintes geometriques locales et globales, PDF,
M. Lhuillier and L. Quan,
Actes du 12eme Congrès Francophone AFRIF-AFIA de Reconnaissance des Formes et Intelligence Artificielle, Paris, Volume 3, pages 215-223, February 2000.

CVPR'99, Image Interpolation by Joint View Triangulation, PDF,
M. Lhuillier and L. Quan,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, USA, Volume 2, pages 139-145, June 1999.

VI'99, Joint View Triangulation for Two Views, PDF,
M. Lhuillier,
Proceedings of the 12th Conference on Vision Interface, Trois Rivieres, Quebec, pages 360-367, May 1999.
Also in INRIA technical report RR-3619 (title: Towards Automatic Interpolation for Real and Distant Image Pairs ), February 1999, PDF.

BMVC'98, Efficient Dense Matching for Textured Scenes Using Region Growing, PDF,
M. Lhuillier,
Proceedings of the 9th British Machine Vision Conference, Southampton, United Kingdom, Volume 2, pages 700-709, September 1998.
Also in INRIA technical report RR-3382, Mars 1998, PDF.


Paragliding and paramotor

Also, I like paragliding and paramotor. Here are videos and shots that i take during flying.

August 22th 2015: Here is an interactive 360 video obtained from four GoPro 3 cameras mounted on a helmet.



July 22th 2013: Here is a (noninteractive) 360 video obtained from four GoPro 3 cameras mounted on a helmet.



October 3th 2012: The following youtube videos (HD) show paramotor crosses near gorges du Verdon and in Auvergne (puy de Dome, Pariou, Sancy).



July 28th 2007: See some omnidirectional shots.



October 25th 2005: I was a participant of the masquerade contest at the 32nd Icarus Cup / Coupe Icare as Skippy the Kangaroo !
Thanks to Frederic Jurie for providing the following images.



October 25th 2005: See some shots of the masquerade contest at the 31st Icarus Cup / Coupe Icare.