Email:
Maxime { dot } Lhuillier { at } free { dot } fr Address:
Institut Pascal, UMR 6602 CNRS/UCA,
Campus Universitaire des Cézeaux (4 av. Blaise Pascal),
63178 Aubière Cedex (TSA 60026, CS 60026) - France. Tel: +33 (0)4 73 40 75 93 Fax: +33 (0)4 73 40 72 62
Links
Demos (updated on June 24th 2022) PhD Students (updated on September 17th 2018) Publications (updated on September 1th 2023)
Welcome !
I'm a researcher at the Institut Pascal (chargé de recherche CNRS, HDR)
working on Computer Vision.
My research topics are, in chronological order from 1998 to now,
3D Reconstruction for Robotic: structure-from-motion, visual SLAM,
localization,
bundle adjustments (hierarchical/global and incremental/local versions),
calibrated cameras (pinhole, fish-eye, Gopro),
key-frame selection, uncertainty, GPS (fusion)
Omnidirectional Vision: camera models (catadioptric, 360 camera,
spherical camera, central, non-central, generic, multi-camera),
structure-from-motion (image and angular bundle adjustments),
surface reconstruction (2-manifold, 3D Delaunay triangulation),
multi-camera self-calibration (synchronization, rolling shutter,
bundle adjustment), virtual reality (VR)
More details are given in the publications and
demos below.
For all topics, the main difficulty is to obtain fully automatic methods
involving low cost hardware (a camera and a PC) most of the time, ground imagery
and uncontrolled environments (neither beacons/turntable/friendly background
nor highly finished light control nor well selected viewpoints using UAV).
Eight 3D scans are available herefor a lot of VR headsets including the Oculus Quest in standalone
mode and other headsets linked to a PC (with Windows Mixed Reality or SteamVR).
The input videos are taken using a helmet-held 360 camera, eg Garmin Virb 360 or
Gopro Max 360, by walking most of times in
a basalt canyon,
Collonges-la-Rouge,
a semi-medieval town,
a campus,
a snowy town,
a forest,
the Verdon's canyon, or
a ruin in a forest.
Videos captured by an Oculus Quest are available from a
playlist.
If you do not have a VR headset, you can explore reduced versions of the scans
(complete geometry, incomplete and reduced texture) at Sketchfab by clicking on
one of the following icons.
Immersive 3D modeling and 360 video using helmet-held gopro cameras
[ 3DV'15, CVIU'13, ICPR'14, CVIU'17, ...]
Here are two 3D models reconstructed from videos which can be
viewed interactively using Sketchfab.
The videos are taken by biking 2.5km in a city, or by walking during 4 minutes
in a forest.
The following video shows a surface reconstructed from a 900m long video
sequence taken by walking in a town.
Thanks to B. Achermann (University of Bern, Switzerland) for providing the
input images.
These faces are ordered in the alphabetical order and automatically morphed.
Thanh-Tin Nguyen did a Ph.D thesis (2014-2017) about the
self-calibration of several consumer cameras including synchronization and
rolling shutter.
Now Thant-Tin is working at Xvisio Technology (firm on VSLAM/AI).
Vadim Litvinov did a Ph.D thesis (2011-2014) about the incremental
surface reconstruction using a calibrated multi-camera and sparse
structure-from-motion data.
Now Vadim is working at Machine Zone (firm on mobile gaming).
Shuda Yu did a Ph.D thesis (2010-2013) about
the surface reconstruction of environments using a catadioptric camera and
sparse structure-from-motion data.
Now Shuda is working at SuperD (firm on glasses-free 3d technologies).
Alexandre Eudes did a Ph.D thesis (2007-2010)
about the Odometry for driving help in urban areas.
Now Alexandre is working at ONERA.
Etienne Mouragnon did a Ph.D thesis (2004-2007)
about the incremental estimation of the geometry of a video.
Now, Etienne is working at AIGL Developpement (firm on photogrammetry).
Eric Royer did (a Master and) a Ph.D thesis (2003-2006)
about the localization of vehicles in urban environments.
Now, Eric is lecturer at the university in Clermont Ferrand.
CVIU'23,
Estimating the vertical direction in a photogrammetric 3D model, with application to visualization, (this pdf is not yet free, but the others are),
M. Lhuillier,
Computer Vision and Image Understanding, 236:1-13, 2023.
CVIU'18,
Surface reconstruction from a sparse point cloud by enforcing visibility
consistency and topology constraints,PDF (also
available at the
CVIU web site),
M. Lhuillier,
Computer Vision and Image Understanding, 175:52-71, 2018.
CVIU'17,
Self-calibration of omnidirectional multi-cameras including
synchronization and rolling shutter,PDF (also
available at the
CVIU web site),
T.T. Nguyen and M. Lhuillier,
Computer Vision and Image Understanding, 162:166-184, 2017.
JMIV'17,
Overview of Shelling for 2-Manifold Surface Reconstruction based on 3D
Delaunay Triangulation,PDF (also available at
the
JMIV web site),
M. Lhuillier,
Journal of Mathematical Imaging and Vision, 59(2):318-340, 2017.
JMIV'15,
2-Manifold Tests for 3D Delaunay Triangulation-based Surface
Reconstruction,PDF (also available at the
JMIV web site),
M. Lhuillier,
Journal of Mathematical Imaging and Vision, 51(1):98-105, 2015.
CVIU'13,
Manifold Surface Reconstruction of an Environment from Sparse Structure-from-Motion Data,PDF (also available at the
CVIU web site),
M. Lhuillier and S. Yu,
Computer Vision and Image Understanding, 117(11):1628-1644, 2013.
PAMI'12,
Incremental Fusion of Structure-from-Motion and GPS using Constrained Bundle Adjustment,PDF (also available at the
PAMI web site),
M. Lhuillier,
IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12), 2012.
IJCV'11,
A Generic Error Model and its Application to Automatic 3D Modeling of Scenes using a Catadioptric Camera,PDF
(also available at the
IJCV web site),
M. Lhuillier,
International Journal of Computer Vision, 91(2):175-199.
TOG'09,
Image-based Street-side City Modeling,
PDF,
J. Xiao, T. Fang, P. Zhao, M. Lhuillier and L. Quan,
ACM Transaction on Graphics, 28(5), 2009
(also in proceedings of SIGGRAPH ASIA'09).
IVC'09,
Generic and Real-Time Structure from Motion using Local Bundle Adjustment,
PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Image and Vision Computing, 27:1178-1193, 2009.
CVIU'08,
Automatic Scene Structure and Camera Motion using a Catadioptric System,
PDF,
M. Lhuillier,
Computer Vision and Image Understanding, 109(2):186-203, 2008.
IJCV'07/IJRR'07,
Monocular vision for mobile robot localization and autonomous navigation,
PDF,
E. Royer, M. Lhuillier, M. Dhome and J.M. Lavest,
International Journal of Computer Vision, 74(3),
(special joint issue on vision and robotics,
with the International Journal of Robotics Research).
TS'06,
Localisation par vision monoculaire pour la navigation autonome,
PDF,
E. Royer, M. Lhuillier, M. Dhome and J.M. Lavest,
Traitement du Signal, 23(1), 2006.
TPAMI'05,
A Quasi-Dense Approach to Surface Reconstruction from Uncalibrated
Images,
PDF, (These files include large figures and
appendix.)
M. Lhuillier and L. Quan,
IEEE Transactions on Pattern Analysis and Machine Intelligence,
27(3):418-433, 2005.
TCSVT'03,
Image-Based Rendering by Joint View Triangulation,
PDF,
M. Lhuillier and L. Quan,
IEEE Transactions on Circuits and Systems for Video Technology,
13(11):1051-1063, 2003.
TPAMI'02,
Match Propagation for Image-Based Modeling and Rendering,
PDF,
M. Lhuillier and L. Quan,
IEEE Transactions on Pattern Analysis and Machine Intelligence,
24(8):1140-1146, 2002.
Book's chapter
ISTE'23,
Reconstruction of environments,
M. Lhuillier,
Chapter of book "Omnidirectional vision: from theory to applications",
ISTE Science Publishing LTD, to appear in 2023.
Conferences
RFIAP'22,
De la reconstruction par photogrammétrie à un environnement de réalité
virtuel immersif,
PDF
M. Lhuillier,
Accepted at the Congrès francophone AFRIF-AFIA de reconnaissance des
formes, image, apprentissage et perception, Vannes, France, June 2022.
IC3D'21,
From photogrammetric reconstruction to immersive VR environment,
PDF
(see also PDF and
slides)
M. Lhuillier,
Proceedings of the International Conference on 3D Immersion,
Brussels, Belgium, December 2021.
IC3D'20,
Toward texturing for immersive modeling of environment reconstructed from 360 multi-camera,
PDF
(see also PDF and
slides)
M. Lhuillier,
Proceedings of the International Conference on 3D Immersion,
Brussels, Belgium, December 2020.
RFIAP'20,
Renforcer la convexité locale pour reconstruire une scène à partir d'un nuage de points épars,
PDF
M. Lhuillier,
Accepted at the Congrès francophone AFRIF-AFIA de reconnaissance des
formes, image, apprentissage et perception, Vannes (virtual conference), France, June 2020.
IC3D'19,
Local-convexity reinforcement for scene reconstruction from sparse point clouds,
PDF
(see also PDF and
slides)
M. Lhuillier,
Proceedings of the International Conference on 3D Immersion,
Brussels, Belgium, December 2019.
3DRW'18,
Improving thin structures in surface reconstruction from sparse point cloud,
PDF,
M. Lhuillier,
Workshop on 3D reconstruction in the wild, in conjunction with ECCV 2018,
Munich, Germany, September 2018.
Also in Lecture Notes in Computer Science 11129, pp.443-458, Springer.
RFIAP'18,
Ajustement de faisceaux multi-caméra intégrant l'estimation
de la synchronisation et du rolling-shutter,
PDF
T.T. Nguyen and M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes, image, apprentissage et perception, Marne-la-Vallée, France, June 2018.
BMVC'16,
Adding Synchronization and Rolling Shutter in Multi-Camera Bundle Adjustment,
PDF (supp. mat.)
T.T. Nguyen and M. Lhuillier,
Proceedings of the 27th British Machine Vision Conference,
York, United Kingdom, September 2016.
RFIA'16,
Synchronisation et auto-étalonnage de plusieurs caméras
fixées sur un casque,
PDF
T.T. Nguyen and M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Clermont Ferrand, France, June 2016.
3DV'15,
Synchronization and Self-Calibration for Helmet-Held Consumer Cameras,
Applications to Immersive 3D Modeling and 360 Video,
PDF
M. Lhuillier and T.T. Nguyen,
Proceedings of the International Conference on 3D Vision, Lyon, France,
October 2015.
ICPR'14,
Incremental Solid Modeling from Sparse Structure-from-Motion Data with Improved Visual Artifacts Removal,
PDF
V. Litvinov and M. Lhuillier,
Proceedings of the IAPR International Conference on Pattern Recognition,
Stockholm, Sweden, August 2014.
RFIA'14,
Estimation incrémentale de surface à partir d'un nuage de
points épars reconstruit à partir d'images
omnidirectionnelles,
PDF
V. Litvinov and M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Rouen, France, July 2014.
BMVC'13,
Incremental Solid Modeling from Sparse and Omnidirectional Structure-from-Motion Data,
PDF
V. Litvinov and M. Lhuillier,
Proceedings of the 22th British Machine Vision Conference, Bristol, United
Kingdom, September 2013.
IC3D'12,
2-Manifold Reconstruction from Sparse Visual Features,
PDF,
V. Litvinov, S. Yu and M. Lhuillier,
Proceedings of the International Conference on 3D Imaging, Liège,
Belgium, December 2012.
ICPR'12,
Genus refinement of a manifold surface reconstructed by sculpting the
3d-Delaunay triangulation of Structure-from-Motion points,
PDF,
S. Yu and M. Lhuillier,
Proceedings of the IAPR International Conference on Pattern
Recognition, Tsukuba, Japan, November 2012.
3DIMPVT'12,
Incremental Reconstruction of Manifold Surface from Sparse Visual
Mapping,
PDF,
S. Yu and M. Lhuillier,
Proceedings of the second joint 3DIM/3DPVT Conference
(3D Imaging, Modeling, Processing, Vizualization and Transmission), Zurich,
Switzerland, October 2012.
RFIA'12,
Fusion GPS-vision à l'aide d'ajustement de faisceaux
contraints,
PDF,
M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Lyon, France, January 2012.
RFIA'12,
Modèles 3d à partir d'un nuage épars de points et
une caméra catadioptrique,
PDF,
S. Yu and M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Lyon, France, January 2012.
MIRAGE'11,
Surface Reconstruction of Scenes using a Catadioptric Camera,
PDF,
S. Yu and M. Lhuillier,
Lecture Notes in Computer Science, Volume 6930, Springer Verlag, 2011
(also in the conference on Computer Vision/Computer Craphics Collaboration
Techniques and Applications, October 2011).
CVPR'11,
Fusion of GPS and Structure-from-Motion using Constrained Bundle
Adjustments,
PDF,
M. Lhuillier,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
Colorado Springs, Colorado, June 2011.
PCV'10,
Environment Modeling from Images taken by a Low Cost Camera,
PDF,
M. Lhuillier,
International Archives of Photogrammetry, Remote Sensing and Spatial
Information Sciences, vol. XXXVIII part 3A, 2010
(also in proceedings of PCV'10).
BMVC'10,
Weighted Local Bundle Adjustment and Application to Odometry and Visual SLAM Fusion,
PDF,
A. Eudes, S. Naudet-Collette, M. Lhuillier and M. Dhome,
Proceedings of the 21th British Machine Vision Conference, Aberystwyth, United
Kingdom, September 2010.
ICPR'10,
Fast Odometry Integration in Local Bundle Adjustment-based Visual SLAM,
PDF,
A. Eudes, M. Lhuillier, S. Naudet-Collette and M. Dhome,
Proceedings of the IAPR International Conference on Pattern Recognition,
Istambul, Turkey, August 2010.
RFIA'10,
Propagations d'erreur pour l'ajustement de faisceaux local,
PDF,
A. Eudes, M. Lhuillier, S. Naudet-Collette,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Caen, France, January 2010.
CVPR'09,
Error Propagations for Local Bundle Adjustment,
PDF,
A. Eudes and M. Lhuillier,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
Miami, Florida, June 2009.
CVPR'08,
Toward Automatic 3D Modeling of Scenes using a Generic Camera Model,
PDF,
M. Lhuillier,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
Anchorage, Alaska, June 2008.
RFIA'08,
Vers la modélisation 3D flexible avec une caméra
catadioptrique,
PDF,
M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Amiens, France, January 2008.
RFIA'08,
Reconstruction 3D générique et temps réel,
PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Amiens, France, January 2008.
BMVC'07,
Generic and Real Time Structure from Motion,
PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the 18th British Machine Vision Conference, Warwick, United
Kingdom, September 2007.
CVPR'07,
Toward Flexible 3D Modeling using a Catadioptric Camera,
PDF,
M. Lhuillier,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
Minneapolis, USA, June 2007.
ICPR'06,
Monocular Vision Based SLAM for Mobile Robots,
PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the IAPR International Conference on Pattern Recognition,
Hong Kong, China, August 2006.
ICPR'06,
Effective and Generic Structure from Motion using Angular Error,
PDF,
M. Lhuillier,
Proceedings of the IAPR International Conference on Pattern Recognition,
Hong Kong, China, August 2006.
CVPR'06,
Real-Time Localization and 3D Reconstruction,
PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
New-York, USA, June 2006.
ICRA'06,
Uncertainty ellipsoids calculations for complex 3D reconstruction,
PDF,
M. Lhuillier and M. Perriollat,
Proceedings of the IEEE International Conference on Robotics and Automation,
Orlando, USA, May 2006.
ICRA'06,
3D Reconstruction of complex structures with bundle adjustment: an
incremental approach,
PDF,
E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser and P. Sayd,
Proceedings of the IEEE International Conference on Robotics and Automation,
Orlando, USA, May 2006.
RFIA'06,
Reconstruction 3D automatique avec une caméra
catadioptrique,
PDF,
M. Lhuillier,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Tours, France, January 2006.
RFIA'06,
Localisation par vision monoculaire pour la navigation autonome d'un robot mobile,
PDF,
E. Royer, M. Lhuillier, M. Dhome and J.M. Lavest,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Tours, France, January 2006.
OMNIVIS'05,
Automatic Structure and Motion using a Catadioptric Camera,
PDF,
M. Lhuillier,
Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and
Non-classical cameras, Beijing, China, October 2005.
BENCOS'05,
Performance Evaluation of a Localization System Relying on Monocular
Vision and Natural Landmarks,
PDF,
E. Royer, M. Lhuillier, M. Dhome and J.M. Lavest,
Proceedings of the ISPRS Workshop BenCOS (Towards Benchmarking Automated
Calibration, Orientation and Surface Reconstruction from Images), Beijing,
China, October 2005.
IROS'05,
Outdoor autonomous navigation using monocular vision,
PDF,
E. Royer, J. Bom, M. Dhome, B. Thuilot, M. Lhuillier and F. Marmoiton,
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and
Systems, Alberta, Canada, August 2005.
CVPR'05,
Localization in urban environments: monocular vision compared to a
differential GPS sensor,
PDF,
E. Royer, M. Lhuillier, M. Dhome and T. Chateau,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
San Diego, USA, June 2005.
GIAE'04,
Recent Methods for Reconstructing Surfaces from Multiple Images,
,
G. Zeng, M. Lhuillier and L. Quan,
Lecture Notes in Computer Science, Volume 3519, Springer Verlag, 2005
(revised selected papers of GIAE/IWMM'2004).
BMVC'04,
Towards an alternative GPS sensor in dense urban environment from visual
memory,
PDF,
E. Royer, M. Lhuillier, M. Dhome and T. Chateau,
Proceedings of the 15th British Machine Vision Conference, London, United
Kingdom, September 2004.
ECCV'04,
Surface Reconstruction by Propagating 3D Stereo Data in Multiple 2D
Images,
PDF,
G. Zeng, S. Paris, L. Quan and M. Lhuillier,
Proceedings of the 8th European Conference on Computer Vision, Prague,
Czech Republic, Volume 1, pages 163-174, May 2004.
Also in Lecture Notes in Computer Science 3021, Springer Verlag.
ACCV'04,
Fast Segmentation-based dense stereo from quasi-dense matching,
PDF,
Y. Wei, M. Lhuillier and L. Quan,
Proceedings of the Asian Conference on Computer Vision, Jeju Island,
Korea, January 2004.
RFIA'04,
Reconstruction quasi-dense et modèles 3D à partir d'une
séquence d'images,
PDF,
M. Lhuillier and L. Quan,
Proceedings of the Congrès francophone AFRIF-AFIA de reconnaissance des
formes et d'intelligence artificielle, Toulouse, France, January 2004.
ICCV'03,
Surface Reconstruction by Integrating 3D and 2D Data of Multiple
Views,
PDF,
M. Lhuillier and L. Quan,
Proceedings of the 9th International Conference on Computer Vision, Nice,
France, October 2003.
ICPR'02,
Structure from Motion from Three Affine Views,
PDF,
L. Quan and M. Lhuillier,
Proceedings of the IAPR International Conference on Pattern Recognition,
Quebec, Volume 4, August 2002.
ECCV'02,
Quasi-Dense Reconstruction from Image Sequence,
PDF,
M. Lhuillier and L. Quan,
Proceedings of the 7th European Conference on Computer Vision, Copenhagen,
Denmark, Volume 2, pages 125-139, May 2002.
Also in Lecture Notes in Computer Science 2351, Springer Verlag.
ICCV'01,
Concentric Mosaics, Planar Motions and 1D Cameras,
L. Quan, L. Lu, H. Shum and M. Lhuillier,
Proceedings of the 8th International Conference on Computer Vision, Vancouver,
Canada, 2001.
CVPR'01,
Relief Mosaics by Joint View Triangulation,
PDF,
M. Lhuillier and L. Quan,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
Hawaii, USA, December 2001.
ICPR'00,
Robust Dense Matching Using Local and Global Geometric Constraints,
PDF,
M. Lhuillier and L. Quan,
Proceedings of the IAPR International Conference on Pattern Recognition,
Barcelone, Spain, Volume 2, pages 968-972, September 2000.
ICPR'2000 Piero Zamperoni Best Student Paper Award.
CVPR'00,
Edge-Constrained Joint View Triangulation for Image Interpolation,
PDF,
M. Lhuillier and L. Quan,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
Hilton Head Island, South Carolina, USA, Volume 2, pages 218-224, June 2000.
RFIA'00,
Appariement dense robuste a l'aide de contraintes geometriques locales et
globales,
PDF,
M. Lhuillier and L. Quan,
Actes du 12eme Congrès Francophone AFRIF-AFIA de Reconnaissance
des Formes et Intelligence Artificielle, Paris, Volume 3, pages 215-223,
February 2000.
CVPR'99,
Image Interpolation by Joint View Triangulation,
PDF,
M. Lhuillier and L. Quan,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
Fort Collins, Colorado, USA, Volume 2, pages 139-145, June 1999.
VI'99,
Joint View Triangulation for Two Views,
PDF,
M. Lhuillier,
Proceedings of the 12th Conference on Vision Interface, Trois Rivieres, Quebec,
pages 360-367, May 1999.
Also in INRIA technical report RR-3619 (title:
Towards Automatic Interpolation for Real and Distant Image Pairs ),
February 1999,
PDF.
BMVC'98,
Efficient Dense Matching for Textured Scenes Using Region Growing,
PDF,
M. Lhuillier,
Proceedings of the 9th British Machine Vision Conference, Southampton,
United Kingdom, Volume 2, pages 700-709, September 1998.
Also in INRIA technical report RR-3382, Mars 1998,
PDF.
