Internship proposal : « Visual-Odometry for Autonomous Vehicles based on RGB-D-T cameras »
Keywords
Computer vision, Visual odometry, Sensor Fusion, Non conventional visual sensors, Thermal sensors, Davis sensors, Optimization, B-splines
Abstract
Visual-Odometry (VO) is the main perception task for a robot in order to estimate incrementally its 3D motion using 2D image sequences. VO is often associated with 3D reconstruction aswell as localization thus allowing the robot to understand its surrounding environment and locating itself (position and attitude) in a continuous manner and in real-time.
In this internship project, we propose a VO task that could take a benefit from all the advantages
of various visual sensors, a conventional camera (RGB), a depth camera (D), and an Infra-Red camera (T), in a continuous-time sensor fusion framework as illustrated in Fig. 1.
More precisely, the main task is to study continuous-time camera trajectory optimization using highly efficient B-splines as in [2, 3] and IR camera (T) features extraction and investigate the methods to integrate them in the same B-spline optimization function for the RGB-D
camera frames modeled in [4, 5]. Some novel works dealt with this problem using discrete time
particle filters as in [6].
Our goal is to integrate this RGB-D-T continuous-time B-spline trajectory as a real-time map into a complete heterogeneous vision system (RGB-D-T + DAVIS sensor). Accordingly, this RGB-D-T map is supposed to be very accurate, fast (continuous-time) and robust for all
challenging environments. The intern student will work closely with a PhD student, as well as the professors involved
in the project.
Competences
1. Solid background in Computer vision and Optimization.
2. Good understanding of thermal sensors features extraction and matching
3. Good skills in programming
4. Highly motivated and results oriented.
Software Environments
- MATLAB – Python (for research and development)
- C/C++ (for implementation and production)
Internship Information
- Period: from February 1st 2021, to July 23th 2021
Application Procedures
Send motivation letter, CV and transcripts of M1 and/or M2 to the contact below :
- The transmission of letters of recommendation, references contact details aswell as master results will be appreciated.
- The deadline for application is January 4th, 2021.
Contact information
- Abanob SOLIMAN, Phd student, IBISC, abanobDOTsolimanATuniv-evryDOTfr
- Samia BOUCHAFA-BRUNEAU, professor, IBISC, samiaDOTbouchafaATibisc.univ-evryDOTfr
- Desire SIDIBE, professor, IBISC, drodesireDOTsidibieATuniv-evryDOTfr
- Fabien BONARDI, assistant professor, IBISC, fabienDOTbonardiATuniv-evryDOTfr
References
[1] S. Cosar, N. Bellotto, Human Re-Identification with a Robot Thermal Camera using Entropy-based Sampling, Journal of Intelligent & Robotic Systems, 2019.
[2] Sommer, Christiane, et al. « Efficient Derivative Computation for Cumulative B-Splines on Lie Groups. » Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.
[3] Patron-Perez, Alonso, Steven Lovegrove, and Gabe Sibley. « A spline-based trajectory representation for sensor fusion and rolling shutter cameras. » International Journal of Computer Vision 113.3 (2015): 208-219.
[4] Kerl, Christian, Jorg Stuckler, and Daniel Cremers. « Dense continuous-time tracking and mapping with rolling shutter RGB-D cameras. » Proceedings of the IEEE international conference on computer vision. 2015.
[5] Kerl, Christian, Jorg Stuckler, and Daniel Cremers. « Dense Continuous-Time Tracking and Mapping with Rolling Shutter RGB-D Cameras Supplementary Material. » TC 1.1: A1.
[6] Spremolla, Ignacio Rocco, et al. « RGB-D and Thermal Sensor Fusion-Application in Person Tracking. » VISIGRAPP (3: VISAPP). 2016.
- Date de l’appel : 11/12/2020
- Statut de l’appel : Pourvu
- Contact coté IBISC : Abanob SOLIMAN (doctorant IBISC équipe SIAM), Samia BOUCHAFA-BRUNEAU (PR Univ. Évry, IBISC équipe SIAM, Désiré SIDIBE (PR Univ. Évry, IBISC équipe SIAM), Fabien BONARDI (MCF Univ. Évry, IBISC équipe SIAM)
- Sujet de stage niveau Master 2 (format PDF)
- Web équipe SIAM
