Environment perception for autonomous cars

Monday, October 17, 2016

Hossein Darai advanced to candidacy with a Ph.D. thesis proposal focusing on sensing technologies for autonomous cars. 

Good job Hossein! Here is the abstract of his thesis proposal:

Environment Perception Using Lidar And Camera Fusion For Autonomous Driving

Autonomous vehicles need sensors of different modalities in order to understand the environment that contain multiple rigidly moving objects. Segmenting the scene, accurate localization of each object, and estimating their motion are all necessary for prediction and planning. At each time instance and based upon raw sensor measurements in a short time window, we are interested in an object list with position, shape model, and velocity vector. In this work, we propose to combine a sparse 4-layer LiDAR with a front-facing camera to perform a low-level (raw measurement level) fusion.

There are several challenges in segmentation and motion inference, mainly due to imperfections of these sensors. LiDAR measures depth at various angles, but the measurements are sparse, have very low resolution, suffer from noise, include erroneous reflections, and contain missing data. Camera images, on the other hand, are dense and rich, but lack depth information. In addition, they contain noise and are adversely affected by lighting conditions, e.g. sun, overcast, night. There are yet more challenges that make fusion of these two sensors very difficult; for instance, they have completely different natures, different timestamps, and even different sampling rates.

The main objective of this work is, then, to overcome these challenges and build an environment perception module which performs joint segmentation, estimation, and tracking through sensor fusion. The proposed estimation method addresses the problem in terms of a joint intermediate model, and then minimizes a robust energy function that considers both visual data (optical flow term) and point cloud data (geometric term), while enforcing a smooth motion field over time.