30hp - Camera-supervised LiDAR semantic segmentation for autonomous trucks

Introduction:
Thesis work is an excellent way to get closer to Scania and build relationships for the future. Many of today's employees began their Scania career with their degree project.

Background:
This thesis will be carried out at the Autonomous Solutions department at Scania, where we develop cutting-edge research solutions for scene perception to enable autonomous driving. In recent years, deep neural networks have outperformed many classical computer vision approaches in tasks such as semantic segmentation, object detection, and tracking.

A crucial factor for training such networks is high-quality labelled data. However, manually annotating large-scale datasets is extremely costly and time-consuming. This is where auto-labelling solutions come into play. By automatically generating labels for recorded sensor data offline (i.e., outside the autonomous vehicle), we can provide training material for onboard perception networks—the networks running in real-time on the vehicle itself.

Currently the semantic class of lidar points is used extensively to understand which points remain static over time, are possible to drive on and are likely to change position in the future. How can we classify LiDAR points accurately without relying on costly manual annotations? One current approach is to use existing camera-based segmentation models through point painting projection. This approach gives very accurate classifications but introduces a lot of projection errors and is therefore far from optimal. Is it possible to utilize this approach to train a new network which classifies LiDAR points directly only based on the ambient, intensity, reflectivity and depth signals of the lidar detections?

Objective:
The objective of this thesis is to investigate methods for generating semantic labels for LiDAR data with supervision from camera-based segmentation. Using camera-based labels can be difficult since there will be a projection error between 2d camera images and 3d lidar output. The thesis will focus on training a LiDAR semantic segmentation network that is robust to these noisy labels and delivers output that is both accurate and consistent with the camera outputs.

Thesis Description:
The thesis may include some of the following tasks:
•    Conducting a literature review on state-of-the-art methods in LiDAR semantic segmentation, weak supervision, self-supervision, label efficient methods and point clustering heuristics for autonomous driving.
•    Developing methods for generating semantic labels using camera-based models and heuristic rules.
•    Exploring how temporal context can improve segmentation quality.
•    Investigating robust training setups that can fit to imperfect labels
•    Comparing inference speed and accuracy of different approaches, such as 2d approaches vs 3d approaches
•    Designing approaches to evaluate and the quality of auto-generated labels.

Through this work, the student will gain a deeper understanding of how offline perception and auto-labelling can support large-scale dataset creation and provide new insights into the design of machine learning models for autonomous driving.

The successful candidate will have the opportunity to work with rich multimodal datasets, cutting-edge perception architectures, and the latest research challenges in autonomous transport. Collaboration with experienced researchers and developers in Scania’s Autonomous Transport Solutions Pre-Development & Research department will provide both industrial relevance and academic depth to the work.

Qualifications:
-    Currently enrolled in a Master program in Computer Science, Electrical Engineering or related field
-    Good understanding of computer vison, machine learning and practice thereof
-    Software development knowledge and proficiency in a relevant programming language such as Python or C++
-    Ability to work in a diverse environment and communicate effectively in English

Preferred qualifications:
-    Passion and confidence to bring your own ideas into the thesis
-    Prior experience with pytorch or mmdet3d
-    Excellent problem-solving skills and the ability to work independently

Time plan :
The project is planned for 20 weeks and can be started any time in early Spring 2025. Applicants will be assessed on a continuous basis until the position is filled.

Number of students: 1

Contact persons and supervisors:
Fredrik Nordin, fredrik.y.nordin@scania.com, Development Engineer, Autonomous Transport Solutions, Traton AB

Application:
Your application must include a CV, personal letter and transcript of grades

 
A background check might be conducted for this position. We are conducting interviews continuously and may close the recruitment earlier than the date specified.