Master Thesis Worker - Automated Quality Evaluation of Developer Assistant Responses
Introduction
Thesis work is an excellent way to get closer to Scania and build relationships for the future. Many of today's employees began their Scania career with their degree project.
Background
The TRATON Developer Portal provides an AI-powered Developer Assistant that helps developers find information and guidance from internal technical documentation. As its use grows, a systematic approach is needed to evaluate response quality and identify opportunities to improve both the assistant and its knowledge sources.
Objective
Develop and evaluate an automated approach for assessing Developer Assistant responses and generating actionable improvement recommendations. These recommendations should help distinguish documentation gaps from retrieval and agent-pipeline issues, supporting targeted improvements by the relevant teams.
Job description
The thesis worker will:
-
Investigate methods for automated evaluation of AI-generated responses.
-
Analyse conversation logs and available pipeline traces.
-
Develop a prototype that identifies recurring quality issues and proposes evidence-backed actions.
-
Validate the findings with experts and assess whether selected recommendations improve response quality.
Education/program/focus
Indicate education, program or focus: Master’s students in Computer Science, Software Engineering, Artificial Intelligence, Data Science, Engineering Physics or a related field.
Number of students: 1 to 2
Start date for the thesis work: Jan-Feb 2027
Estimated time required: 6 months
Contact persons and supervisors
Kazi Mehedi Hasan, supervisor and fullstack developer in Traton Developer Portal team kazi.mehedi.hasan@scania.com
Schnell Melander Vilja, supervisor and industrial PhD student in Research team, vilja.schnell.melander@scania.com
Annergren, Mariette, recruiting manager, mariette.annergren@scania.com
Application:
Your application must include a CV, personal letter and transcript of grades
A background check might be conducted for this position. We are conducting interviews continuously and may close the recruitment earlier than the date specified.
Publication date from - to:
2026-10-26 - 2026-11-20
Södertälje, SE, 151 38
