Empowering Automotive Software Development with LLM-RAG Integration - A study on leveraging the RAG-framework for AUTOSAR and automotive safety standards and specifications

Abstract

The modern automotive industry increasingly relies on complex software systems such as Electronic Control Units that govern essential vehicle functions. The AUTOSAR standard provides a framework for automotive software development, ensuring interoperability, scalability, and compliance with safety regulations. However, adhering to AUTOSAR standards is challenging due to their complexity and the manual effort required. This thesis investigates the potential of integrating the Retrieval Augmented Generation framework with Large Language Models to improve the explainability and usability of AUTOSAR specifications for software developers. The primary objectives are to develop a RAG-based model capable of interpreting queries about AUTOSAR specifications and generating clear, actionable steps for developers, and to evaluate the model’s effectiveness in integrating retrieved context. Our findings indicate that the RAG-framework with advanced techniques improves the contextual relevance and accuracy of generated outputs in the AUTOSAR domain. Specifically, the developed RAG model demonstrates higher levels of detail and precision but sometimes lacks fully actionable guidance. In comparison, GPT-4 and a naive RAG model, while generally accurate, often fail to provide the specificity required for complex software engineering tasks. Additionally, the study evaluates the efficiency of LLMs in synthesizing retrieved-context, noting significant improvements in newer models like GPT-4, while also recognizing ongoing challenges in consistently integrating complex information. Limitations of this study include the focus on the AUTOSAR Classic Platform and the exclusion of graphical data. Future research should expand data extraction to include multi-modal inputs and explore fine-tuning and synthetic dataset generation to further improve model outputs. There is also potential for further research into context integration using more complex datasets to better understand the limitations of the model’s capabilities.