LLM-Assisted TARA - Review Automating Verification and Quality Assurance in Automotive Cybersecurity Engineering

Abstract

Context: Threat Analysis and Risk Assessment (TARA) is a central activity in automotive cybersecurity engineering. TARA artefacts are used to document cybersecurity risks, threat scenarios, risk treatment decisions, cybersecurity goals, and supporting evidence for compliance-oriented engineering activities. In industrial environments, these artefacts are often updated iteratively as system designs evolve and therefore require repeated review to ensure completeness, consistency, traceability, and documentation quality. Problem: Reviewing completed TARA artefacts is a time-consuming and expert intensive task. Manual sanity checking requires cybersecurity engineers to inspect structured artefacts, verify relationships between elements, identify missing or in consistent information, and assess whether the documentation is suitable for further engineering and compliance activities. This becomes challenging in large-scale development environments where expert availability is limited and TARA information may be distributed across multiple teams. At the same time, fully relying on Large Language Models (LLMs) for review decisions introduces risks related to hallucination, inconsistent outputs, and limited auditability. Method: This thesis follows a Design Science Research approach and presents a hybrid LLM-assisted TARA review prototype. The prototype processes structured XSAM/XML artefacts and combines deterministic checklist-based validation with an AI/RAG-based explanation layer. The deterministic component performs repeatable rule-based checks for machine-checkable aspects such as completeness and traceability. The AI/RAG component uses retrieved domain knowledge, parsed XML evidence, and deterministic findings to support semantic review tasks, including consistency-oriented observations, compliance-oriented explanation, and re viewer support. The system was evaluated through an iterative expert-driven process using representative XSAM/TARA artefacts and predefined sanity-check rules. Result: The evaluation indicates that the hybrid approach can support TARA sanity checking by producing structured review reports, improving consistency of rule-based validation, and providing grounded explanations for expert reviewers. Across refinement stages, agreement with expert assessment increased from 51.9% to 74.1%, while verdict accuracy improved from 70.4% to 88.9%. The results show that deterministic validation is suitable for repeatable structural checks, while the iv LLM/RAG layer is useful for explanation and interpretation. However, the study also shows that expert judgement remains necessary, especially for review criteria requiring project-specific context, architectural information, or evidence not represented in the analysed artefacts. Overall, the thesis demonstrates the feasibility of using a hybrid deterministic and LLM-assisted approach as a decision-support mechanism for reviewing completed TARA artefacts in automotive cybersecurity engineering.