Exploring LLM-Powered Software Testing: A Study on Multi-Agent Systems for REST APIs

Abstract

In this thesis, we explore the integration of Large Language Models (LLMs) with Multi-Agent Systems (MAS) to enhance the process of REST API testing. By leveraging the contextual understanding of LLMs, this research aims to develop a framework, which starts by enriching the original API specification, generates test cases and analyzes their results, offering an LLM-powered automated solution for API testing. We begin by examining the current industry practices in API testing, identifying the extent of automation and the specific difficulties faced by developers. Subsequently, we implement and evaluate the enhanced specification across three different API specifications. Results indicate that while LLMs show promise in automating parts of the testing process, there are limitations, particularly in handling the extended length of API specifications. Despite these challenges, the integration of LLM agents in the testing process shows significant potential for reducing manual effort and strengthening software quality assurance.