Requirements Elicitation From User Feedback Using Real-Time Conversational AI

Abstract

After releasing software to public use, the subsequent requirements elicitation phase becomes a critical aspect of the software’s evolution and longevity. User feedback, in turn, becomes a vital source for this process because new requirements can be elicited from this feedback leading to the addition of new functionalities in subsequent releases. This process significantly influences the development process and shapes the form of the software. User feedback can be collected using different approaches like social media networks, application marketplaces, support hotlines and custom feedback forms. However, those approaches often impose challenges, including feedback incompleteness and unclarity leading to additional communication between the development team and end-users making this process time-intensive and causing release delays. Utilizing Design Science Research (DSR), this study investigates the design process and evaluation of a conversational AI chatbot to collect user feedback during a real-time chat conversation to elicit end-user’s requirements. This conversational AI chatbot, developed using the RASA platform, enables real-time interaction with end-users to elicit their requirement. This study is structured as a twocycle development process and evaluation phases of DSR. At the end of the first cycle, we applied a semi-structured interview to discover the chatbot’s downsides. After the second development cycle, we employed an experiment to evaluate the chatbot’s effectiveness in terms of elicited requirements completeness and understandability. The results of the experimental comparison show that the completeness of the elicited requirement is higher than in the filling out standard feedback form approach, however, there is no difference in the understandability of the elicited requirement. The chatbot got positive feedback from primary end-users.