Automated Assessment and Enhancement of User Story Quality Using Large Language Models in Agile Development

Abstract

Low-quality user stories in agile software development often lead to misunderstandings, delays, and costly rework. This paper presents a dual agent framework that leverages large language models (LLMs) to automatically assess and refine user story quality, guided by the INVEST and Quality User Story (QUS) criteria. The first agent emulates a Product Owner (PO), evaluating dimensions such as clarity and testability. The second agent acts as a Quality Analyst (QA), rewriting stories to improve their quality while preserving intent. We evaluated our approach on a curated dataset of 42 real-world user stories, showing that LLMs can reliably detect quality issues and produce significantly enhanced versions. By improving backlog quality and reducing manual review effort, our framework supports agile early-phase development processes.