Requirements Smells in Prompts (ReSPro)

• Applicant: Professor Dr. Andreas Vogelsang
• Subject Area: Software Engineering and Programming Languages
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 566352773

The “Requirements Smells in Prompts (ReSPro)” project investigates how linguistic issues in natural language (NL) requirements affect the performance of large language models (LLMs) in software engineering tasks. As developers increasingly rely on LLMs for activities such as code generation, test case derivation, and requirements traceability, the project seeks to understand how requirements smells—ambiguities, inconsistencies, or vagueness—impact AI-generated software artifacts. The research follows two key objectives: first, analyzing the impact of requirements smells on different software engineering tasks, and second, developing tools to detect and mitigate these issues. The project will examine LLM performance in four tasks: code generation, test case generation, automated traceability, and model generation from textual requirements. By conducting controlled experiments with datasets containing both high-quality and smelly requirements, the researchers will evaluate the extent to which LLM-generated artifacts are affected by unclear or inconsistent input. Different prompting strategies, including zero-shot, few-shot, and chain-of-thought approaches, will be tested with multiple LLMs, such as GPT, Llama, Claude, and Mistral. In the second phase, the project will develop practical tools to support engineers in improving prompt quality. A static checker will highlight potential issues, an interactive support bot will assist users in refining unclear requirements, and an automated prompt tuning system will adjust prompts before they are sent to an LLM. These tools aim to reduce errors and inconsistencies in AI-generated outputs, making LLMs more reliable for software engineering applications. ReSPro is expected to contribute significantly to AI-assisted software development by establishing best practices for prompt engineering and improving the clarity of requirements used in LLMs. By addressing the impact of linguistic quality on AI-generated software artifacts, the project will help software engineers produce more accurate and reliable results when using generative AI. The study will span three years, with an initial focus on understanding the problem, followed by the development of solutions to enhance AI-driven automation in software engineering.

• DFG Programme: Research Grants