Adaptive Testing of Large Language Model–Enhanced Software Systems: A Comprehensive Framework for Requirements, Test-Case Generation, Prioritization, and Evaluation

Abstract

Background: The rise of large language models (LLMs) and their integration into software development toolchains has introduced new dimensions to software testing, from automated test-case generation to system-level validation, while simultaneously complicating requirements testing and configurable-system evaluation (Wang, 2024; dos Santos, 2020). Existing literature on software product lines, configurable systems, and regression test prioritization offers foundational methods that must be reinterpreted when LLMs participate as both test artifact producers and application components (Agh, 2024; Souto, 2017; Elbaum, 2002).

Objective: This article proposes a unified, publication-ready theoretical and methodological framework for testing LLM-enhanced software systems that spans requirements elicitation and testing, automated unit and integration test generation using LLMs, focal-method mapping, test-case selection and prioritization, and empirical evaluation strategies. The framework emphasizes balancing soundness and efficiency in configurable environments while employing machine learning–based prioritization and leveraging recent advances in LLM tool-use and web-agent architectures (Pan, 2022; Tufano, 2022; Schick et al., 2023).

Methods: We synthesize evidence from systematic literature reviews, empirical studies, and recent preprints to build a layered methodology: (1) requirements-level formalization and traceable test intent extraction; (2) LLM-driven test-case generation templates and focal-method mapping; (3) hybrid selection and prioritization using feature-aware ML models and historical regression data; (4) orchestration for end-to-end testing and evaluation in realistic web environments; and (5) continuous monitoring and adaptive re-prioritization. Each component is described with prescriptive guidelines and evaluative metrics. Literature-based rationale and thought experiments ground the proposed choices (dos Santos, 2020; Wang, 2024; Chandra, 2025).

Results: The conceptual framework yields measurable improvements along three axes in thought experiments and empirical analogues discussed herein: coverage of requirement-derived behaviors, fault-revealing power of generated test suites, and regression test efficiency under budget constraints. Drawing on methods from test-case prioritization research and focal-method mapping, we show how LLM-generated tests can be filtered and ranked to achieve higher early-fault detection compared to naïve generation, while addressing configurable-system explosion through sampling and soundness-efficiency trade-offs (Elbaum, 2000; He, 2024; Souto, 2017).

Conclusions: LLMs offer unprecedented capabilities for automating parts of the testing lifecycle, but their effective integration requires principled pipelines that combine requirements engineering, focal-method guidance, adaptive prioritization, and environment realism. The proposed framework provides a roadmap for researchers and practitioners to construct, evaluate, and iterate robust testing systems for contemporary software that integrates LLMs either as tooling or as functional components. Future work should empirically validate the framework across diverse domains, quantify human–LLM collaboration dynamics in testing, and extend the approach to continual learning settings.