Feature Model Inspection: Leveraging Defect Patterns and AI Assistance

Abstract

Feature models (FM) are vital for representing variability in software product lines (PL), making defect analysis critical. Manual inspections are essential for finding issues tied to domain understanding, requirements organization, and defects beyond syntactic checks. Checklist-based inspections are systematic, reproducible, and less reliant on inspector expertise. Existing methods for FM inspections typically rely on requirements-based defect classifications or easily recalled defect types, often referencing supplementary models or documents. These methods overlook non-obvious or easily forgotten defect patterns. Additionally, they lack accessibility in resource-constrained settings, with dependence on external artifacts further complicating inspections. This study proposes an inspection approach independent of external models, integrating defect classifications from literature with defect patterns. The method focuses on behavioral features and employs large language models (LLMs) to enhance inspection efficiency and accuracy. Central to this approach is a hierarchical defect classification tree, empirically derived. Empirical results for our approach indicate that 23% of defect patterns were non-obvious, while 23% were easy to forget. Nonobvious patterns helped detect 60% of ambiguity defects, 14.5% of business rulerelated defects, and 11.7% of incompleteness defects. LLMs contributed to 15.5%–59.4% of total defects identified. Our approach is accessible to resourcelimited teams. Our approach is suited for agile environments, early design stages, education, and complex industries, bridging theory and practice while establishing AI-assisted inspection as a valuable tool.