State of Test Optimization for Variability in Industry

This paper presents an exploratory survey on test optimization practices for variant-intensive systems in the industry, focusing on understanding current challenges and prospects. Testing variant-intensive systems, particularly in domains like automotive, embedded systems, and telecom, involves considerable complexity due to the high number of configurations and variability in hardware and software. The survey responses reveal varying adoption of test categorization, prioritization, and selection techniques across different domains, with some employing feature-based testing approaches and others relying on more ad-hoc methods. Challenges identified include difficulties with change impact analysis, the scalability of test optimization in systems with many configurations, and the ineffective reuse of test cases across product variants. The study highlights the need for automated tools to support test categorization, prioritization, and coverage-based test selection to address these challenges. The findings highlight the importance of developing more scalable solutions tailored to the needs of specific domains for optimizing tests in variant-intensive systems.