You are required to read and agree to the below before accessing a full-text version of an article in the IDE article repository.

The full-text document you are about to access is subject to national and international copyright laws. In most cases (but not necessarily all) the consequence is that personal use is allowed given that the copyright owner is duly acknowledged and respected. All other use (typically) require an explicit permission (often in writing) by the copyright owner.

For the reports in this repository we specifically note that

  • the use of articles under IEEE copyright is governed by the IEEE copyright policy (available at http://www.ieee.org/web/publications/rights/copyrightpolicy.html)
  • the use of articles under ACM copyright is governed by the ACM copyright policy (available at http://www.acm.org/pubs/copyright_policy/)
  • technical reports and other articles issued by M‰lardalen University is free for personal use. For other use, the explicit consent of the authors is required
  • in other cases, please contact the copyright owner for detailed information

By accepting I agree to acknowledge and respect the rights of the copyright owner of the document I am about to access.

If you are in doubt, feel free to contact webmaster@ide.mdh.se

Software Test Results Exploration and Visualization with Continuous Integration and Nightly Testing

Fulltext:


Publication Type:

Journal article

Venue:

International Journal on Software Tools for Technology Transfer


Abstract

Software testing is key for quality assurance of embedded systems. However, with increased development pace, the amount of test results data risks growing to a level where exploration and visualization of the results are unmanageable. This paper covers a tool, Tim, implemented at a company developing embedded systems, where software development occurs in parallel branches and nightly testing is partitioned over software branches, test systems and test cases. Tim aims to replace a previous solution with problems of scalability, requirements and technological flora. Tim was implemented with a reference group over several months. For validation, data was collected both from reference group meetings and logs from the usage of the tool. Data was analyzed quantitatively and qualitatively. The main contributions from the study include the implementation of eight views for test results exploration and visualization, the identification of four solutions patterns for these views (filtering, aggregation, previews and comparisons), as well as six challenges frequently discussed at reference group meetings (expectations, anomalies, navigation, integrations, hardware details and plots). Results are put in perspective with related work and future work is proposed, e.g. enhanced anomaly detection and integrations with more systems such as risk management, source code and requirements repositories.

Bibtex

@article{Strandberg6369,
author = {Per Erik Strandberg and Wasif Afzal and Daniel Sundmark},
title = {Software Test Results Exploration and Visualization with Continuous Integration and Nightly Testing},
volume = {1},
number = {1},
pages = {1--25},
month = {February},
year = {2022},
journal = { International Journal on Software Tools for Technology Transfer},
url = {http://www.es.mdu.se/publications/6369-}
}