An empirical comparison of two safe regression test selection techniques

Phyllis Frankl, G. Rothermel, K. Sayre, F. I. Vokolos

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Regression test selection techniques reduce the cost of regression testing by selecting a subset of an existing test suite to use in retesting a modified program. Safe regression test selection techniques guarantee (under specific conditions) that the selected subset will not omit faults that could have been revealed by the entire suite. Many regression test selection techniques have been described in the literature. Empirical studies of some of these techniques have shown that they can be beneficial, but only a few studies have empirically compared different techniques, and fewer still have considered safe techniques. In this paper, we report the results of a comparative empirical study of implementations of two safe regression test selection techniques: DejaVu and Pytia. Our results show that, despite differences in their approaches, and despite the theoretically greater ability of DejaVu to select smaller test suites than Pythia, the two techniques often selected equivalent test suites in practice, at comparable costs. These results suggest that factors such as ease of implementation, generality, and availability of supporting tools and data may play a greater role than cost-effectiveness for practitioners choosing between these techniques.

    Original languageEnglish (US)
    Title of host publicationProceedings - 2003 International Symposium on Empirical Software Engineering, ISESE 2003
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages195-204
    Number of pages10
    ISBN (Print)0769520022, 9780769520025
    DOIs
    StatePublished - 2003
    Event2003 International Symposium on Empirical Software Engineering, ISESE 2003 - Rome, Italy
    Duration: Sep 30 2003Oct 1 2003

    Other

    Other2003 International Symposium on Empirical Software Engineering, ISESE 2003
    CountryItaly
    CityRome
    Period9/30/0310/1/03

    Fingerprint

    Cost effectiveness
    Costs
    Availability
    Testing

    Keywords

    • Computer science
    • Costs
    • Fault detection
    • Information science
    • Performance evaluation
    • Safety
    • Software engineering
    • Software testing
    • Software tools

    ASJC Scopus subject areas

    • Software

    Cite this

    Frankl, P., Rothermel, G., Sayre, K., & Vokolos, F. I. (2003). An empirical comparison of two safe regression test selection techniques. In Proceedings - 2003 International Symposium on Empirical Software Engineering, ISESE 2003 (pp. 195-204). [1237978] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISESE.2003.1237978

    An empirical comparison of two safe regression test selection techniques. / Frankl, Phyllis; Rothermel, G.; Sayre, K.; Vokolos, F. I.

    Proceedings - 2003 International Symposium on Empirical Software Engineering, ISESE 2003. Institute of Electrical and Electronics Engineers Inc., 2003. p. 195-204 1237978.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Frankl, P, Rothermel, G, Sayre, K & Vokolos, FI 2003, An empirical comparison of two safe regression test selection techniques. in Proceedings - 2003 International Symposium on Empirical Software Engineering, ISESE 2003., 1237978, Institute of Electrical and Electronics Engineers Inc., pp. 195-204, 2003 International Symposium on Empirical Software Engineering, ISESE 2003, Rome, Italy, 9/30/03. https://doi.org/10.1109/ISESE.2003.1237978
    Frankl P, Rothermel G, Sayre K, Vokolos FI. An empirical comparison of two safe regression test selection techniques. In Proceedings - 2003 International Symposium on Empirical Software Engineering, ISESE 2003. Institute of Electrical and Electronics Engineers Inc. 2003. p. 195-204. 1237978 https://doi.org/10.1109/ISESE.2003.1237978
    Frankl, Phyllis ; Rothermel, G. ; Sayre, K. ; Vokolos, F. I. / An empirical comparison of two safe regression test selection techniques. Proceedings - 2003 International Symposium on Empirical Software Engineering, ISESE 2003. Institute of Electrical and Electronics Engineers Inc., 2003. pp. 195-204
    @inproceedings{eec32a9377b5480ba1449ca78ceb5354,
    title = "An empirical comparison of two safe regression test selection techniques",
    abstract = "Regression test selection techniques reduce the cost of regression testing by selecting a subset of an existing test suite to use in retesting a modified program. Safe regression test selection techniques guarantee (under specific conditions) that the selected subset will not omit faults that could have been revealed by the entire suite. Many regression test selection techniques have been described in the literature. Empirical studies of some of these techniques have shown that they can be beneficial, but only a few studies have empirically compared different techniques, and fewer still have considered safe techniques. In this paper, we report the results of a comparative empirical study of implementations of two safe regression test selection techniques: DejaVu and Pytia. Our results show that, despite differences in their approaches, and despite the theoretically greater ability of DejaVu to select smaller test suites than Pythia, the two techniques often selected equivalent test suites in practice, at comparable costs. These results suggest that factors such as ease of implementation, generality, and availability of supporting tools and data may play a greater role than cost-effectiveness for practitioners choosing between these techniques.",
    keywords = "Computer science, Costs, Fault detection, Information science, Performance evaluation, Safety, Software engineering, Software testing, Software tools",
    author = "Phyllis Frankl and G. Rothermel and K. Sayre and Vokolos, {F. I.}",
    year = "2003",
    doi = "10.1109/ISESE.2003.1237978",
    language = "English (US)",
    isbn = "0769520022",
    pages = "195--204",
    booktitle = "Proceedings - 2003 International Symposium on Empirical Software Engineering, ISESE 2003",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - An empirical comparison of two safe regression test selection techniques

    AU - Frankl, Phyllis

    AU - Rothermel, G.

    AU - Sayre, K.

    AU - Vokolos, F. I.

    PY - 2003

    Y1 - 2003

    N2 - Regression test selection techniques reduce the cost of regression testing by selecting a subset of an existing test suite to use in retesting a modified program. Safe regression test selection techniques guarantee (under specific conditions) that the selected subset will not omit faults that could have been revealed by the entire suite. Many regression test selection techniques have been described in the literature. Empirical studies of some of these techniques have shown that they can be beneficial, but only a few studies have empirically compared different techniques, and fewer still have considered safe techniques. In this paper, we report the results of a comparative empirical study of implementations of two safe regression test selection techniques: DejaVu and Pytia. Our results show that, despite differences in their approaches, and despite the theoretically greater ability of DejaVu to select smaller test suites than Pythia, the two techniques often selected equivalent test suites in practice, at comparable costs. These results suggest that factors such as ease of implementation, generality, and availability of supporting tools and data may play a greater role than cost-effectiveness for practitioners choosing between these techniques.

    AB - Regression test selection techniques reduce the cost of regression testing by selecting a subset of an existing test suite to use in retesting a modified program. Safe regression test selection techniques guarantee (under specific conditions) that the selected subset will not omit faults that could have been revealed by the entire suite. Many regression test selection techniques have been described in the literature. Empirical studies of some of these techniques have shown that they can be beneficial, but only a few studies have empirically compared different techniques, and fewer still have considered safe techniques. In this paper, we report the results of a comparative empirical study of implementations of two safe regression test selection techniques: DejaVu and Pytia. Our results show that, despite differences in their approaches, and despite the theoretically greater ability of DejaVu to select smaller test suites than Pythia, the two techniques often selected equivalent test suites in practice, at comparable costs. These results suggest that factors such as ease of implementation, generality, and availability of supporting tools and data may play a greater role than cost-effectiveness for practitioners choosing between these techniques.

    KW - Computer science

    KW - Costs

    KW - Fault detection

    KW - Information science

    KW - Performance evaluation

    KW - Safety

    KW - Software engineering

    KW - Software testing

    KW - Software tools

    UR - http://www.scopus.com/inward/record.url?scp=18044368551&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=18044368551&partnerID=8YFLogxK

    U2 - 10.1109/ISESE.2003.1237978

    DO - 10.1109/ISESE.2003.1237978

    M3 - Conference contribution

    SN - 0769520022

    SN - 9780769520025

    SP - 195

    EP - 204

    BT - Proceedings - 2003 International Symposium on Empirical Software Engineering, ISESE 2003

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -