Experimental comparison of the effectiveness of branch testing and data flow testing

Phyllis Frankl, Stewart N. Weiss

    Research output: Contribution to journalArticle

    Abstract

    An experiment comparing the effectiveness of the all-uses and all-edges test data adequacy criteria was performed. The experiment was designed so as to overcome some of the deficiencies of previous software testing experiments. A large number of test sets was randomly generated for each of nine subject programs with subtle errors. For each test set, the percentages of executable edges and definition-use associations covered were measured and it was determined whether the test set exposed an error. Hypothesis testing was used to investigate whether all-uses adequate test sets are more likely to expose errors than are all-edges adequate test sets. All-uses was significantly more effective than all-edges for five of the subjects, and appeared guaranteed to detect the error in four of them. Further analysis showed that in four of these subjects, all-uses adequate test sets were more effective than all-edges adequate test sets of similar size. Logistic regression analysis was used to investigate whether the probability that a test set exposes an error increases as the percentage of definition-use associations or edges covered by it increases. The evidence did not strongly support this conjecture. Error exposing ability was shown to strongly positively correlated to percentage of covered definition-use associations in only four of the nine subjects. Error exposing ability was also shown to be positively correlated to the percentage of covered edges in four (different) subjects, but the relationship was weaker.

    Original languageEnglish (US)
    Pages (from-to)774-787
    Number of pages14
    JournalIEEE Transactions on Software Engineering
    Volume19
    Issue number8
    DOIs
    StatePublished - Aug 1993

    Fingerprint

    Testing
    Software testing
    Experiments
    Regression analysis
    Logistics

    ASJC Scopus subject areas

    • Computer Graphics and Computer-Aided Design
    • Software
    • Electrical and Electronic Engineering

    Cite this

    Experimental comparison of the effectiveness of branch testing and data flow testing. / Frankl, Phyllis; Weiss, Stewart N.

    In: IEEE Transactions on Software Engineering, Vol. 19, No. 8, 08.1993, p. 774-787.

    Research output: Contribution to journalArticle

    @article{3a4d1e7f793b44d2b52439c6b3d69432,
    title = "Experimental comparison of the effectiveness of branch testing and data flow testing",
    abstract = "An experiment comparing the effectiveness of the all-uses and all-edges test data adequacy criteria was performed. The experiment was designed so as to overcome some of the deficiencies of previous software testing experiments. A large number of test sets was randomly generated for each of nine subject programs with subtle errors. For each test set, the percentages of executable edges and definition-use associations covered were measured and it was determined whether the test set exposed an error. Hypothesis testing was used to investigate whether all-uses adequate test sets are more likely to expose errors than are all-edges adequate test sets. All-uses was significantly more effective than all-edges for five of the subjects, and appeared guaranteed to detect the error in four of them. Further analysis showed that in four of these subjects, all-uses adequate test sets were more effective than all-edges adequate test sets of similar size. Logistic regression analysis was used to investigate whether the probability that a test set exposes an error increases as the percentage of definition-use associations or edges covered by it increases. The evidence did not strongly support this conjecture. Error exposing ability was shown to strongly positively correlated to percentage of covered definition-use associations in only four of the nine subjects. Error exposing ability was also shown to be positively correlated to the percentage of covered edges in four (different) subjects, but the relationship was weaker.",
    author = "Phyllis Frankl and Weiss, {Stewart N.}",
    year = "1993",
    month = "8",
    doi = "10.1109/32.238581",
    language = "English (US)",
    volume = "19",
    pages = "774--787",
    journal = "IEEE Transactions on Software Engineering",
    issn = "0098-5589",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    number = "8",

    }

    TY - JOUR

    T1 - Experimental comparison of the effectiveness of branch testing and data flow testing

    AU - Frankl, Phyllis

    AU - Weiss, Stewart N.

    PY - 1993/8

    Y1 - 1993/8

    N2 - An experiment comparing the effectiveness of the all-uses and all-edges test data adequacy criteria was performed. The experiment was designed so as to overcome some of the deficiencies of previous software testing experiments. A large number of test sets was randomly generated for each of nine subject programs with subtle errors. For each test set, the percentages of executable edges and definition-use associations covered were measured and it was determined whether the test set exposed an error. Hypothesis testing was used to investigate whether all-uses adequate test sets are more likely to expose errors than are all-edges adequate test sets. All-uses was significantly more effective than all-edges for five of the subjects, and appeared guaranteed to detect the error in four of them. Further analysis showed that in four of these subjects, all-uses adequate test sets were more effective than all-edges adequate test sets of similar size. Logistic regression analysis was used to investigate whether the probability that a test set exposes an error increases as the percentage of definition-use associations or edges covered by it increases. The evidence did not strongly support this conjecture. Error exposing ability was shown to strongly positively correlated to percentage of covered definition-use associations in only four of the nine subjects. Error exposing ability was also shown to be positively correlated to the percentage of covered edges in four (different) subjects, but the relationship was weaker.

    AB - An experiment comparing the effectiveness of the all-uses and all-edges test data adequacy criteria was performed. The experiment was designed so as to overcome some of the deficiencies of previous software testing experiments. A large number of test sets was randomly generated for each of nine subject programs with subtle errors. For each test set, the percentages of executable edges and definition-use associations covered were measured and it was determined whether the test set exposed an error. Hypothesis testing was used to investigate whether all-uses adequate test sets are more likely to expose errors than are all-edges adequate test sets. All-uses was significantly more effective than all-edges for five of the subjects, and appeared guaranteed to detect the error in four of them. Further analysis showed that in four of these subjects, all-uses adequate test sets were more effective than all-edges adequate test sets of similar size. Logistic regression analysis was used to investigate whether the probability that a test set exposes an error increases as the percentage of definition-use associations or edges covered by it increases. The evidence did not strongly support this conjecture. Error exposing ability was shown to strongly positively correlated to percentage of covered definition-use associations in only four of the nine subjects. Error exposing ability was also shown to be positively correlated to the percentage of covered edges in four (different) subjects, but the relationship was weaker.

    UR - http://www.scopus.com/inward/record.url?scp=0027644148&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=0027644148&partnerID=8YFLogxK

    U2 - 10.1109/32.238581

    DO - 10.1109/32.238581

    M3 - Article

    VL - 19

    SP - 774

    EP - 787

    JO - IEEE Transactions on Software Engineering

    JF - IEEE Transactions on Software Engineering

    SN - 0098-5589

    IS - 8

    ER -