Combining screening and metamodel-based methods

An efficient sequential approach for the sensitivity analysis of model outputs

Qiao Ge, Biagio Ciuffo, Monica Menendez

    Research output: Contribution to journalArticle

    Abstract

    Sensitivity analysis (SA) is able to identify the most influential parameters of a given model. Application of SA is usually critical for reducing the complexity in the subsequent model calibration and use. Unfortunately it is hardly applied, especially when the model is in the form of a computationally expensive black-box computer program. A possible solution concerns applying SA to the metamodel (i.e., an approximation of the computationally expensive model) instead. Among the other options, the use of Gaussian process metamodels (also known as Kriging metamodels) has been recently proposed for the SA of computationally expensive traffic simulation models. However, the main limitation of this approach is its dependence on the model dimensionality. When the model is high-dimensional, the estimation of the Kriging metamodel may still be problematic due to its high computational cost. In order to overcome this problem, in the present paper, the Kriging-based approach has been combined with the quasi-optimized trajectory based elementary effects (quasi-OTEE) approach for the SA of high-dimensional models. The quasi-OTEE SA is used first to screen the influential and non-influential parameters of a high-dimensional model; then the Kriging-based SA is used to calculate the variance-based sensitivity indices, and to rank the most influential parameters in a more accurate way. The application of the proposed sequential SA is illustrated with several numerical experiments. Results show that the method can properly identify the most influential parameters and their ranks, while the number of model evaluations is considerably less than the variance-based SA (e.g., in one of the tests the sequential SA requires over 50 times less model evaluations than the variance-based SA).

    Original languageEnglish (US)
    Pages (from-to)334-344
    Number of pages11
    JournalReliability Engineering and System Safety
    Volume134
    DOIs
    StatePublished - Jan 1 2015

    Fingerprint

    Metamodel
    Sensitivity analysis
    Screening
    Sensitivity Analysis
    Output
    Kriging
    Sequential Analysis
    Model
    Model Evaluation
    High-dimensional
    Trajectories
    Trajectory
    Traffic Simulation
    Model Calibration
    Traffic Model
    Black Box
    Gaussian Process
    Dimensionality
    Computer program listings
    Computational Cost

    Keywords

    • High-dimensional and computationally expensive model
    • Metamodel
    • Screening
    • Sensitivity analysis
    • Variance-based approach

    ASJC Scopus subject areas

    • Safety, Risk, Reliability and Quality
    • Industrial and Manufacturing Engineering
    • Applied Mathematics

    Cite this

    Combining screening and metamodel-based methods : An efficient sequential approach for the sensitivity analysis of model outputs. / Ge, Qiao; Ciuffo, Biagio; Menendez, Monica.

    In: Reliability Engineering and System Safety, Vol. 134, 01.01.2015, p. 334-344.

    Research output: Contribution to journalArticle

    @article{ec1e4b4cf6bc4b88b97d19dc9d9b34cb,
    title = "Combining screening and metamodel-based methods: An efficient sequential approach for the sensitivity analysis of model outputs",
    abstract = "Sensitivity analysis (SA) is able to identify the most influential parameters of a given model. Application of SA is usually critical for reducing the complexity in the subsequent model calibration and use. Unfortunately it is hardly applied, especially when the model is in the form of a computationally expensive black-box computer program. A possible solution concerns applying SA to the metamodel (i.e., an approximation of the computationally expensive model) instead. Among the other options, the use of Gaussian process metamodels (also known as Kriging metamodels) has been recently proposed for the SA of computationally expensive traffic simulation models. However, the main limitation of this approach is its dependence on the model dimensionality. When the model is high-dimensional, the estimation of the Kriging metamodel may still be problematic due to its high computational cost. In order to overcome this problem, in the present paper, the Kriging-based approach has been combined with the quasi-optimized trajectory based elementary effects (quasi-OTEE) approach for the SA of high-dimensional models. The quasi-OTEE SA is used first to screen the influential and non-influential parameters of a high-dimensional model; then the Kriging-based SA is used to calculate the variance-based sensitivity indices, and to rank the most influential parameters in a more accurate way. The application of the proposed sequential SA is illustrated with several numerical experiments. Results show that the method can properly identify the most influential parameters and their ranks, while the number of model evaluations is considerably less than the variance-based SA (e.g., in one of the tests the sequential SA requires over 50 times less model evaluations than the variance-based SA).",
    keywords = "High-dimensional and computationally expensive model, Metamodel, Screening, Sensitivity analysis, Variance-based approach",
    author = "Qiao Ge and Biagio Ciuffo and Monica Menendez",
    year = "2015",
    month = "1",
    day = "1",
    doi = "10.1016/j.ress.2014.08.009",
    language = "English (US)",
    volume = "134",
    pages = "334--344",
    journal = "Reliability Engineering and System Safety",
    issn = "0951-8320",
    publisher = "Elsevier Limited",

    }

    TY - JOUR

    T1 - Combining screening and metamodel-based methods

    T2 - An efficient sequential approach for the sensitivity analysis of model outputs

    AU - Ge, Qiao

    AU - Ciuffo, Biagio

    AU - Menendez, Monica

    PY - 2015/1/1

    Y1 - 2015/1/1

    N2 - Sensitivity analysis (SA) is able to identify the most influential parameters of a given model. Application of SA is usually critical for reducing the complexity in the subsequent model calibration and use. Unfortunately it is hardly applied, especially when the model is in the form of a computationally expensive black-box computer program. A possible solution concerns applying SA to the metamodel (i.e., an approximation of the computationally expensive model) instead. Among the other options, the use of Gaussian process metamodels (also known as Kriging metamodels) has been recently proposed for the SA of computationally expensive traffic simulation models. However, the main limitation of this approach is its dependence on the model dimensionality. When the model is high-dimensional, the estimation of the Kriging metamodel may still be problematic due to its high computational cost. In order to overcome this problem, in the present paper, the Kriging-based approach has been combined with the quasi-optimized trajectory based elementary effects (quasi-OTEE) approach for the SA of high-dimensional models. The quasi-OTEE SA is used first to screen the influential and non-influential parameters of a high-dimensional model; then the Kriging-based SA is used to calculate the variance-based sensitivity indices, and to rank the most influential parameters in a more accurate way. The application of the proposed sequential SA is illustrated with several numerical experiments. Results show that the method can properly identify the most influential parameters and their ranks, while the number of model evaluations is considerably less than the variance-based SA (e.g., in one of the tests the sequential SA requires over 50 times less model evaluations than the variance-based SA).

    AB - Sensitivity analysis (SA) is able to identify the most influential parameters of a given model. Application of SA is usually critical for reducing the complexity in the subsequent model calibration and use. Unfortunately it is hardly applied, especially when the model is in the form of a computationally expensive black-box computer program. A possible solution concerns applying SA to the metamodel (i.e., an approximation of the computationally expensive model) instead. Among the other options, the use of Gaussian process metamodels (also known as Kriging metamodels) has been recently proposed for the SA of computationally expensive traffic simulation models. However, the main limitation of this approach is its dependence on the model dimensionality. When the model is high-dimensional, the estimation of the Kriging metamodel may still be problematic due to its high computational cost. In order to overcome this problem, in the present paper, the Kriging-based approach has been combined with the quasi-optimized trajectory based elementary effects (quasi-OTEE) approach for the SA of high-dimensional models. The quasi-OTEE SA is used first to screen the influential and non-influential parameters of a high-dimensional model; then the Kriging-based SA is used to calculate the variance-based sensitivity indices, and to rank the most influential parameters in a more accurate way. The application of the proposed sequential SA is illustrated with several numerical experiments. Results show that the method can properly identify the most influential parameters and their ranks, while the number of model evaluations is considerably less than the variance-based SA (e.g., in one of the tests the sequential SA requires over 50 times less model evaluations than the variance-based SA).

    KW - High-dimensional and computationally expensive model

    KW - Metamodel

    KW - Screening

    KW - Sensitivity analysis

    KW - Variance-based approach

    UR - http://www.scopus.com/inward/record.url?scp=84913529830&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84913529830&partnerID=8YFLogxK

    U2 - 10.1016/j.ress.2014.08.009

    DO - 10.1016/j.ress.2014.08.009

    M3 - Article

    VL - 134

    SP - 334

    EP - 344

    JO - Reliability Engineering and System Safety

    JF - Reliability Engineering and System Safety

    SN - 0951-8320

    ER -