Reproducible research in signal processing: What, why, and how

Patrick Vandewalle, Jelena Kovacevic, Martin Vetterli

Research output: Contribution to journalArticle

Abstract

Have you ever tried to reproduce the results presented in a research paper? For many of our current publications, this would unfortunately be a challenging task. For a computational algorithm, details such as the exact data set, initialization or termination procedures, and precise parameter values are often omitted in the publication for various reasons, such as a lack of space, a lack of self-discipline, or an apparent lack of interest to the readers, to name a few. This makes it difficult, if not impossible, for someone else to obtain the same results. In our experience, it is often even worse as even we are not always able to reproduce our own experiments, making it difficult to answer questions from colleagues about details. Following are some examples of e-mails we have received: "I just read your paper X. It is very completely described, however I am confused by Y. Could you provide the implementation code to me for reference if possible?" "Hi! I am also working on a project related to X. I have implemented your algorithm but cannot get the same results as described in your paper. Which values should I use for parameters Y and Z?".

Original languageEnglish (US)
Pages (from-to)37-47
Number of pages11
JournalIEEE Signal Processing Magazine
Volume26
Issue number3
DOIs
StatePublished - Jan 1 2009

Fingerprint

Signal Processing
Signal processing
Computational Algorithm
Electronic Mail
Initialization
Termination
Experiments
Experiment
Experience

Keywords

  • Book reviews
  • Data mining
  • Databases
  • MATLAB
  • Probability density function
  • Signal processing
  • Signal processing algorithms

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering
  • Applied Mathematics

Cite this

Reproducible research in signal processing : What, why, and how. / Vandewalle, Patrick; Kovacevic, Jelena; Vetterli, Martin.

In: IEEE Signal Processing Magazine, Vol. 26, No. 3, 01.01.2009, p. 37-47.

Research output: Contribution to journalArticle

Vandewalle, Patrick ; Kovacevic, Jelena ; Vetterli, Martin. / Reproducible research in signal processing : What, why, and how. In: IEEE Signal Processing Magazine. 2009 ; Vol. 26, No. 3. pp. 37-47.
@article{fc88b7e1e13e4651a9161dcd6c41a922,
title = "Reproducible research in signal processing: What, why, and how",
abstract = "Have you ever tried to reproduce the results presented in a research paper? For many of our current publications, this would unfortunately be a challenging task. For a computational algorithm, details such as the exact data set, initialization or termination procedures, and precise parameter values are often omitted in the publication for various reasons, such as a lack of space, a lack of self-discipline, or an apparent lack of interest to the readers, to name a few. This makes it difficult, if not impossible, for someone else to obtain the same results. In our experience, it is often even worse as even we are not always able to reproduce our own experiments, making it difficult to answer questions from colleagues about details. Following are some examples of e-mails we have received: {"}I just read your paper X. It is very completely described, however I am confused by Y. Could you provide the implementation code to me for reference if possible?{"} {"}Hi! I am also working on a project related to X. I have implemented your algorithm but cannot get the same results as described in your paper. Which values should I use for parameters Y and Z?{"}.",
keywords = "Book reviews, Data mining, Databases, MATLAB, Probability density function, Signal processing, Signal processing algorithms",
author = "Patrick Vandewalle and Jelena Kovacevic and Martin Vetterli",
year = "2009",
month = "1",
day = "1",
doi = "10.1109/MSP.2009.932122",
language = "English (US)",
volume = "26",
pages = "37--47",
journal = "IEEE Signal Processing Magazine",
issn = "1053-5888",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "3",

}

TY - JOUR

T1 - Reproducible research in signal processing

T2 - What, why, and how

AU - Vandewalle, Patrick

AU - Kovacevic, Jelena

AU - Vetterli, Martin

PY - 2009/1/1

Y1 - 2009/1/1

N2 - Have you ever tried to reproduce the results presented in a research paper? For many of our current publications, this would unfortunately be a challenging task. For a computational algorithm, details such as the exact data set, initialization or termination procedures, and precise parameter values are often omitted in the publication for various reasons, such as a lack of space, a lack of self-discipline, or an apparent lack of interest to the readers, to name a few. This makes it difficult, if not impossible, for someone else to obtain the same results. In our experience, it is often even worse as even we are not always able to reproduce our own experiments, making it difficult to answer questions from colleagues about details. Following are some examples of e-mails we have received: "I just read your paper X. It is very completely described, however I am confused by Y. Could you provide the implementation code to me for reference if possible?" "Hi! I am also working on a project related to X. I have implemented your algorithm but cannot get the same results as described in your paper. Which values should I use for parameters Y and Z?".

AB - Have you ever tried to reproduce the results presented in a research paper? For many of our current publications, this would unfortunately be a challenging task. For a computational algorithm, details such as the exact data set, initialization or termination procedures, and precise parameter values are often omitted in the publication for various reasons, such as a lack of space, a lack of self-discipline, or an apparent lack of interest to the readers, to name a few. This makes it difficult, if not impossible, for someone else to obtain the same results. In our experience, it is often even worse as even we are not always able to reproduce our own experiments, making it difficult to answer questions from colleagues about details. Following are some examples of e-mails we have received: "I just read your paper X. It is very completely described, however I am confused by Y. Could you provide the implementation code to me for reference if possible?" "Hi! I am also working on a project related to X. I have implemented your algorithm but cannot get the same results as described in your paper. Which values should I use for parameters Y and Z?".

KW - Book reviews

KW - Data mining

KW - Databases

KW - MATLAB

KW - Probability density function

KW - Signal processing

KW - Signal processing algorithms

UR - http://www.scopus.com/inward/record.url?scp=85032750960&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85032750960&partnerID=8YFLogxK

U2 - 10.1109/MSP.2009.932122

DO - 10.1109/MSP.2009.932122

M3 - Article

AN - SCOPUS:85032750960

VL - 26

SP - 37

EP - 47

JO - IEEE Signal Processing Magazine

JF - IEEE Signal Processing Magazine

SN - 1053-5888

IS - 3

ER -