Reproducible Research

Reproducible Research from the Trenches

I am involved in promoting Reproducible Research in Software Engineering together with Prof. Barbara Kitchenham. Some of my invited talks and seminars concerning this topic:

1) Reproducible Research from the Trenches – How I do reproducible research by example, Invited Talk at CREST/SSE group headed by Prof. Mark Harman, UCL, London, UK, June 25, 2014.

2) Reproducible Research from the Trenches, Invited Talk at School of Computing and Mathematics, Software Engineering Group (Prof. Pearl Brereton and Prof. Barbara Kitchenham), Keele University, UK, July 2, 2014.

My Reproducible Research papers

Readers of my papers should note that some of my papers (especially recent ones) have been produced in accordance with the principles of reproducible research and that data sets I use are available from my R package ‘reproducer’ available from CRAN:

  • Barbara A. Kitchenham, Lech Madeyski, David Budgen, Jacky Keung, Pearl Brereton, Stuart Charters, Shirley Gibbs, and Amnart Pohthong, “Robust Statistical Methods for Empirical Software Engineering,” Empirical Software Engineering, 2016 (in press). DOI: 10.1007/s10664-016-9437-5.
@article{KitchenhamMadeyski17ESE,
 Author = {Barbara A. Kitchenham and Lech Madeyski and David Budgen and Jacky Keung and Pearl Brereton and Stuart Charters and Shirley Gibbs and Amnart Pohthong},
 Doi = {10.1007/s10664-016-9437-5},
 Journal = {Empirical Software Engineering},
 Title = {{Robust Statistical Methods for Empirical Software Engineering}},
 Volume = {22}, Number = {2}, Pages = {579--630},
 Year = {2017}}
@article{Madeyski15SQJ,
 Author = {Lech Madeyski and Marian Jureczko},
 Doi = {10.1007/s11219-014-9241-7},
 Journal = {Software Quality Journal},
 Number = {3},
 Pages = {393--422},
 Title = {{Which Process Metrics Can Significantly Improve Defect Prediction Models? An Empirical Study}},
 Volume = {23},
 Year = {2015}}
  • Marian Jureczko and Lech Madeyski, “Cross–project defect prediction with respect to code ownership model: An empirical study”, e-Informatica Software Engineering Journal, vol. 9, no. 1, p. 21-35, 2015. DOI: 10.5277/e-Inf150102
@article{JureczkoMadeyski15,
 Author = {Marian Jureczko and Lech Madeyski},
 Doi = {10.5277/e-Inf150102},
 Journal = {{e-Informatica Software Engineering Journal}},
 Number = {1},
 Pages = {21--35},
 Title = {Cross--Project Defect Prediction With Respect To Code Ownership Model: An Empirical Study},
 Volume = {9},
 Year = {2015}}

My R package – ‘reproducer’

The R package ‘reproducer’ includes data analysis functions and data sets (e.g., related to software defect prediction) to streamline reproducible research in software engineering.

The package is avavilable from CRAN: http://cran.r-project.org/web/packages/reproducer/index.html

Lech Madeyski, reproducer: Reproduce Statistical Analyses and Meta-Analyses, 2023. R package version 0.5.2 (http://CRAN.R-project.org/package=reproducer).
@manual{R-reproducer,
 Author = {Lech Madeyski and Barbara Kitchenham},
 Note = {{R} package version 0.5.2 (http://CRAN.R-project.org/package=reproducer)},
 Title = {{reproducer: Reproduce Statistical Analyses and Meta-Analyses}},
 Url = {https://madeyski.e-informatyka.pl/reproducible-research/},
 Year = {2023}}
To cite the ‘reproducer’ R package in publications please use:

1) Barbara A. Kitchenham, Lech Madeyski, David Budgen, Jacky Keung, Pearl Brereton, Stuart Charters, Shirley Gibbs, and Amnart Pohthong, “Robust Statistical Methods for Empirical Software Engineering,” Empirical Software Engineering, vol. 22, no.2, p. 579-630, 2017. DOI: 10.1007/s10664-016-9437-5.

URL: http://dx.doi.org/10.1007/s10664-016-9437-5 or https://madeyski.e-informatyka.pl/download/KitchenhamMadeyski17ESE.pdf or e-Offprinthttps://madeyski.e-informatyka.pl/download/KitchenhamMadeyski17ESEeOffprint.pdf

@article{KitchenhamMadeyski17ESE,
 Author = {Barbara A. Kitchenham and Lech Madeyski and David Budgen and Jacky Keung and Pearl Brereton and Stuart Charters and Shirley Gibbs and Amnart Pohthong},
 Doi = {10.1007/s10664-016-9437-5},
 Journal = {Empirical Software Engineering},
 Title = {{Robust Statistical Methods for Empirical Software Engineering}},
 Volume = {22}, Number = {2}, Pages = {579--630},
 Year = {2017}}

2) Lech Madeyski and Marian Jureczko, “Which Process Metrics Can Significantly Improve Defect Prediction Models? An Empirical Study“, Software Quality Journal, vol. 23, iss. 3, pp. 393-422, September 2015, Springer. DOI: 10.1007/s11219-014-9241-7

@article{Madeyski15SQJ,
 Author = {Lech Madeyski and Marian Jureczko},
 Doi = {10.1007/s11219-014-9241-7},
 Journal = {Software Quality Journal},
 Number = {3},
 Pages = {393--422},
 Title = {{Which Process Metrics Can Significantly Improve Defect Prediction Models? An Empirical Study}},
 Volume = {23},
 Year = {2015}}

3) Marian Jureczko and Lech Madeyski, “Cross-project defect prediction with respect to code ownership model: An empirical study”, e-Informatica Software Engineering Journal, vol. 9, no. 1, p. 21-35, 2015. DOI: 10.5277/e-Inf150102

@article{JureczkoMadeyski15,
 Author = {Marian Jureczko and Lech Madeyski},
 Doi = {10.5277/e-Inf150102},
 Journal = {{e-Informatica Software Engineering Journal}},
 Number = {1},
 Pages = {21--35},
 Title = {Cross--Project Defect Prediction With Respect To Code Ownership Model: An Empirical Study},
 Volume = {9},
 Year = {2015}}

Reproducible Research on Code Smells prediction

The most up-to-date list of code samples with smells (validated by developers) is available using the data set described in the following paper:

Lech Madeyski and Tomasz Lewowski. 2020. MLCQ: Industry-relevant code smell data set. In Evaluation and Assessment in Software Engineering (EASE2020), April 15–17, 2020, Trondheim, Norway.ACM, New York, NY, USA, 6 pages, DOI: 10.1145/3383219.3383264 URL: https://doi.org/10.1145/3383219.3383264

Stay tuned for further updates.