SEGRESS: Software Engineering Guidelines for REporting Secondary Studies

Awesome news once again! Happy to have my 4th IEEE TSE paper (TSE is #1 journal in Software Engineering).

Article “SEGRESS: Software Engineering Guidelines for REporting Secondary Studies” by Barbara Kitchenham, Lech Madeyski, and David Budgen has been accepted by IEEE Transactions on Software Engineering. Preprint: https://doi.org/10.1109/TSE.2022.3174092. Twitter: https://twitter.com/LechMadeyski/status/1524697213641060353

Abstract:

Context: Several tertiary studies have criticized the reporting of software engineering secondary studies.
Objective: Our objective is to identify guidelines for reporting software engineering (SE) secondary studies which would address problems observed in the reporting of software engineering systematic reviews (SRs).
Method: We review the criticisms of SE secondary studies and identify the major areas of concern. We assess the PRISMA 2020 (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement as a possible solution to the need for SR reporting guidelines, based on its status as the reporting guideline recommended by the Cochrane Collaboration whose SR guidelines were a major input to the guidelines developed for SE. We report its advantages and limitations in the context of SE secondary studies. We also assess reporting guidelines for mapping studies and qualitative reviews, and compare their structure and content with that of PRISMA 2020.
Results: Previous tertiary studies confirm that reports of secondary studies are of variable quality. However, \emph{ad hoc} recommendations that amend reporting standards may result in unnecessary duplication of text. We confirm that the PRISMA 2020 statement addresses SE reporting problems, but is mainly oriented to quantitative reviews, mixed-methods reviews and meta-analyses. However, we show that the PRISMA 2020 item definitions can be extended to cover the information needed to report mapping studies and qualitative reviews.
Conclusions: In this paper and its Supplementary Material, we present and illustrate an integrated set of guidelines called SEGRESS (Software Engineering Guidelines for REporting Secondary Studies), suitable for quantitative systematic reviews (building upon PRISMA 2020), mapping studies (PRISMA-ScR), and qualitative reviews (ENTREQ and RAMESES), that addresses reporting problems found in current SE SRs.

The accepted version of SEGRESS Guidelines: https://madeyski.e-informatyka.pl/download/KitchenhamMadeyskiBudgen22TSE-SEGRESS.pdf

Supplementary Material: https://madeyski.e-informatyka.pl/download/SEGRESS22supplement.pdf

 

Kitchenham, Barbara Ann, Madeyski, Lech, and Budgen, David. SEGRESS: Software Engineering Guidelines for REporting Secondary Studies. IEEE Transactions on Software Engineering, DOI: 10.1109/TSE.2022.3174092

Kitchenham, Barbara Ann, Madeyski, Lech, and Budgen, David. SEGRESS: Software Engineering Guidelines for REporting Secondary Studies. IEEE Transactions on Software Engineering, DOI: 10.1109/TSE.2022.3174092

Teaser:

SEGRESS: Software Engineering Guidelines for REporting Secondary Studies

SEGRESS: Software Engineering Guidelines for REporting Secondary Studies

Posted in News | Tagged , , , , , , , , , , , , , , | Comments closed

How should software engineering secondary studies include grey material?

Awesome news! Article “How should software engineering secondary studies include grey material?” by Barbara Kitchenham, Lech Madeyski, David Budgen has been accepted by IEEE Transactions on Software Engineering. Preprint: https://doi.org/10.1109/TSE.2022.3165938. Twitter: https://twitter.com/LechMadeyski/status/1512804644128501763

Happy to have my 3rd IEEE TSE paper (TSE is #1 journal in Software Engineering).

Abstract:

Context: Recent papers have proposed the use of grey literature (GL) and multivocal reviews. These papers have raised issues about the practices used for systematic reviews (SRs) in software engineering (SE) and suggested that there should be changes to the current SR guidelines.
Objective: To investigate whether current SR guidelines need to be changed to support GL and multivocal reviews.
Method: We discuss the definitions of GL and the importance of GL and of industry-based field studies in SE SRs. We identify properties of SRs that constrain the material used in SRs: a) the nature of primary studies b) the requirements of SRs to be auditable, traceable, and reproducible, and explain why these requirements restrict the use of blogs in SRs.
Results: SR guidelines have always considered GL as a possible source of primary studies and never supported excluding field studies that incorporate the practitioners’ viewpoint. However, GL which was meant to refer to documents that were not formally published, is now being extended to information from blogs/tweets/Q&A posts. Thus, it might seem that SRs do not make full use of GL because they do not include such information. However, the unit of analysis in SR is the primary study. Thus, it is not the source but the type of information that is important. Any report describing a rigorous empirical evaluation is a candidate primary study. Whether it is actually included in a SR depends on the SR eligibility criteria. However, any study that cannot be guaranteed to be publicly available in the long term should not be used as a primary study in an SR. This does not prevent such information being aggregated in surveys of social media and used in the context of evidence-based software engineering (EBSE).
Conclusions: Current guidelines for SRs do not require extensions, but their scope needs to be better defined. SE researchers require guidelines for analysing social media posts (e.g., blogs, tweets, vlogs), but these should be based on qualitative primary (not secondary) study guidelines. SE researchers can use mixed-methods SRs and/or the fourth step of EBSE to incorporate findings from social media surveys with those from SRs and to develop industry-relevant recommendations.
Barbara Kitchenham, Lech Madeyski, and David Budgen, “How should software engineering secondary studies include grey material?”, IEEE Transactions on Software Engineering, 2022. DOI: 10.1109/TSE.2022.3165938 URL: https://doi.org/10.1109/TSE.2022.3165938

Barbara Kitchenham, Lech Madeyski, and David Budgen, “How should software engineering secondary studies include grey material?”, IEEE Transactions on Software Engineering, 2022. DOI: 10.1109/TSE.2022.3165938 URL: https://doi.org/10.1109/TSE.2022.3165938

Posted in News | Tagged , , , , , , , , , , , | Comments closed

In the Top 2% of researchers

It is an honour to be in the top 2% of the most influential researchers in the world in all scientific disciplines in 2021. The list by Jeroen Baas (Elsevier), Kevin Boyack (SciTech Strategies) and John P.A. Ioannidis (Stanford University) includes almost 190,000 scientists and was published by Elsevier [1].

Over 40 researchers from my university are on the list [2,3]:

On a basis of the most recent data from 2020:

[1] Baas, Jeroen; Boyack, Kevin; Ioannidis, John P.A. (2021), “August 2021 data-update for “Updated science-wide author databases of standardized citation indicators””, Mendeley Data, V3, doi: 10.17632/btchxktzyw.3

[2] https://pwr.edu.pl/uczelnia/aktualnosci/ponad-40-naukowcow-pwr-wysoko-w-rankingu-top-2-12213.html 

[3] https://pwr.edu.pl/fcp/2GBUKOQtTKlQhbx08SlkATxYCEi8pMgQGS39TBUpVFiwNUFhxRkkjFSg/1/public/banertop2proc_v2a.jpg

Posted in News | Tagged , , , | Comments closed

The Importance of the Correlation in Crossover Experiments

Happy to share link https://doi.org/10.1109/TSE.2021.3070480 to my just accepted, recent joint IEEE TSE paper “The Importance of the Correlation in Crossover Experiments” with Barbara Kitchenham, Giuseppe Scanniello and Carmine Gravino

Posted in News | Tagged , , , , | Comments closed

OECD Recommendation’s Draft Concerning Access to Research Data from Public Funding: A Review

Happy to share a result of joint work with Prof. Barbara Kitchenham and my PhD student Tomasz Lewowski:

Lech Madeyski, Tomasz Lewowski, and Barbara Kitchenham, “OECD Recommendation’s Draft Concerning Access to Research Data from Public Funding: A Review”, Bulletin of the Polish Academy of Sciences: Technical Sciences, vol. 69, no. 1, p. e135401, 2021. DOI: 10.24425/bpasts.2020.135401 Draft of the accepted paper: https://madeyski.e-informatyka.pl/download/MadeyskiLewowskiKitchenham21BPASTS.pdf

Posted in News | Comments closed

Empirical Software Engineering Reviewer Certificate of Recognition

Reviewing papers for the most influential international journal for Empirical Software Engineering (ESE) research may help to strengthen the ESE community around the journal, but can also be rewarding.

Empirical Software Engineering Reviewer Certificate of Recognition

Posted in News | Comments closed

Co-Chairing EASE’2020 Workshops in Trondheim

Together with Prof. Apostolos Ampatzoglou, I’ve a pleasure to chair Workshops Track of the 24th International Conference on Evaluation and Assessment in Software Engineering EASE’2020 (Trondheim, Norway, April 15-17, 2020) ranked as “A” in the EAR/CORE Rankings of Worldwide Journals and Conferences.

We’ve accepted three workshops (https://www.ntnu.edu/web/ease2020/workshops):

WWW: https://www.ntnu.edu/web/ease2020/home

Posted in News | Tagged , , , , , | Comments closed

Co-Chairing the Emerging Results and Vision Papers Track of EASE’2019 in Copenhagen

Together with Prof. Mika Mäntylä, I’ve a pleasure to chair the Emerging Results and Vision Papers Track of 23th International Conference on Evaluation and Assessment in Software Engineering EASE’2019 (University of Copenhagen, Copenhagen, Denmark, April 15-17, 2019) ranked as “A” in the EAR/CORE Rankings of Worldwide Journals and Conferences.

CfP: https://madeyski.e-informatyka.pl/download/EASE2019EV.pdf

WWW: http://ease2019.org

Posted in News | Tagged , , , , | Comments closed

ICSE’18 in Gothenburg, Sweden

I had a great pleasure to be invited to present results my joint research with Prof. Barbara Kitchenham on “Effect Sizes and their Variance for AB/BA Crossover Design Studies” at the highest ranked (CORE-2017 A*) International Conference on Software Engineering - see photo by Prof. Andrzej Wąsowski on Twitter.

It was exciting to meet, talk with, or listen to the keynotes of the SE legends and authorities like Barry Boehm (who told me that he was in Poland once invited by his friend “Wlad Turski”), Fred Brooks, Brian Randell, Margaret Hamilton, Andreas Zeller, Andrew J. Ko, Walter Tichy, Richard DeMillo (one of the originators of mutation testing I cite usually at the beginning of my mutation testing papers!), Natalia Juristo, Oscar Dieste…

Btw. I’ve to admit that I did not know that Prof. Władysław Marek Turski gave a keynote on “Software Engineering As it Will Be” (entitled “Look Ahead at Software Engineering”) at the 4′th ICSE in 1979. Other keynote speakers were Brian Randell (“Software Engineering As it Was”), Barry Boehm (“Software Engineering As it Is”), and Edsger Dijkstra (“Software Engineering As it Should Be”).

Posted in News | Tagged , , , , , , , , , , , , , , , , , | Comments closed

Special Issue on Software Engineering Methods, Tools and Products Improvement and Evaluation in Foundations of Computing and Decision Sciences

Grab the opportunity and submit your paper to special issue on Software Engineering Methods, Tools and Products Improvement and Evaluation in Foundations of Computing and Decision Sciences (open access, no fee journal indexed by Web of Science-ESCI and Scopus).

Twitter

Posted in News | Tagged , , , , , , , , , | Comments closed