To overcome the reproducibility crisis we need a culture shift towards reproducibility in EC, with reproducibility playing a bigger role in education, funding decisions, recruitment and reputation. While this requires some extra effort, especially early on, the reward will be faster scientific progress, less frustration trying to build on other’s work, and a higher reputation for the field as a whole. The journey has already begun.
https://arxiv.org/abs/2102.03380
Really important is the distinction between
- repeatability,
- reproducibility,
- replicability, and
- generalisability.
The preprint of this article is available on arXiv.
Abstract. Experimental studies are prevalent in Evolutionary Computation (EC), and concerns about the reproducibility and replicability of such studies have increased in recent times, reflecting similar concerns in other scientific fields. In this article, we discuss, within the context of EC, the different types of reproducibility and suggest a classification that refines the badge system of the Association of Computing Machinery (ACM) adopted by ACM Transactions on Evolutionary Learning and Optimization (this https URL). We identify cultural and technical obstacles to reproducibility in the EC field. Finally, we provide guidelines and suggest tools that may help to overcome some of these reproducibility obstacles.