The GECCO Industrial Challenge is a yearly competition for CI researchers and practitioners taking place at the GECCO. It poses difficult real-world problems provided by industry partners from various fields.
Simulation models are valuable tools for resource usage estimation and capacity planning. The simulator, BaBSim.Hospital, explicitly covers difficulties for hospitals caused by the COVID-19 pandemic. The simulator can handle many aspects of resource planning in hospitals, such as ICU beds, ventilators or personal, while taking into consideration several cohorts as age or current health status.
The task represents an instance of an expensive, high-dimensional computer simulation-based optimization problem. The simulations will be executed through an interface and hosted on one of our servers (similar to our last year’s challenge).
Your goal is to find an optimal parameter configuration for the BaBSim.Hospital simulator with a very limited budget of objective function evaluations. The participants will be free to apply one or multiple optimization algorithms of their choice.
The publication “Expected Improvement versus Predicted Value in Surrogate-Based Optimization”, written by Frederik Rehbach, Martin Zaefferer, Boris Naujoks, and Thomas Bartz-Beielstein, deals with the correct parameterization for model-based optimization algorithms. The publication is available on Cologne Open Science.
The most recent version of the article “Benchmarking in Optimization: Best Practice and Open Issues”, which was written by Thomas Bartz-Beielstein, Carola Doerr, Jakob Bossek, Sowmya Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, Manuel Lopez-Ibanez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, and Thomas Weise can be downloaded from arXiv http://arxiv.org/abs/2007.03488.
This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world. Promoting best practice in benchmarking is its main goal. The article discusses eight essential topics in benchmarking: clearly stated goals, well- specified problems, suitable algorithms, adequate performance measures, thoughtful analysis, effective and efficient designs, comprehensible presentations, and guaranteed reproducibility. The final goal is to provide well-accepted guidelines (rules) that might be useful for authors and reviewers. As benchmarking in optimization is an active and evolving field of research this manuscript is meant to co-evolve over time by means of periodic updates.
The PDF version of this survey is available here and will be published on arXiv soon.