SPOT

About

The Sequential Parameter Optimization Toolbox (SPOT) provides a set of tools for model-based optimization and tuning of algorithms. It includes surrogate models, optimizers, and design of experiment approaches. The main interface is spot, which uses sequentially updated surrogate models for the purpose of efficient optimization. The main goal is to ease the burden of objective function evaluations, when a single evaluation requires a significant amount of resources.

For more than 15 years, SPOT is used as an efficient and effective hyperparameter optimization (HPO) tool. The figure below shows results from the 2005 CEC paper: performance of a particle swarm optimization with default and tuned hyperparameters.

Run length distribution comparing the default and the tuned PSO on the Rosenbrock function. This is only one example that demonstrates how SPO improves the efficiency of search heuristics. Consider the arrow: 80 % of the runs from the tuned algorithm were able to complete the run successfully after 1000 function evaluations, whereas none of the PSO with the default configuration was able to reach the pre–specified goal with the budget.

Software

SPOT (Core Package)

SPOT is open source and can be downloaded from CRAN:
https://cran.r-project.org/package=SPOT

The most recent SPOT version is available here:
http://www.gm.fh-koeln.de/~bartz/R/SPOT_2.5.4.tar.gz

SPOTMisc (Additional Features)

SPOTMisc implements additional functions. It is open source and can be downloaded from CRAN:
https://cran.r-project.org/package=SPOTMisc

The most recent SPOT version is available here:
http://www.gm.fh-koeln.de/~bartz/R/SPOTMisc_1.2.4.tar.gz

SPOTMisc can be installed with the following command, where “~/Downloads/” is the name of the local folder:

install.packages("~/Downloads/SPOTMisc_1.2.4.tar.gz", repos=NULL, type="source")

spotGUI (Graphical User Interface)

A graphical user interface for the Sequential Parameter Optimization Toolbox (package ‘SPOT’). It includes a quick, graphical setup for spot, interactive 3D plots, export possibilities and more.
https://cran.r-project.org/package=spotGUI

Team

Active: Thomas Bartz-Beielstein, Martin Zaefferer, and Frederik Rehbach
Maintainer: Thomas Bartz-Beielstein

Literature

The old reference from 2005:

  • T. Bartz-Beielstein, C.W.G. Lasarczyk, and M. Preuss. Sequential parameter optimization. In Evolutionary Computation, 2005. The 2005 IEEE Congress on, volume 1, pages 773–780, New York, NY, USA, Sept 2005. IEEE.

Recommended reading:

  • Bartz-Beielstein, T., Zaefferer, M., and Rehbach, F. In a Nutshell – The Sequential Parameter Optimization Toolbox. arXiv e-prints (Dec. 2021), https://arxiv.org/abs/1712.04076.