SPOT – SPOTSeven Lab

About

The Sequential Parameter Optimization Toolbox (SPOT) provides a set of tools for model-based optimization and tuning of algorithms. It includes surrogate models, optimizers, and design of experiment approaches. The main interface is spot, which uses sequentially updated surrogate models for the purpose of efficient optimization. The main goal is to ease the burden of objective function evaluations, when a single evaluation requires a significant amount of resources.

For more than 15 years, SPOT is used as an efficient and effective hyperparameter optimization (HPO) tool. The figure below shows results from the 2005 CEC paper: performance of a particle swarm optimization with default and tuned hyperparameters.

Run length distribution comparing the default and the tuned PSO on the Rosenbrock function. This is only one example that demonstrates how SPO improves the efficiency of search heuristics. Consider the arrow: 80 % of the runs from the tuned algorithm were able to complete the run successfully after 1000 function evaluations, whereas none of the PSO with the default configuration was able to reach the pre–specified goal with the budget.

spotPython: Software and Documentation

spot is available as a python (“spotPython”) package:

[GitHub]: https://github.com/sequential-parameter-optimization/spotPython
[Documentation]: https://sequential-parameter-optimization.github.io/spotPython/

[Hyperparameter Tuning Cookbook]: https://sequential-parameter-optimization.github.io/Hyperparameter-Tuning-Cookbook/

The related package spotRiver implements tools for analyzing and tuning of River:

[GitHub]: https://github.com/sequential-parameter-optimization/spotRiver
[Documentation]: https://sequential-parameter-optimization.github.io/spotRiver/

Python: Software and Doumentation

Useful links:

The Python tutorial: https://docs.python.org/3/tutorial/.

R: Software

The R version of SPOT is not maintained any more.

SPOT (Core Package, Version 2.11.14)

SPOT is open source and can be downloaded from CRAN:
https://cran.r-project.org/package=SPOT

The most recent SPOT version is available here (Hint for Safari users: use right-click download for downloading the compressed *.tar.gz archive):
http://www.gm.fh-koeln.de/~bartz/R/SPOT_LATEST.tar.gz

SPOT can be installed with the following command, where “~/Downloads/” is the name of the local folder:

install.packages("~/Downloads/SPOT_LATEST.tar.gz", repos=NULL, type="source")

SPOTMisc (Additional Features, Version 1.19.52)

SPOTMisc implements additional functions. It is open source and can be downloaded from CRAN:
https://cran.r-project.org/package=SPOTMisc

The most recent SPOT version is available here:
http://www.gm.fh-koeln.de/~bartz/R/SPOTMisc_LATEST.tar.gz

SPOTMisc can be installed with the following command, where “~/Downloads/” is the name of the local folder:

install.packages("~/Downloads/SPOTMisc_LATEST.tar.gz", repos=NULL, type="source")

After successful installation, the package versions can be checked as follows:

packageVersion("SPOT")
[1] ‘2.11.14’
packageVersion("SPOTMisc")
[1] ‘1.19.52’

spotGUI (Graphical User Interface)

A graphical user interface for the Sequential Parameter Optimization Toolbox (package ‘SPOT’). It includes a quick, graphical setup for spot, interactive 3D plots, export possibilities and more.
https://cran.r-project.org/package=spotGUI

Team

Active: Thomas Bartz-Beielstein, Martin Zaefferer, and Frederik Rehbach
Maintainer: Thomas Bartz-Beielstein

Literature

The old reference from 2005:

T. Bartz-Beielstein, C.W.G. Lasarczyk, and M. Preuss. Sequential parameter optimization. In Evolutionary Computation, 2005. The 2005 IEEE Congress on, volume 1, pages 773–780, New York, NY, USA, Sept 2005. IEEE.