GECCO Challenge 2013 Results

Forecast Evaluation

The original test data was just as aperiodic as the given training data. So to enable a sound evaluation of the submitted forecasts, the test data has been linear interpolated to generate equidistant testdata.

For the evaluation of the predictions the rmse between the prediction ŷ and the related test data y has been used. This rmse value has been mapped to a scale from zero to one, with one beeing an errorless prediction and zero a prediction as good as the baseline prediction ybase. For the baseline prediction we chose the mean of the training data. The quality q of the prediction ŷ was than defined as
qualityFormula
The quality q of predictions with an rmse greater than the rmse of the baseline prediction were set to zero.
For the ranking of the overall submission the square root of the product of both quality values has been used.

Detailed Results

Please click on a row for additional information.

forecast temperature humidity overall
Test Data 1.00000000 1.00000000 1.00000000
Farzad Noorian 0.63851699 0.3468524 0.4706072
G. Kronberger &
M. Kommenda
0.58581900 0.3143241 0.4291119
Rommel Vergara 0.64193120 0.1183770 0.2756626
Stephan Hutterer 0.29561874 0.2535294 0.2737664
Sylvain Cussat-Blanc 0.08875647 0.2795216 0.1575098
Himanshu Jain 0.06282025 0.1691203 0.1030737
B. Calvo & C. Blum 0.23920215 0.0000000 0.0000000
Baseline 0.00000000 0.00000000 0.00000000

Additional Downloads

Download the GECCO 2013 IC Resource Package, containing the required datasets, software, and the official rules and regulations in ZIP format. In contrast to the original resource package, this package has been extended by the original test data and the evaluation script.

You can download the slides of the Industrial Challenge Results Presentation at GECCO as pdf File. The presentations of the three winning submissions are also separately available. You find all additional information about the submissions in the results table above by clicking the respective row.