|
        
NN3 Results
Results
on the Complete Dataset of 111 Time Series
This represents the actual
benchmark of the NN3 competition, as the reduced dataset of 11 series is
included in the 111. Congratulations to all of you that were able to
forecast this many time series automatically! Please find the results for
the top 50% of submissions released below by name and description. All other
participants must contact the competition organisers via email to agree the
disclosure of their name and method with their rank.
Please
note that you must have
the login-information to download the descriptions.
If you have not yet registered, or have forgotten your login, please
visit the [NN3 Homepage] and register your
email. The access information will then be sent to you.
|
Rank on
SMAPE |
Participant |
SMAPE |
CONFERENCE
PRESENTATION |
DESCRIPTION |
|
- |
Stat. Contender - Wildi |
14,84% |
|
|
|
- |
Stat. Benchmark - Theta Method (Nikolopoulos) |
14,89% |
|
description missing |
|
1 |
Illies, Jäger,
Kosuchinas, Rincon, Sakenas, Vaskevcius |
15,18% |
|
|
|
- |
Stat. Benchmark - ForecastPro (Stellwagen) |
15,44% |
|
|
|
- |
CI Benchmark - Theta AI (Nikolopoulos) |
15,66% |
presentation
missing |
description missing |
|
- |
Stat. Benchmark - Autobox (Reilly) |
15,95% |
|
|
|
2 |
Adeodato,
Vasconcelos, Arnaud, Chunha, Monteiro |
16,17% |
|
|
|
3 |
Flores, Anaya,
Ramirez, Morales |
16,31% |
presentation
missing |
|
|
4 |
Chen, Yao |
16,55% |
presentation
missing |
|
|
5 |
D'yakonov |
16,57% |
|
|
|
6 |
Kamel, Atiya,
Gayar, El-Shishiny |
16,92% |
|
|
|
7 |
Abou-Nasr |
17,54% |
|
|
|
8 |
Theodosiou,
Swamy |
17,55% |
|
|
|
- |
CI Benchmark - Naive MLP (Crone) |
17,84% |
|
|
|
9 |
de Vos |
18,24% |
|
|
|
10 |
Yan |
18,58% |
|
|
|
- |
CI Benchmark - Naive SVR (Crone, Pietsch) |
18,60% |
|
|
|
11 |
C49 |
18,72% |
|
not disclosed by author |
|
12 |
Perfilieva, Novak,
Pavliska, Dvorak, Stepnicka |
18,81% |
|
|
|
13 |
Kurogi, Koyama,
Tanaka, Sanuki |
19,00% |
presentation
missing |
|
|
14 |
Stat.
Contender - Beadle |
19,14% |
|
|
|
15 |
Stat.
Contender - Lewicke |
19,17% |
|
|
|
16 |
Sorjamaa,
Lendasse |
19,60% |
|
|
|
17 |
Isa |
20,00% |
|
|
|
18 |
C28 |
20,54% |
|
not disclosed by author |
|
19 |
Duclos-Gosselin |
20,85% |
|
|
|
- |
Stat. Benchmark - Naive |
22,69% |
|
not disclosed by author |
|
20 |
Papadaki, Amaxopolous |
22,70% |
|
|
|
21 |
Stat. Benchmark -
Hazarika |
23,72% |
|
|
|
22 |
C17 |
24,09% |
|
not disclosed by author |
|
23 |
Stat.
Contender - Njimi, Mélard |
24,90% |
|
|
|
24 |
Pucheta, Patino,
Kuchen |
25,13% |
|
|
|
25 |
Corzo,
Hong |
27,53% |
|
|
Submissions in RED are statistical methods that entered the competition as
"benchmarks". They can either be existing and estabished statistical methods
or novel methods entered to be evaluated through the competition (e.g.
Wildi). Submissions in BLUE are CI methods that
entered the competition as "benchmarks" but were computed by the organisers
as points of reference (e.g. Theta AI, Naive MLP etc.).
Only original submissions with mthods from compuational Intelligence were
eligible to win the competition (no benchmarks, no statistical methods and
were in part calculated by the NN3 supervisors).
Results
on the Reduced Dataset of 11 Time Series (subset of the complete)
Please find the results
for the top 50% of submissions released below by name and description (plus
the ones already disclosed on the complete dataset). All other participants
must contact the competition organisers via email to agree the disclosure of
their name and method with their rank.
|
Rank on SMAPE |
Participant |
SMAPE |
CONFERENCE
PRESENTATION |
DESCRIPTION |
|
|
CI
Benchmark - Theta AI (Nikolopoulos) |
13,07% |
|
pending
|
|
|
Stat.
Benchmark - Autobox (Reily) |
13,49% |
|
|
|
|
Stat.
Benchmark - ForecastPro (Stellwagen) |
13,52% |
|
|
|
1 |
Yan |
13,68% |
|
|
|
|
Stat.
Benchmark - Theta (Nikolopoulos) |
13,70% |
|
description missing |
|
2 |
llies,
Jäger, Kosuchinas, Rincon, Sakenas, Vaskevcius |
14,26% |
|
|
|
3 |
Chen, Yao |
14,46% |
presentation
missing |
|
|
4 |
Yousefi,
Miromeni, Lucas |
14,49% |
|
|
|
5 |
Ahmed, Atiya, Gayar, El-Shishiny |
14,52% |
|
|
|
6 |
Flores, Anaya,
Ramirez, Morales |
15,00% |
presentation
missing |
|
|
7 |
Adeodato,
Vasconcelos, Arnaud, Chunha, Monteiro |
15,10% |
|
|
|
|
Stat.
Contender - Wildi |
15,32% |
|
|
|
8 |
Luna,
Soares, Ballini |
15,35% |
|
|
|
9 |
Theodosiou,
Swamy |
16,19% |
|
|
|
10 |
Hwang, Song,
Kasabov |
16,31% |
|
|
|
11 |
Duclos-Gosselin |
16,37% |
|
|
|
12 |
Kurogi, Koyama,
Tanaka, Sanuki |
16,49% |
presentation
missing |
|
|
13 |
White |
16,56% |
presentation
missing |
|
|
14 |
Abou-Nasr |
16,69% |
|
|
|
|
Stat.
Contender - Beadle |
17,14% |
|
|
|
15 |
Sorjamaa,
Lendasse |
17,16% |
|
|
|
|
Stat.
Contender - Njimi, Mélard |
17,19% |
|
|
|
16 |
Rabie |
17,24% |
|
|
|
17 |
Jimenez,
Rebuzzi Vellasco, Tanscheit |
17,78% |
|
|
|
18 |
Ruta, Gabrys |
17,90% |
|
|
|
19 |
Isa |
18,07% |
|
|
|
|
CI
Benchmark - Naive SVR |
18,37% |
|
|
|
20 |
Fillon, Bartoli,
Poloni |
18,39% |
|
|
|
|
CI
Benchmark - Naive MLP |
18,69% |
|
|
|
|
Stat.
Contender - Lewicke |
19,51% |
|
|
|
21 |
Perfilieva, Novak,
Pavliska, Dvorak, Stepnicka |
19,77% |
|
|
|
22 |
Safavieh, Andalib, Andalib |
20,04% |
|
|
|
23 |
de Vos |
20,26% |
|
|
|
24 |
C52 |
20,35% |
|
not disclosed by author |
|
25 |
D'yakunov |
20,45% |
|
|
|
26 |
C32 |
20,63% |
|
not disclosed by author |
|
27 |
Weng,
Liu, Cheng, Hwang |
20,69% |
|
|
|
28 |
C29 |
20,76% |
|
not disclosed by author |
|
29 |
C49 |
21,05% |
|
not disclosed by author |
|
|
Stat.
Benchmark - X12 ARIMA (McElroy) |
21,48% |
|
|
|
30 |
C35 |
24,03% |
|
not disclosed by author |
|
31 |
C6 |
24,05% |
|
not disclosed by author |
|
32 |
C28 |
24,05% |
|
not disclosed by author |
|
33 |
Phienthrakul,
Kijsirikul |
25,69% |
|
|
|
|
Stat.
Benchmark - Naive |
25,71% |
|
not disclosed by author |
|
34 |
C45 |
26,04% |
|
not disclosed by author |
|
35 |
Pucheta, Patino,
Kuchen |
27,39% |
|
|
|
36 |
Papadaki,
Amaxopolous |
28,10% |
|
|
|
37 |
Stat. Benchmark -
Hazarika |
28,62% |
|
|
|
38 |
C39 |
29,14% |
|
not disclosed by author |
|
39 |
C17 |
30,81% |
|
not disclosed by author |
|
40 |
C56 |
32,15% |
|
not disclosed by author |
|
41 |
C14 |
33,42% |
|
not disclosed by author |
|
42 |
Carjabal |
34,77% |
|
|
|
43 |
Peralta, Gutiereez,
Sanchis |
36,71% |
|
|
|
44 |
Kuremoto, Obayashi,
Kobayashi |
38,45% |
|
|
|
45 |
Corzo,
Hong |
67,38% |
|
|
Submissions in RED are statistical methods that entered the competition as
"benchmarks". They can either be existing and estabished statistical methods
or novel methods entered to be evaluated through the competition (e.g.
Wildi). Submissions in BLUE are CI methods that
entered the competition as "benchmarks" but were computed by the organisers
as points of reference (e.g. Theta AI, Naive MLP etc.).
Only original submissions with mthods from compuational Intelligence were
eligible to win the competition (no benchmarks, no statistical methods and
were in part calculated by the NN3 supervisors).
|