ORE Competition

Participating Systems

Chainsaw: http://sourceforge.net/projects/chainsaw/
ELepHant: https://code.google.com/p/elephant-reasoner/
ELK: http://code.google.com/p/elk-reasoner/
FaCT++: http://code.google.com/p/factplusplus/
HermiT: http://www.hermit-reasoner.com/
jcel: http://jcel.sourceforge.net/
Jfact: http://sourceforge.net/projects/jfact/
Konclude: http://www.derivo.de/en/produkte/konclude/
MORe: http://code.google.com/p/more-reasoner/
TReasoner: https://code.google.com/p/treasoner/
TrOWL: http://trowl.eu/

Competition

In addition to the workshop, ORE organises a Live OWL Reasoner Competition, which is part of the 1st FLoC Olympic Games. The FLoC Olympic Games are intended as a new FLoC tradition and Games will bring together a multitude of established solver competitions by different research communities. The winners in the competition categories are awarded Kurt Goedel medals at the FLoC Olympic Games award ceremony (July 21, 2014).

For ORE 2014, the Kurt Goedel medals were awarded according to the ratio of Expected Results / Tasks Attempted and the winner are:

Prize: Konclude with 95.5%
by Andreas Steigmiller (U Ulm), Thorsten Liebig (derivo GmbH), Birte Glimm (U Ulm)
Prize: ELK with 86.4%
by Yevgeny Kazakov (U Ulm), Markus Krötzsch (TU Dresden), František Simančík (google), Pavel Klinov (U Ulm)
Prize: MORe with 85.7% by
Ana Armas Romero (U Oxford), Ernesto Jimenez Ruiz (U Oxford), Bernardo Cuenca Grau (U Oxford), Ian Horrocks (U Oxford), Cristina Feier (U Oxford)

Apart from the Olympic Games awards, ORE awarded prizes in six reasoning disciplies as follows:

OWL DL Consistency Konclude Chainsaw HermiT	OWL EL Consistency ELK Konclude MORe
OWL DL Classification Konclude HermiT MORe	OWL EL Classification Konclude MORe ELK
OWL DL Realisation Konclude FaCT++ TrOWL	OWL EL Realisation Konclude TrOWL FaCT++

The results presentation at DL 2014 and the presentation at the FLoC Olympic Games award ceremony are now available for download and a static version of the live competition web pages is available at http://dl.kr.org/ore2014.

The competition framework, the complete dataset, the competition results, and the competing reasoners with a license to rerun them within the competiton framework are now also available online:
Competition Framework: https://github.com/andreas-steigmiller/ore-2014-competition-framework/
Dataset: https://github.com/andreas-steigmiller/ore-2014-competition-framework/
Reasoners: https://zenodo.org/record/11145/
Results: https://zenodo.org/record/11142/

Original Call:

We invite both reasoner developers and developers of challenging ontologies to submit their systems/ontologies for participation in the OWL reasoner competition. The preliminary outline for the competition is as follows:

The test sets will include OWL ontologies from all OWL 2 profiles and of varying sizes, from a large corpus of ontologies (including ontologies from the NCBO BioPortal, a web crawl, and user-submitted ontologies).
The tested tasks will include consistency checking, classification, and realisation.
Developers will have to submit an executable wrapper script that triggers the reasoning system, and the system itself to the competition organizers, who will then carry out the benchmarks.

Further instructions for reasoner developers and a testing framework is available at:
https://bitbucket.org/andreas-steigmiller/ore-2014-testing-framework

Please note: The deadline for all system submissions is 16th May 2014.

In addition to the system submissions, both reasoner and ontology developers are invited to submit a short paper describing their system/ontology. Note that the deadline for paper submissions is the 15th April 2014 (EXTENDED).