Download PDFOpen PDF in browser

Evaluating Automated Theorem Provers Using Adimen-SUMO

9 pagesPublished: March 1, 2017


We report on the results of evaluating the performance automated theorem provers using \ADIMENSUMO{}. The evaluation follows the adaptation of the methodology based on competency questions \cite{GrF95} to the framework of first-order logic, which is presented in \cite{ALR15}, and is applied to \ADIMENSUMO{} \cite{ALR12}. The set of competency questions used for this evaluation has been semi-automatically generated from a small set of semantic patterns and the mapping of \WORDNET{} to \SUMO{}, also introduced in \cite{ALR15}. Our experimental results demonstrate that improved versions of the proposed set of competency questions could be really valuable for the development of automated theorem provers.

Keyphrases: benchmark, Ontologies, system evaluation

In: Laura Kovács and Andrei Voronkov (editors). Vampire 2016. Proceedings of the 3rd Vampire Workshop, vol 44, pages 74--82

Download PDFOpen PDF in browser