KYOTO Logo
Knowledge Yielding Ontologies
for Transition-based Organization
  • Increase font size
  • Default font size
  • Decrease font size

Evaluation

E-mail Print PDF
Article Index
Evaluation
Evaluation of Module
Evaluation of facts
Evaluation of semantic search
Evaluation of the knowledge editor
All Pages

KYOTO EVALUATION

The ultimate goal of KYOTO is the extraction of facts from text.This is done by the Kybot system at the end of the pipeline of processing. The Kybots depend on the results of all the previous processing modules. Errors in the processing may be stacked. For example, if the parser assigns the wrong part-of-speech to a word, the word-sense-disambiguation will fail to assign the proper concept, and the onto tagger will not insert the correct ontological implications into the KAF file. Consequently, the Kybots may fail because a profile specified for a certain part-of-speech does not match or the wrong profile specified for a concept is matched, thus affecting recall and or precision.

This being said, modules may also introduce errors that have no effect on the extraction of facts. Therefore, it is a wise thing to evaluate both the modules and the end-applications that depend on it to learn about the effects and relevance of errors. Likewise, the results of KYOTO have been evaluated in different ways:

  1. System evaluation:
    1. Evaluation of different modules
    2. Evaluation of the facts extracted by the KYOTO system
  2. User evaluation:
    1. Use of the knowledge editor
    2. Use of the Semantic search in the extracted facts


 

ICT-211423 - 2008 © Kyoto Consortium