KYOTO Logo
Knowledge Yielding Ontologies
for Transition-based Organization
  • Increase font size
  • Default font size
  • Decrease font size
Evaluation

Evaluation - Evaluation of facts

E-mail Print PDF
Article Index
Evaluation
Evaluation of Module
Evaluation of facts
Evaluation of semantic search
Evaluation of the knowledge editor
All Pages


Evaluation of facts:

We developed a complete package to evaluate the output of the event/fact mining by the Kybots.We defined a neutral triplet representation for representing the facts, which consists of:

  1. a relation
  2. a list of word token identifiers that represent the event
  3. a list of word token identifiers that represent the participant
If a fact has more than one participant, it is broken down into separate triplets. We created a gold-standard and defined a baseline. We experimented with various settings. The details are described in D5.4. With the best settings, we obtain a precision of 50% and a recall of 40%. We recover 100% of the events with a precision of 29%.

The evaluation data and software can be downloaded from the following URL:

https://kyoto.let.vu.nl/~kyoto/files/data/kybotevaluation/11767/KybotEvaluationDataMarch2011.zip (12 MB)

We also carried out an open competition evaluation in combination with the 2nd KYOTO workshop for which we created a different gold-standard. Two other groups participated. The details can be found on the workshop-event page.

 

 

 



 

ICT-211423 - 2008 © Kyoto Consortium