DBpedia Release Evaluation: Difference between revisions

From Mediawiki1
Jump to navigationJump to search
No edit summary
No edit summary
Line 1: Line 1:
== DBpedia Release Evaluation ==
== DBpedia Release Evaluation ==


The [http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/dbpedia/raw-file/946fc9d51783/qualityAssessmentFramework/Kreis2011__Design_of_a_Quality_Assessment_Framework.pdf Quality Assessment Framework (QAF)] is developed to document the quality of the knowledge base and furthermore the progress of DBpedia's extraction framework. The main idea of this framework is a comparison between a manually created best-case dataset and the output from DBpedia's ontology based extraction. The QAF estimates the precision of the extraction framework and the completeness (recall) of DBpedia compared to its source Wikipedia.
The [http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/dbpedia/raw-file/946fc9d51783/qualityAssessmentFramework/Kreis2011__Design_of_a_Quality_Assessment_Framework.pdf Quality Assessment Framework (QAF)] is developed to document the quality of the knowledge base and furthermore the progress of DBpedia's extraction framework. The main idea of the QAF is a comparison between a manually created best-case dataset and the output from DBpedia's ontology based extraction. The QAF estimates the precision of the extraction framework and the completeness (recall) of DBpedia compared to its source Wikipedia.


== Evaluation Results ==
== Evaluation Results ==


{| class="wikitable"
{| class="wikitable"
|+ Completeness (fixed mappings from 3.5.1)
|+ Completeness / Recall (fixed mappings from 3.5.1)
|-
|-
! Category
! Category
Line 72: Line 72:
| 57,1%
| 57,1%
| 71,4%
| 71,4%
|}
{| class="wikitable"
|+ Precision (fixed mappings from 3.5.1)
|-
! Category
! DBpedia 3.5.1
! DBpedia 3.6
! DBpedia 3.7
|-
| Total
| 91,2%
| 92,3%
| 92,4%
|-
| Plain Property
| 96,3%
| 96,6%
| 97,4%
|-
| Number-Unit
| 85,4%
| 85,4%
| 85,0%
|-
| Coordinate
| 100,0%
| 100,0%
| 100,0%
|-
| Interval
| 100,0%
| 100,0%
| 88,9%
|-
| List
| 91,5%
| 92,8%
| 93,2%
|-
| One-Property-Table
| 32,5%
| 36,8%
| 34,1%
|-
| Multi-Poprty-Table
| na
| na
| na
|-
| Open Property
| 100,0%
| 100,0%
| 100,0%
|-
| Open Property Table
| na
| na
| na
|-
| Internal Template
| 83,3%
| 75,0%
| 75,0%
|-
| Merged Properties
| 80,0%
| 80,0%
| 100,0%
|}
|}

Revision as of 14:15, 14 September 2011

DBpedia Release Evaluation

The Quality Assessment Framework (QAF) is developed to document the quality of the knowledge base and furthermore the progress of DBpedia's extraction framework. The main idea of the QAF is a comparison between a manually created best-case dataset and the output from DBpedia's ontology based extraction. The QAF estimates the precision of the extraction framework and the completeness (recall) of DBpedia compared to its source Wikipedia.

Evaluation Results

Completeness / Recall (fixed mappings from 3.5.1)
Category DBpedia 3.5.1 DBpedia 3.6 DBpedia 3.7
Total 45,7% 60,2% 61,8%
Plain Property 80,5% 83,7% 86,0%
Number-Unit 68,6% 68,6% 66,7%
Coordinate 100,0% 100,0% 100,0%
Interval 72,7% 68,2% 72,7%
List 33,9% 75,7% 77,8%
One-Property-Table 5,4% 5,8% 5,8%
Multi-Poprty-Table 0,0% 0,0% 0,0%
Open Property 23,1% 23,1% 30,8%
Open Property Table na na na
Internal Template 8,6% 10,3% 10,3%
Merged Properties 57,1% 57,1% 71,4%
Precision (fixed mappings from 3.5.1)
Category DBpedia 3.5.1 DBpedia 3.6 DBpedia 3.7
Total 91,2% 92,3% 92,4%
Plain Property 96,3% 96,6% 97,4%
Number-Unit 85,4% 85,4% 85,0%
Coordinate 100,0% 100,0% 100,0%
Interval 100,0% 100,0% 88,9%
List 91,5% 92,8% 93,2%
One-Property-Table 32,5% 36,8% 34,1%
Multi-Poprty-Table na na na
Open Property 100,0% 100,0% 100,0%
Open Property Table na na na
Internal Template 83,3% 75,0% 75,0%
Merged Properties 80,0% 80,0% 100,0%
Retrieved from "https://mediawiki1.informatik.uni-mannheim.de/index.php?title=DBpedia_Release_Evaluation&oldid=15115"