DBpedia Release Evaluation - Revision history

Kreis at 15:56, 14 September 2011

2011-09-14T15:56:23Z

← Older revision		Revision as of 17:56, 14 September 2011
Line 144:		Line 144:
	! DBpedia 3.6		! DBpedia 3.6
	! DBpedia 3.7		! DBpedia 3.7
	\|-		\|- align="right"
	\| Total		\| Total
	\| 45.7%		\| 45.7%
	\| 60.2%		\| 60.2%
	\| 61.8%		\| 61.8%
	\|-		\|- align="right"
	\| Plain Property		\| Plain Property
	\| 80.5%		\| 80.5%
	\| 83.7%		\| 83.7%
	\| 86.0%		\| 86.0%
	\|-		\|- align="right"
	\| Number-Unit		\| Number-Unit
	\| 68.6%		\| 68.6%
	\| 68.6%		\| 68.6%
	\| 66.7%		\| 66.7%
	\|-		\|- align="right"
	\| Coordinate		\| Coordinate
	\| 100.0%		\| 100.0%
	\| 100.0%		\| 100.0%
	\| 100.0%		\| 100.0%
	\|-		\|- align="right"
	\| Interval		\| Interval
	\| 72.7%		\| 72.7%
	\| 68.2%		\| 68.2%
	\| 72.7%		\| 72.7%
	\|-		\|- align="right"
	\| List		\| List
	\| 33.9%		\| 33.9%
	\| 75.7%		\| 75.7%
	\| 77.8%		\| 77.8%
	\|-		\|- align="right"
	\| One-Property-Table		\| One-Property-Table
	\| 5.4%		\| 5.4%
	\| 5.8%		\| 5.8%
	\| 5.8%		\| 5.8%
	\|-		\|- align="right"
	\| Multi-Poprty-Table		\| Multi-Poprty-Table
	\| 0.0%		\| 0.0%
	\| 0.0%		\| 0.0%
	\| 0.0%		\| 0.0%
	\|-		\|- align="right"
	\| Open Property		\| Open Property
	\| 23.1%		\| 23.1%
	\| 23.1%		\| 23.1%
	\| 30.8%		\| 30.8%
	\|-		\|- align="right"
	\| Open Property Table		\| Open Property Table
	\| na		\| na
	\| na		\| na
	\| na		\| na
	\|-		\|- align="right"
	\| Internal Template		\| Internal Template
	\| 8.6%		\| 8.6%
	\| 10.3%		\| 10.3%
	\| 10.3%		\| 10.3%
	\|-		\|- align="right"
	\| Merged Properties		\| Merged Properties
	\| 57.1%		\| 57.1%
Line 213:		Line 213:
	! DBpedia 3.6		! DBpedia 3.6
	! DBpedia 3.7		! DBpedia 3.7
	\|-		\|- align="right"
	\| Total		\| Total
	\| 91.2%		\| 91.2%
	\| 92.3%		\| 92.3%
	\| 92.4%		\| 92.4%
	\|-		\|- align="right"
	\| Plain Property		\| Plain Property
	\| 96.3%		\| 96.3%
	\| 96.6%		\| 96.6%
	\| 97.4%		\| 97.4%
	\|-		\|- align="right"
	\| Number-Unit		\| Number-Unit
	\| 85.4%		\| 85.4%
	\| 85.4%		\| 85.4%
	\| 85.0%		\| 85.0%
	\|-		\|- align="right"
	\| Coordinate		\| Coordinate
	\| 100.0%		\| 100.0%
	\| 100.0%		\| 100.0%
	\| 100.0%		\| 100.0%
	\|-		\|- align="right"
	\| Interval		\| Interval
	\| 100.0%		\| 100.0%
	\| 100.0%		\| 100.0%
	\| 88.9%		\| 88.9%
	\|-		\|- align="right"
	\| List		\| List
	\| 91.5%		\| 91.5%
	\| 92.8%		\| 92.8%
	\| 93.2%		\| 93.2%
	\|-		\|- align="right"
	\| One-Property-Table		\| One-Property-Table
	\| 32.5%		\| 32.5%
	\| 36.8%		\| 36.8%
	\| 34.1%		\| 34.1%
	\|-		\|- align="right"
	\| Multi-Poprty-Table		\| Multi-Poprty-Table
	\| na		\| na
	\| na		\| na
	\| na		\| na
	\|-		\|- align="right"
	\| Open Property		\| Open Property
	\| 100.0%		\| 100.0%
	\| 100.0%		\| 100.0%
	\| 100.0%		\| 100.0%
	\|-		\|- align="right"
	\| Open Property Table		\| Open Property Table
	\| na		\| na
	\| na		\| na
	\| na		\| na
	\|-		\|- align="right"
	\| Internal Template		\| Internal Template
	\| 83.3%		\| 83.3%
	\| 75.0%		\| 75.0%
	\| 75.0%		\| 75.0%
	\|-		\|- align="right"
	\| Merged Properties		\| Merged Properties
	\| 80.0%		\| 80.0%

Kreis at 15:54, 14 September 2011

2011-09-14T15:54:58Z

← Older revision		Revision as of 17:54, 14 September 2011
Line 23:		Line 23:
	! Mapped Triples		! Mapped Triples
	! %		! %
	\|-		\|- align="right"
	\| Total		\| Total
	\| 3221		\| 3221
Line 32:		Line 32:
	\| 1732		\| 1732
	\| 53.8		\| 53.8
	\|-		\|- align="right"
	\| Plain Property		\| Plain Property
	\| 893		\| 893
Line 41:		Line 41:
	\| 599		\| 599
	\| 67.1		\| 67.1
	\|-		\|- align="right"
	\| Number-Unit		\| Number-Unit
	\| 76		\| 76
Line 50:		Line 50:
	\| 57		\| 57
	\| 75.0		\| 75.0
	\|-		\|- align="right"
	\| Coordinate		\| Coordinate
	\| 54		\| 54
Line 59:		Line 59:
	\| 48		\| 48
	\| 88.9		\| 88.9
	\|-		\|- align="right"
	\| Interval		\| Interval
	\| 31		\| 31
Line 68:		Line 68:
	\| 24		\| 24
	\| 77.4		\| 77.4
	\|-		\|- align="right"
	\| List		\| List
	\| 801		\| 801
Line 77:		Line 77:
	\| 560		\| 560
	\| 69.9		\| 69.9
	\|-		\|- align="right"
	\| One-Property-Table		\| One-Property-Table
	\| 447		\| 447
Line 86:		Line 86:
	\| 236		\| 236
	\| 52.8		\| 52.8
	\|-		\|- align="right"
	\| Multi-Poprty-Table		\| Multi-Poprty-Table
	\| 625		\| 625
Line 95:		Line 95:
	\| 109		\| 109
	\| 17.4		\| 17.4
	\|-		\|- align="right"
	\| Open Property		\| Open Property
	\| 139		\| 139
Line 104:		Line 104:
	\| 16		\| 16
	\| 11.5		\| 11.5
	\|-		\|- align="right"
	\| Open Property Table		\| Open Property Table
	\| 26		\| 26
Line 113:		Line 113:
	\| 0		\| 0
	\| 0.0		\| 0.0
	\|-		\|- align="right"
	\| Internal Template		\| Internal Template
	\| 116		\| 116
Line 122:		Line 122:
	\| 76		\| 76
	\| 65.5		\| 65.5
	\|-		\|- align="right"
	\| Merged Properties		\| Merged Properties
	\| 13		\| 13

Kreis at 15:45, 14 September 2011

2011-09-14T15:45:00Z

← Older revision		Revision as of 17:45, 14 September 2011
Line 4:		Line 4:

	For a significant evaluation, only potentially extractable triples are considered. Only if a triple arise from a mapped property it can be extracted. Here this triples are called mapped triples.		For a significant evaluation, only potentially extractable triples are considered. Only if a triple arise from a mapped property it can be extracted. Here this triples are called mapped triples.
	The following table shows the number of ~~mapped~~ triples, the ~~total~~ number of triples ~~in the Gold Standard~~ and the percentage of mapped triples for each category. The categories result from different patterns in which the information is given in Wikipedia infoboxes. The number of cases differ in a high extent depending on the category. The results for the categories based on small numbers should be handled with care.		The following table shows the total number of triples in the Gold Standard, the number of mapped triples and the percentage of mapped triples for each category. The categories result from different patterns in which the information is given in Wikipedia infoboxes. The number of cases differ in a high extent depending on the category. The results for the categories based on small numbers should be handled with care.

	{\| class="wikitable"		{\| class="wikitable"

Kreis at 15:43, 14 September 2011

2011-09-14T15:43:47Z

← Older revision		Revision as of 17:43, 14 September 2011
Line 1:		Line 1:
	The [http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/dbpedia/raw-file/946fc9d51783/qualityAssessmentFramework/Kreis2011__Design_of_a_Quality_Assessment_Framework.pdf Quality Assessment Framework (QAF)] is developed to document the quality of the knowledge base and furthermore the progress of DBpedia's extraction framework. The main idea of the QAF is a comparison between a manually created best-case dataset (Gold Standard) and the output from DBpedia's ontology based extraction. The QAF estimates the precision of the extraction framework and the completeness (recall) of DBpedia compared to its source Wikipedia.		The [http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/dbpedia/raw-file/946fc9d51783/qualityAssessmentFramework/Kreis2011__Design_of_a_Quality_Assessment_Framework.pdf Quality Assessment Framework (QAF)] is developed to document the quality of the knowledge base and furthermore the progress of DBpedia's extraction framework. The main idea of the QAF is a comparison between a manually created best-case dataset (Gold Standard) and the output from DBpedia's ontology based extraction. The QAF estimates the precision of the extraction framework and the completeness (recall) of DBpedia compared to its source Wikipedia.

	== ~~Sample Data /~~ Gold Standard ==		== Gold Standard & Mappings ==

	For a significant evaluation, only potentially extractable triples are considered. Only if a triple arise from a mapped property it can be extracted. Here this triples are called mapped triples.		For a significant evaluation, only potentially extractable triples are considered. Only if a triple arise from a mapped property it can be extracted. Here this triples are called mapped triples.

Kreis at 15:41, 14 September 2011

2011-09-14T15:41:24Z

Show changes

Kreis at 15:38, 14 September 2011

2011-09-14T15:38:42Z

← Older revision		Revision as of 17:38, 14 September 2011
Line 17:		Line 17:
	! colspan="2" \| DBpedia 3.7		! colspan="2" \| DBpedia 3.7
	\|-		\|-
			! Mapped Triples
			! %
			! Mapped Triples
			! %
	! Mapped Triples		! Mapped Triples
	! %		! %

Kreis at 15:38, 14 September 2011

2011-09-14T15:38:05Z

← Older revision		Revision as of 17:38, 14 September 2011
Line 11:		Line 11:
	! rowspan="3" \| Category		! rowspan="3" \| Category
	! rowspan="3" \| Gold Triples		! rowspan="3" \| Gold Triples
	! colspan="2" \| DBpedia Release		! colspan="6" \| DBpedia Release
	\|-		\|-
	! colspan="2" \| DBpedia 3.5.1		! colspan="2" \| DBpedia 3.5.1
			! colspan="2" \| DBpedia 3.6
			! colspan="2" \| DBpedia 3.7
	\|-		\|-
	! Mapped Triples		! Mapped Triples
Line 19:		Line 21:
	\|-		\|-
	\| Total		\| Total
			\| 3221
	\| 1504		\| 1504
	~~\| 3221~~
	\| 46.7		\| 46.7
			\| 1522
			\| 47.3
			\| 1732
			\| 53.8
	\|-		\|-
	\| Plain Property		\| Plain Property
			\| 893
	\| 514		\| 514
	~~\| 893~~
	\| 57.6		\| 57.6
			\| 521
			\| 58.3
			\| 599
			\| 67.1
	\|-		\|-
	\| Number-Unit		\| Number-Unit
			\| 76
	\| 51		\| 51
	~~\| 76~~
	\| 67.1		\| 67.1
			\| 52
			\| 68.4
			\| 57
			\| 75.0
	\|-		\|-
	\| Coordinate		\| Coordinate
			\| 54
	\| 36		\| 36
	~~\| 54~~
	\| 66.7		\| 66.7
			\| 39
			\| 72.2
			\| 48
			\| 88.9
	\|-		\|-
	\| Interval		\| Interval
			\| 31
			\| 22
			\| 71.0
	\| 22		\| 22
	~~\| 31~~
	\| 71.0		\| 71.0
			\| 24
			\| 77.4
	\|-		\|-
	\| List		\| List
			\| 801
	\| 478		\| 478
	~~\| 801~~
	\| 59.7		\| 59.7
			\| 482
			\| 60.2
			\| 560
			\| 69.9
	\|-		\|-
	\| One-Property-Table		\| One-Property-Table
			\| 447
	\| 242		\| 242
	~~\| 447~~
	\| 54.1		\| 54.1
			\| 244
			\| 54.6
			\| 236
			\| 52.8
	\|-		\|-
	\| Multi-Poprty-Table		\| Multi-Poprty-Table
			\| 625
			\| 83
			\| 13.3
	\| 83		\| 83
	~~\| 625~~
	\| 13.3		\| 13.3
			\| 109
			\| 17.4
	\|-		\|-
	\| Open Property		\| Open Property
			\| 139
			\| 13
			\| 9.4
	\| 13		\| 13
	~~\| 139~~
	\| 9.4		\| 9.4
			\| 16
			\| 11.5
	\|-		\|-
	\| Open Property Table		\| Open Property Table
			\| 26
			\| 0
			\| 0.0
			\| 0
			\| 0.0
	\| 0		\| 0
	~~\| 26~~
	\| 0.0		\| 0.0
	\|-		\|-
	\| Internal Template		\| Internal Template
			\| 116
	\| 58		\| 58
	~~\| 116~~
	\| 50.0		\| 50.0
			\| 59
			\| 50.9
			\| 76
			\| 65.5
	\|-		\|-
	\| Merged Properties		\| Merged Properties
			\| 13
			\| 7
			\| 53.8
			\| 7
			\| 53.8
	\| 7		\| 7
	~~\| 13~~
	\| 53.8		\| 53.8
	\|}		\|}

Kreis at 15:24, 14 September 2011

2011-09-14T15:24:15Z

← Older revision		Revision as of 17:24, 14 September 2011
Line 9:		Line 9:
	\|+ Mapped triples (Mappings from 3.5.1 DBpedia release)		\|+ Mapped triples (Mappings from 3.5.1 DBpedia release)
	\|-		\|-
	! rowspan="2" \| Category		! rowspan="3" \| Category
	! rowspan="2" \| Gold Triples		! rowspan="3" \| Gold Triples
	! colspan="2" \| DBpedia Release		! colspan="2" \| DBpedia Release
	\|-		\|-

Kreis at 15:23, 14 September 2011

2011-09-14T15:23:46Z

← Older revision		Revision as of 17:23, 14 September 2011
Line 9:		Line 9:
	\|+ Mapped triples (Mappings from 3.5.1 DBpedia release)		\|+ Mapped triples (Mappings from 3.5.1 DBpedia release)
	\|-		\|-
	! Category		! rowspan="2" \| Category
	! Gold Triples		! rowspan="2" \| Gold Triples
	! colspan="2" \| DBpedia Release		! colspan="2" \| DBpedia Release
	\|-		\|-
	!
	!
	! colspan="2" \| DBpedia 3.5.1		! colspan="2" \| DBpedia 3.5.1
	\|-		\|-
	!
	!
	! Mapped Triples		! Mapped Triples
	! %		! %

Kreis at 15:22, 14 September 2011

2011-09-14T15:22:48Z

← Older revision		Revision as of 17:22, 14 September 2011
Line 10:		Line 10:
	\|-		\|-
	! Category		! Category
	! colspan="3" \| DBpedia Release		! Gold Triples
			! colspan="2" \| DBpedia Release
	\|-		\|-
	!		!
Line 16:		Line 17:
	! colspan="2" \| DBpedia 3.5.1		! colspan="2" \| DBpedia 3.5.1
	\|-		\|-
	~~\| Total~~		!
	\| Mapped Triples		!
	~~\| Triples~~		! Mapped Triples
	\| %		! %
	\|-
	\|-		\|-
	\| Total		\| Total