Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get all Wikipedia Infobox Templates and all Pages using them

Given a Wikipedia page like Wikipedia: Stack Overflow there are often Infoboxes (mostly on the right hand at the top of the page). Example screenshot:

Stackoverflow Infobox at Wikipedia

  1. DBPedia lists all these attributes as RDF triples. You can see the example at DBPedia: Stack Overflow. There you see the property dbpprop:wikiPageUsesTemplate with the value dbpedia:Template:Infobox_website which is interesting. I want to know which Wikipedia pages use this template. How can i do that and list all pages which use the Infobox_website template? Preferably with a SPARQL query but i am open to other easy solutions.

  2. Next thing is a list of all Infobox Templates. Wikipedia: Category Infobox Templates shows the hierarchy of the desired Wikipedia categories - that looks like what i am seeking. But i want all of these in a machine readable format, on one page. Maybe DBPedia is the right thing here too? At DBPedia: Category Infox Templates and DBPedia: INFOBOX i find very few information. But these are looking very promising. How can i use SPARQL to find all Infobox Types so that i can do step 1 repeatedly for each of them?

You can use this for testing the SPARQL queries: http://dbpedia.org/snorql/

Update 1

I seem to have solved problem number 1: SPARQL: list all pages with Infobox_website

Update 2

Also, this seems to be the query for problem number 2: SPARQL: list all Infoboxes

like image 352
Alp Avatar asked Nov 03 '11 18:11

Alp


People also ask

How do I add infobox?

Click Edit to enter Visual Editor. Click on the Insert dropdown menu, and the option for Template. Type Infobox Wikipedia user and select the first option. Then click 'Add template'.

How do I edit infobox?

The infobox preview's items can be configured by clicking on them and using the options pane in the right sidebar. The items can also be rearranged by dragging them around. To remove an item, click on it and use the trash can button on the right. There are 4 types of items in the visual infobox editor.


2 Answers

Ok, since i seem to have found a solution (most probably not the best) i want to share them.

1) This SPARQL query can be used to find all pages that include a specific Infobox type:

SELECT * WHERE { ?page dbpedia2:wikiPageUsesTemplate <http://dbpedia.org/resource/Template:Infobox_website> . ?page dbpedia2:name ?name . }

Link at SNORQL


2) This SPARQL query can be used to find all Infobox types:

SELECT DISTINCT ?template WHERE { ?page dbpedia2:wikiPageUsesTemplate ?template . FILTER (regex(?template, "Infobox")) . } ORDER BY ?template

Link at SNORQL

like image 199
Alp Avatar answered Oct 13 '22 19:10

Alp


The previous answers seem to have stopped working. Only a small change is required to get them working at the new dbpedia query endpoint at http://live.dbpedia.org/sparql though.

To get a list of all of the pages and the templates that they use this query works:

SELECT * WHERE {  ?page  dbpprop:wikiPageUsesTemplate ?template . }

See results (limited to 100)

If you're looking for a specific template:

SELECT * WHERE {  
   ?page  
   dbpprop:wikiPageUsesTemplate 
   <http://dbpedia.org/resource/Template:Infobox_website> . 
}

See results

And for my use case I'm interested in the Wikipedia URL rather than the DBPedia page, so I'm using this query:

SELECT ?wikipedia_url WHERE {  
   ?page  
   dbpprop:wikiPageUsesTemplate 
   <http://dbpedia.org/resource/Template:Infobox_website> . 
   ?page foaf:isPrimaryTopicOf ?wikipedia_url .
}

See results

I'm also using curl to pull the results into a script:

$ curl -s "http://live.dbpedia.org/sparql?default-graph-uri=http%3A%2F%2Fdbpedia.org&query=SELECT+%3Fwikipedia_url+WHERE+%7B+%0D%0A%09+%3Fpage+%0D%0A%09+dbpprop%3AwikiPageUsesTemplate+%0D%0A%09+%3Chttp%3A%2F%2Fdbpedia.org%2Fresource%2FTemplate%3AInfobox_website%3E+.+%0D%0A+%3Fpage+foaf%3AisPrimaryTopicOf+%3Fwikipedia_url+.%0D%0A%0D%0A%09%7D&format=text%2Ftab-separated-values" \
| tr -d \" | grep -v "^wikipedia_url$" | head
http://en.wikipedia.org/wiki/U.S._News_&_World_Report
http://en.wikipedia.org/wiki/FriendFinder
http://en.wikipedia.org/wiki/Debkafile
http://en.wikipedia.org/wiki/GTPlanet
http://en.wikipedia.org/wiki/Lithuanian_Wikipedia
http://en.wikipedia.org/wiki/Connexions
http://en.wikipedia.org/wiki/Hypno5ive
http://en.wikipedia.org/wiki/Scoop_(website)
http://en.wikipedia.org/wiki/Bhoomi_(software)
http://en.wikipedia.org/wiki/Brainwashed_(website)

I'm not sure if this gives the full result set though, because it returns 1698 results whereas wmflabs.org seems to suggest there should be 4439.


For the second part of your question, only a small change is needed from the previous query to get a list of all templates:

SELECT DISTINCT ?template WHERE { 
    ?page  
    dbpprop:wikiPageUsesTemplate  
    ?template . 
    FILTER (regex(?template, "Infobox")) . 
} ORDER BY ?template

See results

like image 29
Ben Dowling Avatar answered Oct 13 '22 17:10

Ben Dowling