{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "![](files/images/EscUpmPolit_p.gif \"UPM\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
\n", "

Course Notes for Learning Intelligent Systems

\n", "

Department of Telematic Engineering Systems

\n", "
Universidad Politécnica de Madrid. © Carlos A. Iglesias
\n", "
\n", " \"UPM\"\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction\n", "\n", "This lecture provides an introduction to RDF and the query language SPARQL.\n", "\n", "This is the first in a series of notebooks about SPARQL, which consists of:\n", "\n", "* This notebook, which basic concepts of RDF and SPARQL\n", "* [A notebook](01_SPARQL_Introduction.ipynb) that provides an introduction of SPARQL through a collection of progressively more difficult exercises]\n", "* [A notebook](02_SPARQL_Custom_Endpoint.ipynb) with queries to a custom dataset, which links to the RDF exercises and it is out of the scope of this course. You can consult it if you are interested." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# RDF basics\n", "This section is taken from [[1](#1), [2](#2)].\n", "\n", "RDF allows us to make statements about resources. The format of these statements is simple. A statement always has the following structure:\n", "\n", " \n", " \n", "An RDF statement expresses a relationship between two resources. The **subject** and the **object** represent the two resources being related; the **predicate** represents the nature of their relationship. The relationship is phrased in a directional way (from subject to object) and is called in RDF a **property**. Because RDF statements consist of three elements they are called **triples**.\n", "\n", "Here are examples of RDF triples (informally expressed in pseudocode):\n", "\n", " .\n", " .\n", " \n", "Resources are identified by IRIs, which can appear in all three positions of a triple. For example, the IRI for Leonardo da Vinci in DBpedia is:\n", "\n", " \n", "\n", "IRIs can be abbreviated as *prefixed names*. For example, \n", " PREFIX dbr: \n", " \n", " \n", "Objects can be literals: \n", " * strings (e.g., \"plain string\" or \"string with language\"@en)\n", " * numbers (e.g., \"13.4\"^^xsd:float)\n", " * dates (e.g., )\n", " * booleans\n", " * etc.\n", " \n", "RDF data is stored in RDF repositories that expose SPARQL endpoints. Let's query one of the most famous RDF repositories: dbpedia. First, we should learn how to execute SPARQL in a notebook." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Executing SPARQL in a notebook\n", "There are several ways to execute SPARQL in a notebook. The most popular ones are:\n", "* using a SPARQL kernel [sparql kernel](https://github.com/paulovn/sparql-kernel) instead of the Python3 kernel\n", "* using the [graph notebook package](https://pypi.org/project/graph-notebook/)\n", "* using libraries such as [sparql-client](https://pypi.org/project/sparql-client/) or [rdflib](https://rdflib.dev/sparqlwrapper/) that enable executing SPARQL within a Python3 kernel\n", "* using other libraries. In our case, a light library has been developed (the file helpers.py) for accessing SPARQL endpoints using an HTTP connection.\n", "\n", "We are going to use the last option to avoid installing new packages.\n", "\n", "For using the library, you need:\n", "1. Import sparql from helpers (the file helpers.py available in the github repository)\n", "2. Use the magic command '%%sparql' to indicate the SPARQL endpoint and then the SPARQL code.\n", "\n", "Let's try it!\n", "\n", "# Queries agains DBPedia\n", "\n", "We are going to execute an SPARQL query agains DBPedia. This section is based on [[8](#8)].\n", "\n", "First, we just create a query to retrieve arbitrary triples (subject, predicate, object) without any restriction (only that we want to limit to 10 results)." ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [], "source": [ "from helpers import sparql" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
spo
http://www.openlinksw.com/virtrdf-data-formats#default-iidhttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#default-iid-nullablehttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#default-iid-nonblankhttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#default-iid-nonblank-nullablehttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#defaulthttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#default-nullablehttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#sql-varcharhttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-nullablehttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-dthttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
http://www.openlinksw.com/virtrdf-data-formats#sql-varchar-dt-nullablehttp://www.w3.org/1999/02/22-rdf-syntax-ns#typehttp://www.openlinksw.com/schemas/virtrdf#QuadMapFormat
" ], "text/plain": [ "" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://live.dbpedia.org/sparql\n", "\n", "SELECT ?s ?p ?o\n", "WHERE {\n", " ?s ?p ?o\n", "}\n", "LIMIT 10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Well, it worked, but the results are not particulary interesting. \n", "Let's search for a famous football player, Fernando Torres." ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
athlete
http://dbpedia.org/resource/Fernando_Torres
" ], "text/plain": [ "" ] }, "execution_count": 46, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://live.dbpedia.org/sparql\n", "\n", "SELECT *\n", "WHERE\n", " {\n", " ?athlete rdfs:label \"Fernando Torres\"@en \n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Amazing. Go to http://dbpedia.org/resource/Fernando_Torres and you will see all the information available about Fernando Torres. Pay attention to the names of predicates to be able to create new queries. For example, we are interesting in knowing where Fernando Torres was born.\n", "\n", "Let's go!" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
athletebirthPlace
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/Spain_national_football_team
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/Fuenlabrada
" ], "text/plain": [ "" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://dbpedia.org/sparql\n", "\n", "PREFIX dbo: \n", "\n", "SELECT *\n", "WHERE\n", " {\n", " ?athlete rdfs:label \"Fernando Torres\"@en ;\n", " dbo:birthPlace ?birthPlace . \n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Observe the SPARQL query:\n", "* PREFIX section URIs of vocabularies and the prefix used below, to avoid long IRIs\n", "* SELECT section: variables we want to return (* is an abbreviation that selects all of the variables in a query)\n", "* WHERE triple pattern: triples where some elements are variables. These variables are bound during the query processing process and bounded variables are returned.\n", "\n", "Pay attention to the WHERE section. Since both triple patterns share the same subject, we omit it in the second one, and link both with \" ;\". Each triple pattern should finish with a \" .\" (the last pattern can omit this). Don't forget the space before \";\" and \".\".\n", "\n", "The result is interesting, we know he was born in Fuenlabrada, but we see an additional (wrong) value, the Spanish national football team. The conversion process from Wikipedia to DBPedia should still be tuned :).\n", "\n", "We can 'fix' it, by adding some information. We want only municipalities as a result. Let's see!\n" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
athletebirthPlace
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/Fuenlabrada
" ], "text/plain": [ "" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://dbpedia.org/sparql\n", "\n", "PREFIX dbo: \n", "PREFIX dbr: \n", "\n", "SELECT *\n", "WHERE\n", " {\n", " ?athlete rdfs:label \"Fernando Torres\"@en ;\n", " dbo:birthPlace ?birthPlace .\n", " ?birthPlace dbo:type dbr:Municipalities_of_Spain \n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Great. Now it looks better.\n", "Do you know of Fuenlabrada is a big city? Let's query!\n", "\n", "Hint: search (as previously) the subject / object / predicate nodes in the RDF graph (http://dbpedia.org/resource/Fuenlabrada).\n" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
area
3.941e+07
" ], "text/plain": [ "" ] }, "execution_count": 49, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://dbpedia.org/sparql\n", "\n", "PREFIX dbo: \n", "PREFIX dbr: \n", "\n", "SELECT *\n", "WHERE\n", " {\n", " dbr:Fuenlabrada dbo:areaTotal ?area \n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Well, it shows 39.1 km$^2$. Let's go back to know more about Fernando Torres. We would want to retrieve the name of the city where he was born instead of the IRI. Let's try!" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
playerbirthPlaceplaceName
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/Fuenlabradaفوينلابرادا
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/Fuenlabradaフエンラブラダ
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/Fuenlabrada푸엔라브라다
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaФуэнлабрада
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaФуенлабрада
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/Fuenlabrada富恩拉夫拉达
" ], "text/plain": [ "" ] }, "execution_count": 51, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://dbpedia.org/sparql\n", "\n", "PREFIX dbo: \n", "PREFIX dbp: \n", "\n", "SELECT *\n", "WHERE\n", " {\n", " ?player rdfs:label \"Fernando Torres\"@en ;\n", " dbo:birthPlace ?birthPlace .\n", " ?birthPlace dbo:type dbr:Municipalities_of_Spain ;\n", " rdfs:label ?placeName \n", " \n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Well, we are almost there. We see that we receive the city name in many languages. We want just the English name. Let's filter!" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
playerbirthPlaceplaceName
http://dbpedia.org/resource/Fernando_Torreshttp://dbpedia.org/resource/FuenlabradaFuenlabrada
" ], "text/plain": [ "" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://dbpedia.org/sparql\n", "\n", "PREFIX dbo: \n", "PREFIX dbp: \n", "\n", "SELECT *\n", "WHERE\n", " {\n", " ?player rdfs:label \"Fernando Torres\"@en ;\n", " dbo:birthPlace ?birthPlace .\n", " ?birthPlace dbo:type dbr:Municipalities_of_Spain ;\n", " rdfs:label ?placeName .\n", " FILTER ( LANG ( ?placeName ) = 'en' )\n", " \n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Awesome. Let's tune a bit more. We only want two results: Fernando's birth date and birth place (name). Let's go!" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
birthDateplaceName
1984-03-20Fuenlabrada
" ], "text/plain": [ "" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://dbpedia.org/sparql\n", "\n", "PREFIX dbo: \n", "PREFIX dbp: \n", "\n", "SELECT ?birthDate, ?placeName\n", "WHERE\n", " {\n", " ?player rdfs:label \"Fernando Torres\"@en ;\n", " dbo:birthDate ?birthDate ;\n", " dbo:birthPlace ?birthPlace .\n", " ?birthPlace dbo:type dbr:Municipalities_of_Spain ;\n", " rdfs:label ?placeName .\n", " FILTER ( LANG ( ?placeName ) = 'en' )\n", " \n", " }" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Great :). Are there many football players born in Fuenlabrada? Let's query!" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
player
http://dbpedia.org/resource/Luismi_(footballer,_born_1979)
http://dbpedia.org/resource/Óscar_Miñambres
http://dbpedia.org/resource/Tachi_(footballer)
http://dbpedia.org/resource/Fernando_Torres
" ], "text/plain": [ "" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sparql https://dbpedia.org/sparql\n", "\n", "PREFIX dbo: \n", "PREFIX dbp: \n", "\n", "SELECT *\n", "WHERE\n", " {\n", " ?player a dbo:SoccerPlayer ; \n", " dbo:birthPlace dbr:Fuenlabrada .\n", " \n", " }" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Well, not that many. Observe we have used 'a' (it is an abbreviation for rdf:type, both can be used).\n", "\n", "If you want additional examples, you can follow the notebook by [Shawn Graham](https://github.com/o-date/sparql-and-lod/blob/master/sparql-intro.ipynb), which is based on the SPARQL tutorial by Matthew Lincoln, available [here in English](https://programminghistorian.org/en/lessons/retired/graph-databases-and-SPARQL) and [here in Spanish](https://programminghistorian.org/es/lecciones/retirada/sparql-datos-abiertos-enlazados]). You have also a local copy of these tutorials together with this notebook [here in English](https://htmlpreview.github.io/?https://github.com/gsi-upm/sitc/blob/master/lod/tutorial/graph-databases-and-SPARQL.html) and [here in Spanish](https://htmlpreview.github.io/?https://github.com/gsi-upm/sitc/blob/master/lod/tutorial/sparql-datos-abiertos-enlazados.html). \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## References" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "* [1] [SPARQL by Example. A Tutorial. Lee Feigenbaum. W3C, 2009](https://www.w3.org/2009/Talks/0615-qbe/#q1)\n", "* [2] [RDF Primer W3C](https://www.w3.org/TR/rdf11-primer/)\n", "* [3] [SPARQL queries of Beatles recording sessions](http://www.snee.com/bobdc.blog/2017/11/sparql-queries-of-beatles-reco.html)\n", "* [4] [RDFLib documentation](https://rdflib.readthedocs.io/en/stable/).\n", "* [5] [Wikidata Query Service query examples](https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/queries/examples)\n", "* [6] [RDF Graph Data Model. Learn about the RDF graph model used by Stardog.](https://www.stardog.com/tutorials/data-model)\n", "* [7] [Learn SPARQL Write Knowledge Graph queries using SPARQL with step-by-step examples.](https://www.stardog.com/tutorials/sparql/)\n", "* [8] [Running Basic SPARQL Queries Against DBpedia.](https://medium.com/virtuoso-blog/dbpedia-basic-queries-bc1ac172cc09)\n", "* [9] [Intro SPARQL based on painters.](https://github.com/o-date/sparql-and-lod/blob/master/sparql-intro.ipynb)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Licence\n", "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/). \n", "\n", "© Carlos A. Iglesias, Universidad Politécnica de Madrid." ] } ], "metadata": { "datacleaner": { "position": { "top": "50px" }, "python": { "varRefreshCmd": "try:\n print(_datacleaner.dataframe_metadata())\nexcept:\n print([])" }, "window_display": false }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false } }, "nbformat": 4, "nbformat_minor": 1 }