mirror of
https://github.com/gsi-upm/sitc
synced 2024-11-24 23:42:29 +00:00
1853 lines
49 KiB
Plaintext
Executable File
1853 lines
49 KiB
Plaintext
Executable File
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "7276f055a8c504d3c80098c62ed41a4f",
|
|
"grade": false,
|
|
"grade_id": "cell-0bfe38f97f6ab2d2",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"<header style=\"width:100%;position:relative\">\n",
|
|
" <div style=\"width:80%;float:right;\">\n",
|
|
" <h1>Course Notes for Learning Intelligent Systems</h1>\n",
|
|
" <h3>Department of Telematic Engineering Systems</h3>\n",
|
|
" <h5>Universidad Politécnica de Madrid</h5>\n",
|
|
" </div>\n",
|
|
" <img style=\"width:15%;\" src=\"../logo.jpg\" alt=\"UPM\" />\n",
|
|
"</header>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Introduction to Linked Open Data\n",
|
|
"\n",
|
|
"This lecture provides a quick introduction to semantic queries in Python using SPARQL.\n",
|
|
"SPARQL is a semantic query language inspired by SQL.\n",
|
|
"\n",
|
|
"This is the first in a series of notebooks about SPARQL, which consists of:\n",
|
|
"\n",
|
|
"* This notebook, which introduces basic concepts using a small public dataset.\n",
|
|
"* [A notebook with queries to a custom dataset](02_SPARQL_Custom_Endpoint.ipynb), which links to the RDF exercises and it is out of the scope of this course. You can consult it if you are interested."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Objectives\n",
|
|
"\n",
|
|
"* Learning SPARQL and the Linked Data principles by defining queries to answer a set of problems of increasing difficulty\n",
|
|
"* Learning how to use integrated SPARQL editors and programming interfaces to SPARQL."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "99aecbad8f94966d92d72dc911d3ff99",
|
|
"grade": false,
|
|
"grade_id": "cell-4f8492996e74bf20",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"## Tools\n",
|
|
"\n",
|
|
"* This notebook\n",
|
|
"* External SPARQL editors (optional)\n",
|
|
" * YASGUI-GSI http://yasgui.cluster.gsi.dit.upm.es\n",
|
|
" * DBpedia virtuoso http://dbpedia.org/sparql\n",
|
|
"\n",
|
|
"Using the YASGUI-GSI editor has several advantages over other options.\n",
|
|
"It features:\n",
|
|
"\n",
|
|
"* Selection of data source, either by specifying the URL or by selecting from a dropdown menu\n",
|
|
"* Interactive query editing\n",
|
|
" * A set of pre-defined queries\n",
|
|
" * Syntax errors\n",
|
|
" * Auto-complete\n",
|
|
"* Data visualization\n",
|
|
" * Total number of results\n",
|
|
" * Different formats (table, pivot table, raw response, etc.)\n",
|
|
" * Pagination of results\n",
|
|
" * Search and filter results"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "99e3107f9987cdddae7866dded27f165",
|
|
"grade": false,
|
|
"grade_id": "cell-70ac24910356c3cf",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"## Instructions\n",
|
|
"\n",
|
|
"We will be using a semantic server, available at: http://fuseki.cluster.gsi.dit.upm.es/sitc.\n",
|
|
"\n",
|
|
"This server contains a dataset about [Beatles songs](http://www.snee.com/bobdc.blog/2017/11/sparql-queries-of-beatles-reco.html), which we will query with SPARQL.\n",
|
|
"\n",
|
|
"We will provide you some example code to get you started, the *question* you will have to answer using SPARQL, a template for the answer.\n",
|
|
"\n",
|
|
"After every query, you will find some python code to test the results of the query.\n",
|
|
"**Make sure you've run the tests before moving to the next exercise**.\n",
|
|
"If the test gives you an error, you've probably done something wrong.\n",
|
|
"You do not need to understand or modify the test code."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "1d332d3d11fd6b57f0ec0ac3c358c6cb",
|
|
"grade": false,
|
|
"grade_id": "cell-eb13908482825e42",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"For convenience, the examples in the notebook are executable (using the `%%sparql` magic command), and they are accompanied by some code to test the results.\n",
|
|
"If the tests pass, you probably got the answer right.\n",
|
|
"\n",
|
|
"**Run this line to enable the `%%sparql` magic command.**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "aca7c5538b8fc53e99c92e94e6818c83",
|
|
"grade": false,
|
|
"grade_id": "cell-b3f3d92fa2100c3d",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from helpers import sparql, solution, show_photos"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "e896b6560e45d5c385a43aa85e3523c7",
|
|
"grade": false,
|
|
"grade_id": "cell-04410e75828c388d",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"The `%%sparql` magic command will allow us to use SPARQL inside normal jupyter cells.\n",
|
|
"\n",
|
|
"For instance, the following code:\n",
|
|
"\n",
|
|
"```python \n",
|
|
"%%sparql http://dbpedia.org/sparql\n",
|
|
"\n",
|
|
"<MY QUERY>\n",
|
|
"``` \n",
|
|
"\n",
|
|
"Is the same as `run_query('<MY QUERY>', endpoint='http://dbpedia.org/sparql')` plus some additional steps, such as saving the results in a nice table format so that they can be used later and storing the results in a variable (`solution()`), which we will use in our tests.\n",
|
|
"\n",
|
|
"You do not need to worry about it, and **you can always use one of the suggested online editors if you wish**."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "96ca90572d6b275fa515c6b976115257",
|
|
"grade": false,
|
|
"grade_id": "cell-2a44c0da2c206d01",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"You can also use any other method to write your queries.\n",
|
|
"Just make sure to copy the working query back into the notebook so you can test it.\n",
|
|
"\n",
|
|
"You may find online query editors particularly useful.\n",
|
|
"In addition to running queries from your browser, they provide useful features such as syntax highlighting and autocompletion.\n",
|
|
"Some examples are:\n",
|
|
"\n",
|
|
"* DBpedia's virtuoso query editor https://dbpedia.org/sparql\n",
|
|
"* A javascript based client hosted at GSI: http://yasgui.cluster.gsi.dit.upm.es/\n",
|
|
"\n",
|
|
"[^1]: http://www.snee.com/bobdc.blog/2017/11/sparql-queries-of-beatles-reco.html"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "79c60bd3d4c13f380aae5778c5ce7245",
|
|
"grade": false,
|
|
"grade_id": "cell-d645128d3af18117",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"## Exercises\n",
|
|
"\n",
|
|
"The following exercises cover the basics of SPARQL with simple use cases."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "f7428fe79cd33383dfd3b09a0d951b6e",
|
|
"grade": false,
|
|
"grade_id": "cell-8391a5322a9ad4a7",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"#### First select - Exploring the dataset\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "f6b5da583694dd5cc9326c670830875d",
|
|
"grade": false,
|
|
"grade_id": "cell-4f56a152e4d70c02",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"Let's start with a simple query to explore the dataset using SPARQL.\n",
|
|
"We will get a list of the types of entities in the dataset.\n",
|
|
"\n",
|
|
"SPARQL syntax is similar to SQL, mixed with turtle.\n",
|
|
"A SPARQL query has two main parts: the `SELECT` block, which specifies what variables we want to get; and the `WHERE` block which, loosely speaking, defines how the variables will be obtained from the graph.\n",
|
|
"\n",
|
|
"In order to construct the `WHERE` block, we have to know the data we want to extract would be represented in Turtle.\n",
|
|
"\n",
|
|
"In particular, to write an entity and its type, we would write this triple:\n",
|
|
"\n",
|
|
"```turtle\n",
|
|
"<my_entity> a <type> .\n",
|
|
"```\n",
|
|
"\n",
|
|
"For example:\n",
|
|
"\n",
|
|
"```turtle\n",
|
|
"example:Timmy a example:Boy\n",
|
|
"```\n",
|
|
"\n",
|
|
"In SPARQL, the parts that we wish to extract are replaced with a variable (e.g. `?name`, `?type`).\n",
|
|
"Hence, we would have something like this:\n",
|
|
"\n",
|
|
"```turtle\n",
|
|
"?entity a ?type\n",
|
|
"```\n",
|
|
"\n",
|
|
"The name of the variable has no effect on the query, but you should use a sensible name.\n",
|
|
"In these notebooks, try to use the names provided in the templates, because they might be used in the tests.\n",
|
|
"\n",
|
|
"There are additional parts in the query.\n",
|
|
"For now, we will only cover the `LIMIT` statement, which limits the number of results we will get.\n",
|
|
"Using `LIMIT` is usually a good idea, especially when trying new queries, because the dataset may be too big. \n",
|
|
"\n",
|
|
"Using all these concepts, we will run our first query, to get the list of entities and their type:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "7a9dc62ab639143c9fc13593e50500d4",
|
|
"grade": false,
|
|
"grade_id": "cell-8ce8c954513f17e7",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"SELECT ?entity ?type\n",
|
|
"WHERE {\n",
|
|
" ?entity a ?type\n",
|
|
"}\n",
|
|
"LIMIT 10"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "d6a79c2f5fd005a9e15a8f67dcfd4784",
|
|
"grade": false,
|
|
"grade_id": "cell-3d6d622c717c3950",
|
|
"locked": true,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"source": [
|
|
"You can check that the results you got match our expectations:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"assert len(solution()['tuples']) == 10 # Make sure we got 10 results \n",
|
|
"assert len(solution()['columns']) >= 1 # In 2 columns (?entity and ?type)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now, use the same concepts to write a query that gets the **list of entities (subjects) and their properties (predicates)**.\n",
|
|
"\n",
|
|
"**Hint**: review the previous query. In there, we fixed a property (`a`, i.e. `rdfs:type`) and used a variable for the objects. Now we are insterested properties, regardless of the value (object)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "69e016b0224f410f03f6217ac30c03a8",
|
|
"grade": false,
|
|
"grade_id": "cell-6e904d692b5facad",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"SELECT ?entity ?prop\n",
|
|
"WHERE {\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}\n",
|
|
"LIMIT 100"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "97bd5d5383bd94a72c7452bc33e4b0f9",
|
|
"grade": true,
|
|
"grade_id": "cell-3fc0d3c43dfd04a3",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert len(s['tuples']) >= 100 # There are at least 100 results\n",
|
|
"assert 'entity' in s['columns'] # A column named entity exists\n",
|
|
"assert 'http://learningsparql.com/ns/musician/RaymondBrown' in s['columns']['entity'] # RaymondBrown is an entity"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Getting a list of DISTINCT types"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"To get a better grip of the dataset, we will get a list of types.\n",
|
|
"\n",
|
|
"We may try to do so with a simple query: "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"SELECT ?type\n",
|
|
"WHERE {\n",
|
|
" ?entity a ?type\n",
|
|
"}\n",
|
|
"LIMIT 10"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"However, this list has many duplicates.\n",
|
|
"In fact, we only get one type (`Musician`).\n",
|
|
"\n",
|
|
"To remove duplicates, we will need the `DISTINCT` statement, which only shows unique (distinct) rows:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"SELECT DISTINCT ?type\n",
|
|
"WHERE {\n",
|
|
" ?entity a ?type\n",
|
|
"}\n",
|
|
"LIMIT 100"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We should see only three types now (`Musician`, `Song`, and `Instrument`)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"assert 'type' in solution()['columns']\n",
|
|
"assert len(solution()['tuples']) == 3\n",
|
|
"assert 'http://learningsparql.com/ns/schema/Musician' in solution()['columns']['type']\n",
|
|
"assert 'http://learningsparql.com/ns/schema/Song' in solution()['columns']['type']\n",
|
|
"assert 'http://learningsparql.com/ns/schema/Instrument' in solution()['columns']['type']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now, **build a query to get the list of unique properties**:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "47c4f68e342ffe59a3804de7b6a3909b",
|
|
"grade": false,
|
|
"grade_id": "cell-e615f9a77c4bc9a5",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"SELECT DISTINCT ?property\n",
|
|
"WHERE {\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "c9ffeba2d4ffc3e0b95f15a0ec6012c5",
|
|
"grade": true,
|
|
"grade_id": "cell-9168718938ab7347",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"assert len(solution()['tuples']) == 182\n",
|
|
"assert 'http://learningsparql.com/ns/instrument/bass' in solution()['columns']['property']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Geting all properties for songs"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The `WHERE` statement can contain more than one line.\n",
|
|
"\n",
|
|
"For example, we can restrict the list of properties from the previous exercise, to only get properties of musicians:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT DISTINCT ?prop\n",
|
|
"WHERE {\n",
|
|
" ?song a s:Musician .\n",
|
|
" ?song ?prop ?value .\n",
|
|
"}\n",
|
|
"LIMIT 20"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"There should be two results:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"assert len(solution()['tuples']) == 2 # There are exactly two results"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Notice the use of prefixes, just like in turtle.\n",
|
|
"Also, these two options are equivalent:\n",
|
|
"\n",
|
|
"```turtle\n",
|
|
"?song a s:Musician ;\n",
|
|
" ?prop ?value .\n",
|
|
"\n",
|
|
"# And\n",
|
|
"\n",
|
|
"?song a s:Musician ;\n",
|
|
"?song ?prop ?value .\n",
|
|
"```\n",
|
|
"\n",
|
|
"The first one is just shorter to write.\n",
|
|
"\n",
|
|
"Alternatively, in this example we can also replace the properties we are not using with square brackets `[]`:\n",
|
|
"\n",
|
|
"```turtle\n",
|
|
"[] a s:Musician ;\n",
|
|
" ?prop [] .\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now, use the same concepts to get a list of **songs and properties**, without duplicates:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "8b0faf938efc1a64a70515da3c132605",
|
|
"grade": false,
|
|
"grade_id": "cell-0223a51f609edcf9",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"WHERE {\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}\n",
|
|
"LIMIT 20"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "e93d7336fd125d95996e60fd312a4e4d",
|
|
"grade": true,
|
|
"grade_id": "cell-3c7943c6382c62f5",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert len(set(s['tuples'])) == len(s['tuples']) # There are no duplicates\n",
|
|
"assert len(s['tuples']) >= 20"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Getting a list of song names"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In the previous exercise, we saw the properties for Songs.\n",
|
|
"One of them is `rdfs:label`, which gives a human readable name for the entity.\n",
|
|
"\n",
|
|
"Using `rdfs:label`, get a list of song names:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "271f2b194c2db4c558a46e8312b593e6",
|
|
"grade": false,
|
|
"grade_id": "cell-8f43547dd788bb33",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT ?name\n",
|
|
"WHERE {\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}\n",
|
|
"LIMIT 20"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "9f1f7cec8ce4674971543728ada86674",
|
|
"grade": true,
|
|
"grade_id": "cell-e13a1c921af2f6eb",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert 'Besame Mucho' in s['columns']['name']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Getting an ordered list of songs (ORDER BY)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The `ORDER BY` statement allows us to determine the way results will be sorted.\n",
|
|
"This makes it easier to find errors, or missing data.\n",
|
|
"\n",
|
|
"The syntax is the following:\n",
|
|
"\n",
|
|
"```sparql\n",
|
|
"\n",
|
|
"SELECT *\n",
|
|
"WHERE { ... }\n",
|
|
"ORDER BY <variable> <variable> ... DESC(<variable>) ASC(<variable>)\n",
|
|
"... other statements like LIMIT ...\n",
|
|
"```\n",
|
|
"\n",
|
|
"The results can be sorted in ascending or descending order, and using several variables."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Use `ORDER BY` to get a list of songs in **descending order**:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "9dcd9c6d51a61ac129cffa06e1463c66",
|
|
"grade": false,
|
|
"grade_id": "cell-a0f0b9d9b05c9631",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT ?name\n",
|
|
"WHERE {\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"LIMIT 50"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "a044b3fd6b8bd4e098bbe4d818cb4e9f",
|
|
"grade": true,
|
|
"grade_id": "cell-bc012ca9d7ad2867",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert len(s['tuples']) >= 20\n",
|
|
"assert s['columns']['name'][0][0] > s['columns']['name'][-1]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Get a list of musicians who collaborated in at least one song (Traversing the graph)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"From our inspection of the properties in previous exercises, we know that each song has a list of properties that link to musicians, and each musician has a name. For example:\n",
|
|
"\n",
|
|
"\n",
|
|
"```turtle\n",
|
|
"song:HeyJude a schema:Song ;\n",
|
|
" instrument:guitar musician:RingoStarr .\n",
|
|
"\n",
|
|
"musician:RingoStarr a schema:Musician ;\n",
|
|
" rdfs:label \"Ringo Starr\" .\n",
|
|
"```\n",
|
|
"\n",
|
|
"Using this structure, and the SPARQL statements you already know, to get the **names** of all musicians that collaborated in at least one song.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "9da7a62b6237078f5eab7e593a8eb590",
|
|
"grade": false,
|
|
"grade_id": "cell-523b963fa4e288d0",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT DISTINCT ?musician\n",
|
|
"WHERE {\n",
|
|
" ?song a s:Song .\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
" \n",
|
|
"}\n",
|
|
"ORDER BY ?name"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "c8e3a929faf2afa72207c6921382654c",
|
|
"grade": true,
|
|
"grade_id": "cell-aa9a4e18d6fda225",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert 'musician' in s['columns']\n",
|
|
"assert 'Paul McCartney' in s['columns']['musician']\n",
|
|
"assert 'Peter Coe' in s['columns']['musician']\n",
|
|
"assert len(solution()['tuples']) >= 200"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### In how many songs did Ringo collaborate? (COUNT)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"Results can be aggregated using different functions.\n",
|
|
"One of the simplest functions is `COUNT`.\n",
|
|
"The syntax for COUNT is:\n",
|
|
" \n",
|
|
"```sparql\n",
|
|
"SELECT (COUNT(?variable) as ?count_name)\n",
|
|
"```\n",
|
|
"\n",
|
|
"Use `COUNT` to get the number of songs in which Ringo collaborated."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "d8419711d2db43ad657e2658a1ea86c4",
|
|
"grade": false,
|
|
"grade_id": "cell-e89d08031e30b299",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"WHERE {\n",
|
|
" ?song a s:Song .\n",
|
|
" ?song ?instrument m:RingoStarr .\n",
|
|
"}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "29404e07edf639cdc0ce0d82e654ec31",
|
|
"grade": true,
|
|
"grade_id": "cell-903d2be00885e1d2",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"assert solution()['columns']['number'][0] == '412'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Getting the frequency of each instrument (GROUP BY)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Results can be grouped by one or more of the variables.\n",
|
|
"\n",
|
|
"Grouping is achieved with the `GROUP BY` statement. \n",
|
|
"The syntax for `GROUP BY` is:\n",
|
|
"\n",
|
|
" \n",
|
|
"```sparql\n",
|
|
"SELECT GROUP BY ?variable1 ?variable2 ...\n",
|
|
"```\n",
|
|
"\n",
|
|
"Once results are grouped, they can be aggregated using any aggregation function, such as `COUNT`.\n",
|
|
"\n",
|
|
"Using `GROUP BY` and `COUNT`, get the count of songs that use each instrument:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "7a0a7206384e7e1d9eb4450dd9e9871f",
|
|
"grade": false,
|
|
"grade_id": "cell-1429e4eb5400dbc7",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT ?instrument (COUNT(?song) as ?number)\n",
|
|
"WHERE {\n",
|
|
" ?song a s:Song .\n",
|
|
" ?song ?instrument m:RingoStarr .\n",
|
|
"}\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"ORDER BY DESC(?number)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "bd4dc379fea969d513be0ea97ee75922",
|
|
"grade": true,
|
|
"grade_id": "cell-907aaf6001e27e50",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert len(s['tuples']) == 37\n",
|
|
"assert s['columns']['number'][-1] == '1'\n",
|
|
"assert s['columns']['number'][0] == '233'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### How many different instruments are there in every song?"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We can use other keywords inside our aggregation.\n",
|
|
"For example, we could use `DISTINCT` to remove duplicates before aggregating.\n",
|
|
"\n",
|
|
"Here is an example, which shows the number of songs each musician collaborated in.\n",
|
|
"It has to use `DISTINCT` because some artists play multiple instruments in a song."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT ?artist (COUNT(DISTINCT ?song) as ?number)\n",
|
|
"WHERE {\n",
|
|
" ?artist a s:Musician .\n",
|
|
" ?song ?instrument ?artist .\n",
|
|
"}\n",
|
|
"GROUP BY ?artist\n",
|
|
"ORDER BY DESC(?number)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now, use the same principle to get the count of **different** instruments in each song.\n",
|
|
"Some songs have several musicians playing the same instrument, but we only care about *different* instruments in each song.\n",
|
|
"\n",
|
|
"Use `?number` for the count."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "4a231b4d6874dad435512b988c17c39e",
|
|
"grade": false,
|
|
"grade_id": "cell-ee208c762d00da9c",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"WHERE {\n",
|
|
" [] a s:Song ;\n",
|
|
" rdfs:label ?song ;\n",
|
|
" ?instrument ?musician .\n",
|
|
"}\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"ORDER BY DESC(?number)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "8118099bf14d9f0eb241c4d93ea6f0b9",
|
|
"grade": true,
|
|
"grade_id": "cell-ddeec32b8ac3d894",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert s['columns']['number'][0] == '27'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Who is the vocalist in every song? (using OPTIONAL)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this exercise, we will get a list of songs and their vocalists.\n",
|
|
"\n",
|
|
"We coul start with this query:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT ?song ?vocalist\n",
|
|
"WHERE {\n",
|
|
" ?song a s:Song .\n",
|
|
" ?song i:vocals ?vocalist\n",
|
|
"}\n",
|
|
"LIMIT 100"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"However, there are some songs that do not have a vocalist (at least, in the dataset).\n",
|
|
"Those songs will not appear in the list above, because we they do not match part of the `WHERE` clause.\n",
|
|
"\n",
|
|
"In these cases, we can specify optional values in a query using the `OPTIONAL` keyword.\n",
|
|
"When a set of clauses are inside an OPTIONAL group, the SPARQL endpoint will try to use them in the query.\n",
|
|
"If there are no results for that part of the query, the variables it specifies will not be bound (i.e. they will be empty).\n",
|
|
"\n",
|
|
"To exemplify this, we can use a property that **does not exist in the dataset**:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT ?song ?musician\n",
|
|
"WHERE {\n",
|
|
" ?song a s:Song .\n",
|
|
" OPTIONAL {\n",
|
|
" ?song i:a_made_up_instrument ?musician\n",
|
|
" }\n",
|
|
"}\n",
|
|
"LIMIT 100"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Although the property does not exist, the query will still return all the songs.\n",
|
|
"In the column for our instrument, it returns an empty value.\n",
|
|
"\n",
|
|
"Now, use the same concept, to get a list of the **names** of the vocalists (if any) in each song."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "4b0a0854457c37640aad67f375ed3a17",
|
|
"grade": false,
|
|
"grade_id": "cell-dcd68c45c1608a28",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT ?song ?vocalist\n",
|
|
"WHERE {\n",
|
|
" ?s a s:Song .\n",
|
|
" ?s rdfs:label ?song .\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}\n",
|
|
"LIMIT 100"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "f7122b2284b5d59d59ce4a2925f0bb21",
|
|
"grade": true,
|
|
"grade_id": "cell-1e706b9c1c1331bc",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert 'Paul McCartney' in s['columns']['vocalist']\n",
|
|
"assert 'Paul McCartney' in s['columns']['vocalist']\n",
|
|
"assert ('Besame Mucho', 'Paul McCartney') in s['tuples']\n",
|
|
"assert '' in s['columns']['vocalist'] # Some songs do not have a vocalist"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### What songs do not have a vocalist? (Bound)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now we only want to list those songs that **do not** have a vocalist.\n",
|
|
"\n",
|
|
"To do so, we can copy the query from the previous exercise, and filter the results with the `BOUND` function.\n",
|
|
"\n",
|
|
"`BOUND` will return `true` if the variable has a value, and `false` otherwise.\n",
|
|
"\n",
|
|
"This is very useful for two purposes.\n",
|
|
"Firstly, it allows us to look for patterns that **do not occur** in the graph, such as missing properties.\n",
|
|
"For instance, we could search for the authors with missing birth information so we can add it.\n",
|
|
"Secondly, we can use bound in filters to get conditional filters.\n",
|
|
"\n",
|
|
"Add a filter below to only get songs without a vocalist:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "09621e7af911faf39a834e8281bc6d1f",
|
|
"grade": false,
|
|
"grade_id": "cell-0c7cc924a13d792a",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\n",
|
|
"\n",
|
|
"SELECT ?song\n",
|
|
"WHERE {\n",
|
|
" ?s a s:Song .\n",
|
|
" ?s rdfs:label ?song .\n",
|
|
" OPTIONAL {\n",
|
|
" ?s i:vocals ?vocalist\n",
|
|
" }\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}\n",
|
|
"LIMIT 100"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "cebff8ce42f3f36923e81e083a23d24c",
|
|
"grade": true,
|
|
"grade_id": "cell-2541abc93ab4d506",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert len(s['tuples']) == 23"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Who played guitar OR bass in the most songs? (Advanced FILTER with GROUP)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this exercise, we want a table with the name of musicians that played either the guitar (`i:guitar`) or the bass (`i:bass`), the instrument they played, and the times they played it.\n",
|
|
"\n",
|
|
"If a musician played both instruments, it should appear twice."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "ea9797f3b2d001ea41d7fa7a5170d5fb",
|
|
"grade": false,
|
|
"grade_id": "cell-d750b6d64c6aa0a7",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> \n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"\n",
|
|
"SELECT ?musician ?instrument (COUNT(DISTINCT ?song) AS ?number)\n",
|
|
"WHERE {\n",
|
|
" ?song ?ins ?player .\n",
|
|
" ?ins rdfs:label ?instrument .\n",
|
|
" ?player rdfs:label ?musician .\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"\n",
|
|
"ORDER BY DESC(?instrument) DESC(?number)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert ('George Harrison', 'guitar', '27') in s['tuples']\n",
|
|
"assert ('Stuart Sutcliffe', 'bass', '3') in s['tuples']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Who played the most instruments? (Advanced FILTER II)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now, count how many instruments each musician have played in a song.\n",
|
|
"\n",
|
|
"**Do not count lead (`i:vocals`) or backing vocals (`i:backingvocals`) as instruments**."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "2d82df272d43f678d3b19bf0b41530c1",
|
|
"grade": false,
|
|
"grade_id": "cell-2f5aa516f8191787",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> \n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"WHERE {\n",
|
|
" ?song ?ins ?player .\n",
|
|
" ?ins rdfs:label ?instrument .\n",
|
|
" ?player rdfs:label ?musician .\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}\n",
|
|
"GROUP BY ?musician\n",
|
|
"ORDER BY DESC(?instrument) DESC(?number)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "bc83dd9577c9111b1f0ef5bd40c4ec08",
|
|
"grade": true,
|
|
"grade_id": "cell-bcd0f7e26b6c11c2",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert ('John Lennon', '52') in s['tuples']\n",
|
|
"assert ('Andy White', '2') in s['tuples']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Which songs had Ringo in dums OR Lennon in lead vocals? (UNION)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We can merge the results of several queries, just like using `JOIN` in SQL.\n",
|
|
"The keyword in SPARQL is `UNION`, because we are merging graphs.\n",
|
|
"\n",
|
|
"`UNION` is useful in many situations.\n",
|
|
"For instance, when there are equivalent properties, or when you want to use two search terms and FILTER would be too inefficient.\n",
|
|
"\n",
|
|
"The syntax is as follows:\n",
|
|
"\n",
|
|
"```sparql\n",
|
|
"SELECT ?title\n",
|
|
"WHERE {\n",
|
|
" { ?book dc10:title ?title }\n",
|
|
" UNION\n",
|
|
" { ?book dc11:title ?title }\n",
|
|
" \n",
|
|
" ... REST OF YOUR QUERY ...\n",
|
|
"\n",
|
|
"}\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "a1e20e2be817a592683dea89eed0120e",
|
|
"grade": false,
|
|
"grade_id": "cell-d3a742bd87d9c793",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> \n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"\n",
|
|
"SELECT DISTINCT ?song\n",
|
|
"WHERE {\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"}"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "087630476d73bb415b065fafbd6024f0",
|
|
"grade": true,
|
|
"grade_id": "cell-409402df0e801d09",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"assert len(solution()['tuples']) == 246"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### In how many songs has each musician collaborated at least 10 times? (HAVING)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"You can filter results after an aggregation, using the `HAVING` statement.\n",
|
|
"Its syntax is:\n",
|
|
" \n",
|
|
"\n",
|
|
"```sparql\n",
|
|
"SELECT ...\n",
|
|
"WHERE ...\n",
|
|
"GROUP BY ...\n",
|
|
"HAVING (<statement>)\n",
|
|
"```\n",
|
|
"\n",
|
|
"e.g.\n",
|
|
"\n",
|
|
"```sparql\n",
|
|
"HAVING (?count > 10)\n",
|
|
"```\n",
|
|
"\n",
|
|
"Use this new statement to get the list of artists that played at least 10 times with the Beatlest, and the number of times they did:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "1d2cb88412c89c35861a4f9fccea3bf2",
|
|
"grade": false,
|
|
"grade_id": "cell-9d1ec854eb530235",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> \n",
|
|
"\n",
|
|
"SELECT ?musician (COUNT(DISTINCT ?song) AS ?number) \n",
|
|
"WHERE {\n",
|
|
" ?song ?instrument [\n",
|
|
" rdfs:label ?musician \n",
|
|
" ]\n",
|
|
"}\n",
|
|
"GROUP BY ?musician\n",
|
|
"# YOUR ANSWER HERE\n",
|
|
"ORDER BY DESC(?number)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"editable": false,
|
|
"nbgrader": {
|
|
"checksum": "aa20aa4d11632ea5bd6004df3187d979",
|
|
"grade": true,
|
|
"grade_id": "cell-a79c688b4566dbe8",
|
|
"locked": true,
|
|
"points": 0,
|
|
"schema_version": 1,
|
|
"solution": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"s = solution()\n",
|
|
"assert len(s['tuples']) == 7\n",
|
|
"assert s['columns']['musician'][0] == 'Paul McCartney'\n",
|
|
"assert s['columns']['musician'][-1] == 'Mal Evans'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## **Optional** exercises"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"These are additional exercises that can be solved with more advanced concepts.\n",
|
|
"\n",
|
|
"If you are curious, you could also check the notebook on Advanced SPARQL concepts."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### What instruments could each musician play? (GROUP_CONCAT)\n",
|
|
"\n",
|
|
"\n",
|
|
"Another option to aggregate results is to concatenate them.\n",
|
|
"You can do so with:\n",
|
|
"\n",
|
|
"```sparql\n",
|
|
"GROUP_CONCAT(?name; separator=\",\")\n",
|
|
"```\n",
|
|
"\n",
|
|
"Using `GROUP_CONCAT`, get a list of the instruments that each musician could play.\n",
|
|
"\n",
|
|
"You can consult how to use GROUP_CONCAT [here](https://www.w3.org/TR/sparql11-query/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "508b7f8656e849838aa93cd38f1c6635",
|
|
"grade": false,
|
|
"grade_id": "cell-7ea1f5154cdd8324",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> \n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"\n",
|
|
"# YOUR ANSWER HERE"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### What types of vocals are there? (REGEX)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In one of the exercises, we excluded lead and backing vocals from the list of instruments.\n",
|
|
"However, are those the only types of vocals?\n",
|
|
"\n",
|
|
"You can check if a string or URI matches a regular expression with `regex(?variable, \"<regex>\", \"i\")`.\n",
|
|
"\n",
|
|
"The documentation for regular expressions in SPARQL is [here](https://www.w3.org/TR/rdf-sparql-query/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"deletable": false,
|
|
"nbgrader": {
|
|
"checksum": "cff1f9c034393f8af055e1f930d5fe32",
|
|
"grade": false,
|
|
"grade_id": "cell-b6bee887a1b1fc60",
|
|
"locked": false,
|
|
"schema_version": 1,
|
|
"solution": true
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%sparql http://fuseki.cluster.gsi.dit.upm.es/sitc/\n",
|
|
"PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> \n",
|
|
"PREFIX s: <http://learningsparql.com/ns/schema/>\n",
|
|
"PREFIX i: <http://learningsparql.com/ns/instrument/>\n",
|
|
"PREFIX m: <http://learningsparql.com/ns/musician/>\n",
|
|
"\n",
|
|
"# YOUR ANSWER HERE"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## References"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"* [SPARQL queries of Beatles recording sessions](http://www.snee.com/bobdc.blog/2017/11/sparql-queries-of-beatles-reco.html)\n",
|
|
"* [RDFLib documentation](https://rdflib.readthedocs.io/en/stable/).\n",
|
|
"* [Wikidata Query Service query examples](https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/queries/examples)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Licence\n",
|
|
"The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/). \n",
|
|
"\n",
|
|
"© Universidad Politécnica de Madrid."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.7.2"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|