Update qlearning.py

Updated image link

README.md

@@ -1,7 +1,7 @@
 # sitc
 Exercises for Intelligent Systems Course at Universidad Politécnica de Madrid, Telecommunication Engineering School. This material is used in the subjects
-- CDAW (Ciencia de datos y aprendizaje en automático en la web de datos) - Master Universitario de Ingeniería de Telecomunicación (MUIT)
-- ABID (Analítica de Big Data) - Master Universitario en Ingeniera de Redes y Servicios Telemáticos)
+- CDAW (Ciencia de datos y aprendizaje en automático en la web de datos) - Máster Universitario de Ingeniería de Telecomunicación (MUIT)
+- ABID (Analítica de Big Data) - Máster Universitario en Ingeniería de Redes y Servicios Telemáticos) 
 
 For following this course:
 - Follow the instructions to install the environment: https://github.com/gsi-upm/sitc/blob/master/python/1_1_Notebooks.ipynb (Just install 'conda')
@@ -10,12 +10,12 @@ For following this course:
 
 Topics
 * Python: a quick introduction to Python
-* ML-1: introduction to machine learning with scikit-learn
-* ML-2: introduction to machine learning with pandas and scikit-learn
-* ML-21: preprocessing and visualizatoin
-* ML-3: introduction to machine learning. Neural Computing
-* ML-4: introduction to Evolutionary Computing
-* ML-5: introduction to Reinforcement Learning
-* NLP: introduction to NLP
+* ML-1: Introduction to machine learning with scikit-learn
+* ML-2: Introduction to machine learning with pandas and scikit-learn
+* ML-21: Preprocessing and visualization
+* ML-3: Introduction to machine learning. Neural Computing
+* ML-4: Introduction to Evolutionary Computing
+* ML-5: Introduction to Reinforcement Learning
+* NLP: Introduction to NLP
 * LOD: Linked Open Data, exercises and example code
 * SNA: Social Network Analysis

ml5/2_6_0_Intro_RL.ipynb

@@ -27,7 +27,7 @@
    "source": [
     "# Introduction to Machine Learning V\n",
     " \n",
-    "In this lab session, we are going to get more insight about reinforcemente learning and, in particular, about Q-learning.\n",
+    "In this lab session, we are going to get more insight about reinforcement learning and, in particular, about Q-learning.\n",
     "\n",
     "# Objectives\n",
     "\n",
@@ -58,7 +58,7 @@
    "metadata": {},
    "source": [
     "## Licence\n",
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos A. Iglesias, Universidad Politécnica de Madrid."
    ]

ml5/2_6_1_Q-Learning_Basic.ipynb

@@ -35,7 +35,7 @@
     "# Introduction\n",
     "The purpose of this practice is to understand better Reinforcement Learning (RL) and, in particular, Q-Learning.\n",
     "\n",
-    "We are going to use [Gymnasium](https://gymnasium.farama.org/). Gymnasium is toolkit for developing and comparing RL algorithms. It is a fork of [Open AI Gym](https://www.gymlibrary.dev/). Take a loot at ther [website](https://github.com/Farama-Foundation/Gymnasium).\n",
+    "We are going to use [Gymnasium](https://gymnasium.farama.org/). Gymnasium is a toolkit for developing and comparing RL algorithms. It is a fork of [Open AI Gym](https://www.gymlibrary.dev/). Take a loot at ther [website](https://github.com/Farama-Foundation/Gymnasium).\n",
     "\n",
     "It implements [algorithm imitation](http://gym.openai.com/envs/#algorithmic), [classic control problems](https://gymnasium.farama.org/environments/classic_control/), [Atari games](https://gymnasium.farama.org/environments/atari/), [Box2D continuous control](https://gymnasium.farama.org/environments/box2d/), [robotics with MuJoCo, Multi-Joint dynamics with Contact](https://gymnasium.farama.org/environments/mujoco/), [simple text based environments](https://gymnasium.farama.org/environments/toy_text/), and [other problems](https://gymnasium.farama.org/environments/third_party_environments/).\n",
     "\n",
@@ -64,7 +64,7 @@
    "metadata": {},
    "source": [
     "## Environments\n",
-    "OpenGym provides a number of problems called *environments*. \n",
+    "OpenGym provides several problems called *environments*. \n",
     "\n",
     "Try 'LunarLander-v2' (or 'CartPole-v1', 'MountainCar-v0', etc.)."
    ]
@@ -1328,7 +1328,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos Á. Iglesias, Universidad Politécnica de Madrid."
    ]

ml5/2_6_1_Q-Learning_Exercises.ipynb

@@ -38,7 +38,7 @@
     "## Taxi\n",
     "Analyze the [Taxi problem](https://gymnasium.farama.org/environments/toy_text/taxi/) and solve it applying Q-Learning. You can find a solution as the one previously presented  [here](https://www.oreilly.com/learning/introduction-to-reinforcement-learning-and-openai-gym), and the notebook is [here](https://github.com/wagonhelm/Reinforcement-Learning-Introduction/blob/master/Reinforcement%20Learning%20Introduction.ipynb). Take into account that Gymnasium has changed, so you will have to adapt the code.\n",
     "\n",
-    "Analyze the impact of not changing the learning rate or changing it in a different way. "
+    "Analyze the impact of not changing the learning rate or changing it differently. "
    ]
   },
   {
@@ -49,7 +49,7 @@
     "Select one of the following exercises.\n",
     "\n",
     "## Blackjack\n",
-    "Analyze how to appy Q-Learning for solving Blackjack.\n",
+    "Analyze how to apply Q-Learning for solving Blackjack.\n",
     "You can find information in this [article](https://gymnasium.farama.org/tutorials/training_agents/blackjack_tutorial/).\n",
     "\n",
     "## Doom\n",
@@ -82,7 +82,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos Á. Iglesias, Universidad Politécnica de Madrid."
    ]

ml5/2_6_1_Q-Learning_Visualization.ipynb

@@ -39,7 +39,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In this section we are going to visualize Q-Learning based on this [link](https://gymnasium.farama.org/tutorials/training_agents/FrozenLake_tuto/#sphx-glr-tutorials-training-agents-frozenlake-tuto-py). The code has been ported to the last version of Gymnasium.\n",
+    "In this section, we are going to visualize Q-Learning based on this [link](https://gymnasium.farama.org/tutorials/training_agents/FrozenLake_tuto/#sphx-glr-tutorials-training-agents-frozenlake-tuto-py). The code has been ported to the last version of Gymnasium.\n",
     "\n",
     "First, we are going to define a class *Params* for the Q-Learning parameters and the environment based on these values."
    ]
@@ -129,7 +129,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Running the environment"
+    "## Running the environment."
    ]
   },
   {
@@ -161,7 +161,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We want to plot the policy the agent has learned in the end. To do that the function *qtable_directions_map* perform these actions: 1. extract the best Q-values from the Q-table for each state, 2. get the corresponding best action for those Q-values, 3. map each action to an arrow so we can visualize it."
+    "We want to plot the policy the agent has learned in the end. To do that, the function *qtable_directions_map* performs these actions: 1. extract the best Q-values from the Q-table for each state, 2. get the corresponding best action for those Q-values, 3. map each action to an arrow so we can visualize it."
    ]
   },
   {
@@ -182,7 +182,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Now we’ll be running our agent on a few increasing maps sizes: \n",
+    "Now we’ll be running our agent on a few increasing map sizes: \n",
     "- 4x4\n",
     "- 7x7\n",
     "- 9x9\n",
@@ -312,7 +312,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos Á. Iglesias, Universidad Politécnica de Madrid."
    ]

python/1_0_Intro_Python.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_1_Notebooks.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_2_Numbers_Strings.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_3_Sequences.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_4_Sets.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_5_ControlFlow.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_6_Functions.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_7_Variables.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_8_Classes.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1_9_Errors_Exceptions.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

python/1__10_Modules_Packages.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "![](http://www.upm.es/sfs/Rectorado/Gabinete%20del%20Rector/Logos/UPM/EscPolitecnica/EscUpmPolit_p.gif \"UPM\")"
+    "![](https://github.com/gsi-upm/sitc/blob/9844820e6653b0e169113a06538f8e54554c4fbc/images/EscUpmPolit_p.gif?raw=true \"UPM\")"
    ]
   },
   {

rdf/RDF.ipynb

@@ -63,7 +63,7 @@
     "\n",
     "# Tools\n",
     "\n",
-    "This notebook is self-contained, but it requires some python libraries.\n",
+    "This notebook is self-contained, but it requires some Python libraries\n",
     "To install them, simply run the following line:"
    ]
   },
@@ -137,13 +137,13 @@
     "This concept may be described as taking care of a dog, for example, whereas a different property `<http://yourwebsite.com/hasDog>` could be described as being the legal owner of the dog.\n",
     "\n",
     "\n",
-    "RDF can be used to embed annotation in many places, including HTML document, using any compatible format.\n",
-    "The options include including RDFa, XML, JSON-LD and [Turtle](https://www.w3.org/TR/turtle/).\n",
+    "RDF can be used to embed annotation in many places, including HTML documents, using any compatible format.\n",
+    "The options include including RDFa, XML, JSON-LD, and [Turtle](https://www.w3.org/TR/turtle/).\n",
     "\n",
     "\n",
     "In the exercises, we will be using turtle notation, because it is very readable.\n",
     "\n",
-    "For instance, turtle allows us to re-use parts of a previous triple to avoid repetition.\n",
+    "For instance, turtle allows us to reuse parts of a previous triple to avoid repetition.\n",
     "Our previous example would look like this:\n",
     "\n",
     "```turtle\n",
@@ -151,7 +151,7 @@
     "                            <http://example.org/age> 7\n",
     "```\n",
     "\n",
-    "Here's another example of document in Turtle, taken from the Turtle specification:\n",
+    "Here's another example of a document in Turtle, taken from the Turtle specification:\n",
     "\n",
     "```turtle\n",
     "@base <http://example.org/> .\n",
@@ -172,7 +172,7 @@
     "\n",
     "The second exercise will show you how to extract this information from any website.\n",
     "\n",
-    "As you can observe in these examples, Turtle defines several ways to specify IRIs in a result. Please, consult the specification for further details. As an overview, IRIs can be:\n",
+    "As you can observe in these examples, Turtle defines several ways to specify IRIs in a result. Please consult the specification for further details. As an overview, IRIs can be:\n",
     " * *relative IRIs*: IRIs resolved relative to the current base IRI. Thus, you should define a base IRI (@base <http://example.org>) and then relative IRIs (i.e. <#spiderman>). The resulting IRI is <http://example.org/spiderman>.\n",
     " * *prefixed names*: a prefixed name (i.e. foaf:Person) is transformed into an IRI by concatenating the IRI of the prefix (@prefix foaf: <http://xmlns.com/foaf/0.1) and the local part of the prefixed name (i.e. Person). So, the resulting IRI is <http://xmlns.com/foaf/0.1/Person\n",
     " * *absolute IRIs*: an already resolved IRI, p.ej. <http://example.com/Auto>."
@@ -200,17 +200,17 @@
     "These vocabularies can be reused in several applications.\n",
     "In the example above, we used the concept of person from an external vocabulary (`foaf:Person`, i.e. http://xmlns.com/foaf/0.1/Person).\n",
     "That way, we do not need to redefine the concept of Person in every application.\n",
-    "There are several well known vocabularies, such as:\n",
+    "There are several well-known vocabularies, such as:\n",
     "\n",
     "* Dublin core, for metadata: http://dublincore.org/\n",
     "* FOAF (Friend-of-a-friend) for social networks: http://www.foaf-project.org/\n",
     "* SIOC for online communities: https://www.w3.org/Submission/sioc-spec/\n",
     "\n",
-    "Using the same vocabularies also makes it easier to automatically process and classify information.\n",
+    "Using the same vocabulary also makes it easier to automatically process and classify information.\n",
     "\n",
     "\n",
-    "That was the motivation behind Schema.org, a collaboration between Google, Microsoft, Yahoo and Yandex.\n",
-    "They aim to provide schemas for structured data annotation of Web sites, e-mails, etc., which can be leveraged by search engines and other automated processes.\n",
+    "That was the motivation behind Schema.org, a collaboration between Google, Microsoft, Yahoo, and Yandex.\n",
+    "They aim to provide schemas for structured data annotation of websites, e-mails, etc., which can be leveraged by search engines and other automated processes.\n",
     "\n",
     "They rely on RDF for representation, and provide a set of common vocabularies that can be shared by every web developer.\n",
     "\n",
@@ -223,7 +223,7 @@
     "* Documentation for hotels: https://schema.org/docs/hotels.html\n",
     "\n",
     "\n",
-    "You can use the documentation to find properties (e.g. `checkinTime`), as well as the type of that property (e.g. `Datetime`)."
+    "You can use the documentation to find properties (e.g., `checkinTime`), as well as the type of that property (e.g. `Datetime`)."
    ]
   },
   {
@@ -326,7 +326,7 @@
     "# YOUR ANSWER HERE\n",
     "```\n",
     "\n",
-    "Depending on the exercise, you might need to fill that part with a Turtle definition (first exercise), some python code (second exercise), or plain text."
+    "Depending on the exercise, you might need to fill that part with a Turtle definition (first exercise), some Python code (second exercise), or plain text."
    ]
   },
   {

sna/0_Intro_Network_Analysis.ipynb

@@ -97,7 +97,7 @@
    },
    "source": [
     "## Licence\n",
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos A. Iglesias, Universidad Politécnica de Madrid."
    ]

sna/1_First_Steps.ipynb

@@ -1244,7 +1244,7 @@
     }
    },
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos A. Iglesias, Universidad Politécnica de Madrid."
    ]

sna/2_Working_with_Graphs.ipynb

@@ -956,7 +956,7 @@
     }
    },
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos A. Iglesias, Universidad Politécnica de Madrid."
    ]

sna/2a_Florentine_Families_Star_Wars.ipynb

@@ -326,7 +326,7 @@
     }
    },
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos A. Iglesias, Universidad Politécnica de Madrid."
    ]

sna/3_Network_Analysis.ipynb

@@ -2173,7 +2173,7 @@
     }
    },
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos A. Iglesias, Universidad Politécnica de Madrid."
    ]

sna/4_Social_Networks.ipynb

@@ -808,7 +808,7 @@
     }
    },
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos A. Iglesias, Universidad Politécnica de Madrid."
    ]

sna/5_Pandas.ipynb

@@ -415,7 +415,7 @@
     }
    },
    "source": [
-    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
+    "The notebook is freely licensed under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
     "\n",
     "© Carlos A. Iglesias, Universidad Politécnica de Madrid."
    ]