{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "![](images/EscUpmPolit_p.gif \"UPM\")" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "# Course Notes for Learning Intelligent Systems" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "Department of Telematic Engineering Systems, Universidad Politécnica de Madrid, © Carlos A. Iglesias" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# Table of Contents\n", "* [Objectives](#Objectives)\n", "* [Transformers](#Transformers)\n", "* [Use cases: how to use pipelines](#Use-cases:-how-to-usepipelines)\n", " * Sentiment Analysis\n", " * Masked Word Completion\n", " * Text generation\n", " * Question Answering\n", " * Text to Speech\n", "* [References](#References)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Objectives" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "In this session we are going to learn the power of transformers." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Transformers\n", "As we saw, transformers are an extremely powerful architecture capable of performing many popular NLP tasks.\n", "\n", "A well-known transformer model repository is available at https://huggingface.co/. \n", "\n", "Let's see how to use it. To go deeper, consult the Hugging tutorial (https://huggingface.co/learn/nlp-course/chapter1/1).\n", "\n", "The transformers package requires to have installed Pytorch or TensorFlow. Check the installation details if you want to configure your environment well. For learning purposes, we are going to install Pytorch.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First of all, you should install Hugging Face. Execute:\n", "* pip install torch transformers" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: torch in /home/cif/miniconda3/lib/python3.12/site-packages (2.10.0)\n", "Requirement already satisfied: transformers in /home/cif/miniconda3/lib/python3.12/site-packages (5.2.0)\n", "Requirement already satisfied: filelock in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (3.24.2)\n", "Requirement already satisfied: typing-extensions>=4.10.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (4.15.0)\n", "Requirement already satisfied: setuptools in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (82.0.0)\n", "Requirement already satisfied: sympy>=1.13.3 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (1.14.0)\n", "Requirement already satisfied: networkx>=2.5.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (3.6.1)\n", "Requirement already satisfied: jinja2 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (3.1.6)\n", "Requirement already satisfied: fsspec>=0.8.5 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (2026.2.0)\n", "Requirement already satisfied: cuda-bindings==12.9.4 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (12.9.4)\n", "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.8.93 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (12.8.93)\n", "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.8.90 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (12.8.90)\n", "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.8.90 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (12.8.90)\n", "Requirement already satisfied: nvidia-cudnn-cu12==9.10.2.21 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (9.10.2.21)\n", "Requirement already satisfied: nvidia-cublas-cu12==12.8.4.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (12.8.4.1)\n", "Requirement already satisfied: nvidia-cufft-cu12==11.3.3.83 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (11.3.3.83)\n", "Requirement already satisfied: nvidia-curand-cu12==10.3.9.90 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (10.3.9.90)\n", "Requirement already satisfied: nvidia-cusolver-cu12==11.7.3.90 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (11.7.3.90)\n", "Requirement already satisfied: nvidia-cusparse-cu12==12.5.8.93 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (12.5.8.93)\n", "Requirement already satisfied: nvidia-cusparselt-cu12==0.7.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (0.7.1)\n", "Requirement already satisfied: nvidia-nccl-cu12==2.27.5 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (2.27.5)\n", "Requirement already satisfied: nvidia-nvshmem-cu12==3.4.5 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (3.4.5)\n", "Requirement already satisfied: nvidia-nvtx-cu12==12.8.90 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (12.8.90)\n", "Requirement already satisfied: nvidia-nvjitlink-cu12==12.8.93 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (12.8.93)\n", "Requirement already satisfied: nvidia-cufile-cu12==1.13.1.3 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (1.13.1.3)\n", "Requirement already satisfied: triton==3.6.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from torch) (3.6.0)\n", "Requirement already satisfied: cuda-pathfinder~=1.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from cuda-bindings==12.9.4->torch) (1.3.4)\n", "Requirement already satisfied: huggingface-hub<2.0,>=1.3.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (1.4.1)\n", "Requirement already satisfied: numpy>=1.17 in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (2.3.5)\n", "Requirement already satisfied: packaging>=20.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (26.0)\n", "Requirement already satisfied: pyyaml>=5.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (6.0.3)\n", "Requirement already satisfied: regex!=2019.12.17 in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (2026.1.15)\n", "Requirement already satisfied: tokenizers<=0.23.0,>=0.22.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (0.22.2)\n", "Requirement already satisfied: typer-slim in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (0.24.0)\n", "Requirement already satisfied: safetensors>=0.4.3 in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (0.7.0)\n", "Requirement already satisfied: tqdm>=4.27 in /home/cif/miniconda3/lib/python3.12/site-packages (from transformers) (4.67.3)\n", "Requirement already satisfied: hf-xet<2.0.0,>=1.2.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface-hub<2.0,>=1.3.0->transformers) (1.2.0)\n", "Requirement already satisfied: httpx<1,>=0.23.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface-hub<2.0,>=1.3.0->transformers) (0.28.1)\n", "Requirement already satisfied: shellingham in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface-hub<2.0,>=1.3.0->transformers) (1.5.4)\n", "Requirement already satisfied: anyio in /home/cif/miniconda3/lib/python3.12/site-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers) (4.12.1)\n", "Requirement already satisfied: certifi in /home/cif/miniconda3/lib/python3.12/site-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers) (2026.2.25)\n", "Requirement already satisfied: httpcore==1.* in /home/cif/miniconda3/lib/python3.12/site-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers) (1.0.9)\n", "Requirement already satisfied: idna in /home/cif/miniconda3/lib/python3.12/site-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers) (3.11)\n", "Requirement already satisfied: h11>=0.16 in /home/cif/miniconda3/lib/python3.12/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers) (0.16.0)\n", "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from sympy>=1.13.3->torch) (1.3.0)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from jinja2->torch) (3.0.3)\n", "Requirement already satisfied: typer>=0.24.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from typer-slim->transformers) (0.24.0)\n", "Requirement already satisfied: click>=8.2.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from typer>=0.24.0->typer-slim->transformers) (8.3.1)\n", "Requirement already satisfied: rich>=12.3.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from typer>=0.24.0->typer-slim->transformers) (14.3.2)\n", "Requirement already satisfied: annotated-doc>=0.0.2 in /home/cif/miniconda3/lib/python3.12/site-packages (from typer>=0.24.0->typer-slim->transformers) (0.0.4)\n", "Requirement already satisfied: markdown-it-py>=2.2.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from rich>=12.3.0->typer>=0.24.0->typer-slim->transformers) (4.0.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from rich>=12.3.0->typer>=0.24.0->typer-slim->transformers) (2.19.2)\n", "Requirement already satisfied: mdurl~=0.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich>=12.3.0->typer>=0.24.0->typer-slim->transformers) (0.1.2)\n" ] } ], "source": [ "!pip install torch transformers" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: huggingface_hub[hf_xet] in /home/cif/miniconda3/lib/python3.12/site-packages (1.4.1)\n", "Requirement already satisfied: filelock in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (3.24.2)\n", "Requirement already satisfied: fsspec>=2023.5.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (2026.2.0)\n", "Requirement already satisfied: hf-xet<2.0.0,>=1.2.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (1.2.0)\n", "Requirement already satisfied: httpx<1,>=0.23.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (0.28.1)\n", "Requirement already satisfied: packaging>=20.9 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (26.0)\n", "Requirement already satisfied: pyyaml>=5.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (6.0.3)\n", "Requirement already satisfied: shellingham in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (1.5.4)\n", "Requirement already satisfied: tqdm>=4.42.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (4.67.3)\n", "Requirement already satisfied: typer-slim in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (0.24.0)\n", "Requirement already satisfied: typing-extensions>=4.1.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from huggingface_hub[hf_xet]) (4.15.0)\n", "Requirement already satisfied: anyio in /home/cif/miniconda3/lib/python3.12/site-packages (from httpx<1,>=0.23.0->huggingface_hub[hf_xet]) (4.12.1)\n", "Requirement already satisfied: certifi in /home/cif/miniconda3/lib/python3.12/site-packages (from httpx<1,>=0.23.0->huggingface_hub[hf_xet]) (2026.2.25)\n", "Requirement already satisfied: httpcore==1.* in /home/cif/miniconda3/lib/python3.12/site-packages (from httpx<1,>=0.23.0->huggingface_hub[hf_xet]) (1.0.9)\n", "Requirement already satisfied: idna in /home/cif/miniconda3/lib/python3.12/site-packages (from httpx<1,>=0.23.0->huggingface_hub[hf_xet]) (3.11)\n", "Requirement already satisfied: h11>=0.16 in /home/cif/miniconda3/lib/python3.12/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->huggingface_hub[hf_xet]) (0.16.0)\n", "Requirement already satisfied: typer>=0.24.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from typer-slim->huggingface_hub[hf_xet]) (0.24.0)\n", "Requirement already satisfied: click>=8.2.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from typer>=0.24.0->typer-slim->huggingface_hub[hf_xet]) (8.3.1)\n", "Requirement already satisfied: rich>=12.3.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from typer>=0.24.0->typer-slim->huggingface_hub[hf_xet]) (14.3.2)\n", "Requirement already satisfied: annotated-doc>=0.0.2 in /home/cif/miniconda3/lib/python3.12/site-packages (from typer>=0.24.0->typer-slim->huggingface_hub[hf_xet]) (0.0.4)\n", "Requirement already satisfied: markdown-it-py>=2.2.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from rich>=12.3.0->typer>=0.24.0->typer-slim->huggingface_hub[hf_xet]) (4.0.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/cif/miniconda3/lib/python3.12/site-packages (from rich>=12.3.0->typer>=0.24.0->typer-slim->huggingface_hub[hf_xet]) (2.19.2)\n", "Requirement already satisfied: mdurl~=0.1 in /home/cif/miniconda3/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich>=12.3.0->typer>=0.24.0->typer-slim->huggingface_hub[hf_xet]) (0.1.2)\n" ] } ], "source": [ "!pip install huggingface_hub[hf_xet]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use cases: how to use pipelines" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Sentiment Analysis\n", "Let's classify sentiments" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "f761aa16051140ee84552e28f855ef69", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Loading weights: 0%| | 0/201 [00:00\n", " \n", " Your browser does not support the audio element.\n", " \n", " " ], "text/plain": [ "" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from transformers import pipeline\n", "\n", "pipe = pipeline(\"text-to-speech\", model=\"suno/bark-small\")\n", "text = \"[clears throat] This is a test ... and I just took a long pause.\"\n", "output = pipe(text)\n", "\n", "from IPython.display import Audio \n", "\n", "Audio(output[\"audio\"], rate=output[\"sampling_rate\"])" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "## References\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "* [Hugging Face Tutorial](https://huggingface.co/learn/nlp-course/chapter1/1) " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "## Licence" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/). \n", "\n", "© Carlos A. Iglesias, Universidad Politécnica de Madrid." ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.2" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false } }, "nbformat": 4, "nbformat_minor": 4 }