sitc/ml3/2_4_2_Exercise_Optional.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/EscUpmPolit_p.gif \"UPM\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Course Notes for Learning Intelligent Systems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Department of Telematic Engineering Systems, Universidad Politécnica de Madrid, © Óscar Araque"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [Introduction to Machine Learning III](2_4_0_Intro_NN.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MultiLayer Perceptron (MLP) applied to Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we are going to apply a MLP to a simple regression task: learning the Fresnel functions.\n",
    "\n",
    "\n",
    "Answer directly in your copy of the exercise and submit it as a moodle task."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-03-27T19:44:45.989818",
     "start_time": "2017-03-27T19:44:45.624698"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Show plots in the notebooks\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-03-27T19:44:47.582964",
     "start_time": "2017-03-27T19:44:46.844267"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "sns.set(color_codes=True)\n",
    "\n",
    "# function generator\n",
    "from scipy.special import fresnel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Change this variables to change the train and test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-03-27T19:44:49.811655",
     "start_time": "2017-03-27T19:44:49.805379"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# variables to change\n",
    "\n",
    "n_examples = 500\n",
    "x_min = 0\n",
    "x_max = 3\n",
    "\n",
    "fresnel_func = 0 # can be 0,1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-03-27T19:45:21.738411",
     "start_time": "2017-03-27T19:45:21.489043"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAegAAAFgCAYAAABws+q5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlYlOe9N/Dv7Oz7MKyiooIgKoorihvuMXuMGrXpm6Yn\nafuepDZN0uQ9dovHtKdJTk/SnjZt07RpEpIm0hjjblRUUEFlFxFkkXUYtmGHmXnePxAS48IAMzyz\nfD/X1etiOsPMj9vJfOe+n3uRCIIggIiIiGyKVOwCiIiI6FYMaCIiIhvEgCYiIrJBDGgiIiIbxIAm\nIiKyQQxoIiIiGyQXuwAiexIdHY2IiAjIZDIAgCAICA0NxZ///GfRatq2bRs2btyIDRs23PT/C4KA\n3/72tzh8+DAAwGAwYNmyZXj++ecH6x/Q3t6Oxx57DN3d3fj444/h7e1tsfrKysrQ2NiIhIQEHD16\nFMePH8euXbss9vxEjooBTTQMEokE7733HgIDA8UuZUgfffQRLl26hD179sDFxQWdnZ148skn8c47\n7+DJJ5+86bFFRUXQ6/U4fvy4xes4cuQIjEYjEhISkJycjOTkZIu/BpEj4hA30TAIgoA77e2zbds2\nvPHGG1i/fj2ys7PR1taG559/HqtXr8bKlSuxZ8+ewce+8cYbWLNmDdasWYPHH38cDQ0NAPp76J99\n9hkeeOABLF68GO++++7g73z00UdYu3YtVqxYgR/96Efo7e29a63FxcWYPHkyXFxcAABubm74/e9/\nj29961s3Pa62thY//vGPodPpsG7dOuTn5yM2Nnbw/urq6sHbqampeOaZZ/Dyyy9j9erVuOeee1Ba\nWgoAaG5uxlNPPYXk5GTcd999OHPmDI4fP463334b7733Hn71q18hNTUV3/72twEAra2tePbZZ7Fm\nzRrcc889+NOf/jT4mndrByJnwYAmsqDCwkJ88cUXmDlzJnbv3g2ZTIZDhw7h448/xptvvomSkhKU\nlJTg4MGD2L9/Pw4ePIiVK1ciPT198DlKS0uRmpqK3//+93jjjTcgCAKysrLw5ptv4r333sOxY8fg\n6emJ//7v/75rLUlJSUhJScGuXbtw/vx59Pb2wtvbG0ql8qbHBQcH41e/+hVCQkKwf/9++Pr6QiKR\n3PSYr99OS0vD1q1bcejQIcydOxd/+9vfAACvvfYaJk+ejKNHj+LVV1/Fj370IyxatAgrV67E9u3b\n8cILL9z0XK+99hq8vb1x8OBBvP/++/jwww9x8eLFu7YDkTNhQBMN0/bt27Fu3brB/+3cuXPwviVL\nlgz+fOLECWzfvh0A4Ovri5UrV+Lw4cPw9PRES0sLPvvsM+j1ejz22GO47777Bn9v4OfY2Fj09vai\nsbERx48fx9q1axEQEAAAePTRRwevLd/J0qVL8ac//Qn19fX4/ve/j3nz5uEnP/kJ9Hr9qP7+SZMm\nYerUqQCAmJgY1NTUAABOnjyJ9evXAwCmTp2KL7/8EgqF4o7Pk5aWhi1btgAAvL29sXLlSpw5c2bw\n/tu1A5Ez4TVoomG62zXor0+u0uv1ePbZZyGTySAIAnp6erBmzRpoNBr8z//8D9555x388pe/xNy5\nc/Hzn/8cGo0GAODh4QEAkEr7vz8bjUa0tbXhyJEjgwFmNBphNBqHrHXBggVYsGABBEHAhQsX8Otf\n/xo///nP8dprr4347/f09Bz8WSaTwWQyAegf4vby8hq8z83N7a7P09TUdFN7eXl5DQ71Aze3gyAI\nZv29RI6EAU00TOYOtWo0Gvzud7/DpEmTbrlv3rx5mDdvHrq7u/Hqq6/iN7/5Df7rv/7rjs8VGBiI\nBx54AM8//7zZdaalpWHWrFnw8PCARCJBQkICnn76abzxxht3/b2vhy4AtLS0mPV6vr6+aG5uRkhI\nCID+a9cDXzpuJyAgAC0tLQgKChp8nYERAiLiEDeR1SxfvhwffvghgP4lTrt378bly5dx5swZ/OIX\nv4AgCHBxcUF0dPQt13wHDHwZWL58OY4cOYLm5mYAwNGjR4dc2vXee+/hN7/5zeBksp6eHhw6dAhz\n5sy56+/5+vpCKpWiuLgYALB3716z/97U1FQAQElJCR588EGYTCbI5fLbDqsvWbIEH330EYD+3vTh\nw4exdOlSs16LyBmwB000DHcK0tvd98wzz+AXv/gF1qxZA4lEgkWLFiE6Ohp9fX3Yt28fVq9eDZVK\nBT8/P/znf/7nbZ9j4HZMTAz+7d/+Ddu2bYMgCPDz88MvfvGLu9b0+uuv49e//jU2bNgAqVQKo9GI\n5cuX49lnn73r36hSqfDv//7veOKJJ6DRaLB169a7N8oNP/7xj/HCCy9g+fLl8PDwwOuvvw6lUoll\ny5bhueeeQ3V1NZYtWzb4+B/+8If42c9+hrVr10Imk+Gpp57CtGnT7toORM5EYs550Lt370ZOTg4k\nEgleeuklxMXFDd73/vvv4/PPP4dMJsO0adPwk5/8xKoFExEROYMhe9CZmZmoqKhASkoKSktL8fLL\nLyMlJQVA/+5Df/nLX3Ds2DFIJBI88cQTyM3NxfTp061eOBERkSMb8hp0RkbG4M4/kZGR0Ov16Ojo\nAAAolUoolUq0t7fDYDCgu7vbolsEEhEROashA1qn08HPz2/wtq+vL3Q6HYD+gP7+978/uH3fjBkz\nEBERYb1qiYiInMSQAf3NS9SCIAxO2Ghvb8cf//hHHD58GEeOHEF2djauXLlinUqJiIicyJABrdFo\nBnvMAKDVagfXKl67dg3h4eHw9vaGQqFAQkICCgoK7vp83K6PiIhoaENOEktMTMRbb72FjRs3orCw\nEBqNZnCHoNDQUFy7dg29vb1QKBTIz8+/aavD25FIJGhoaLNM9Q5OrfZkW5mB7WQetpP52FbmYTuZ\nT632HPpB3zBkQMfHxyM2NhabNm2CTCbDzp07kZqaCk9PTyQnJ+OJJ57Atm3bIJfLER8fj9mzZ4+o\neCIiIvqKWeugLY3fuMzDb6fmYTuZh+1kPraVedhO5htJD5pbfRIREdkgBjQREZENYkATERHZIAY0\nERGRDWJAExER2SAGNBERkQ1iQBMREdkgBjQREZENYkATERHZIAY0ERGRDWJAExER2SAGNBERkQ1i\nQBMREdkgBjQREZENYkATERHZIAY0ERGRDWJAExER2SAGNBERkQ1iQBMREdkgBjQREZENYkATERHZ\nIAY0ERGRDWJAExER2SAGNBERkQ1iQBMREdkgBjQREZENYkATERHZIAY0ERGRDZKLXQAR3V13rwHZ\nJToUV7agrK4NLe096OoxQCmXwctdifBAD0wO88asKWr4eKjELpeILIQBTWSj6ps68cXZCpy/XI/e\nPhMAQC6Tws9TBR8PFXr7jGjSd6NG14FzhfV4/3Ax4iL9sW5+BKaE+4hcPRGNllkBvXv3buTk5EAi\nkeCll15CXFwcAKC+vh7PPfccJBIJBEFAVVUVnnvuOaxfv96qRRM5svauPnx6shRpOTUQBEDt44IF\nsUGYMSkA4YEekMu+ujJlEgQ0tHQhr7QR5wrrkVvaiNzSRsycFIAtyZMR4OMq4l9CRKMxZEBnZmai\noqICKSkpKC0txcsvv4yUlBQAgEajwXvvvQcAMBqN2L59O5YvX27diokc2KWrDXj3QBHaOvsQEuCO\n+xZNwOwoNaQSyW0fL5VIoPF1gybBDckJ4Si+3oLUtGvILtGhsLwJW1ZOweLpwZDc4feJyHYNGdAZ\nGRlITk4GAERGRkKv16OjowPu7u43PW7Pnj1YtWoVXF35jZ1ouAxGEz45UYrDmdehkEvxyLJIrEwI\nv6m3bI4p4T54fks8zhbU4/0jxXj3QBGuVLbg8bVRVqqciKxlyIDW6XSYNm3a4G1fX1/odLpbAvqT\nTz7BO++8Y/kKiRxcV48B//uvfOSXNSHY3w1P3zcNYYEeI34+iUSCBdOCMDnMG//7WQEyCurQqO/G\nz7+7wIJVE5G1DRnQgiD
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fad88a8aa58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Generate dataset (do not change)\n",
    "x = np.linspace(x_min, x_max, num=n_examples)\n",
    "y = fresnel(x)[fresnel_func]\n",
    "\n",
    "X = x.reshape(-1,1)\n",
    "\n",
    "plt.plot(x, y)\n",
    "plt.title('Fresnel S function')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make the MLP learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-03-27T19:45:33.201217",
     "start_time": "2017-03-27T19:45:33.184488"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# do not change\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import r2_score\n",
    "from sklearn.neural_network import MLPRegressor\n",
    "\n",
    "# split the dataset\n",
    "X_train, X_test, y_train, y_test = \\\n",
    "    train_test_split(X, y, test_size=0.3, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Change the parameters of the MLPRegressor to generate a network that learns the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-03-27T19:44:57.970215",
     "start_time": "2017-03-27T19:44:57.916064"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MLPRegressor(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,\n",
       "       beta_2=0.999, early_stopping=False, epsilon=1e-08,\n",
       "       hidden_layer_sizes=(100,), learning_rate='constant',\n",
       "       learning_rate_init=0.001, max_iter=200, momentum=0.9,\n",
       "       nesterovs_momentum=True, power_t=0.5, random_state=None,\n",
       "       shuffle=True, solver='adam', tol=0.0001, validation_fraction=0.1,\n",
       "       verbose=False, warm_start=False)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# CHANGE THIS CODE TO MAKE THE MLP LEARN\n",
    "\n",
    "# train the network\n",
    "mlp = MLPRegressor()\n",
    "mlp.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this next cell, we can see how the network approximates the original function. \n",
    "\n",
    "The R score (a regression metric) must be between 0.95 and 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-03-27T19:45:44.523305",
     "start_time": "2017-03-27T19:45:44.221047"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAFXCAYAAABk0q3rAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcU3e+P/7XSQIhQFgCIQoIIpsIuG+IS6tUW6utnbla\n2to7neu0M23n1+k4997pcqedO3d69d4Z25m5/c6d9rYznS5TrDPS6a7d1FqxooiKqAgoqyxhTwhb\ncn5/0KSAYQuB5CSv5+PRx6PhhOT96aF557O9P4IoiiKIiIhIEmSuDoCIiIjGjombiIhIQpi4iYiI\nJISJm4iISEKYuImIiCSEiZuIiEhCJpS4S0pKcNNNN+GNN9647tqxY8ewdetWZGdn4/e///1E3oaI\niIi+5nDiNplM+OUvf4mMjAy715955hk8//zzePPNN/Hll1+irKzM4SCJiIion8OJW6lU4qWXXkJE\nRMR116qqqhASEgKdTgdBELBmzRocP358QoESERHRBBK3TCaDr6+v3Wt6vR4ajcb2WKPRoKGhwdG3\nIiIioq9NyuK0oVVURVGEIAiT8VZEREReZVISt06nQ2Njo+1xfX09tFrtiL/DkulERESjU0zGi0ZF\nRcFoNKK2thYRERE4dOgQ9uzZM+LvCIKAxsaOyQjHLWi1arZPojy5bQDbJ3Vsn3RptWqHfs/hxH3+\n/Hns3r0btbW1UCgUOHDgANauXYvo6GhkZWXh6aefxs6dOwEAmzZtQmxsrKNvRURERF9zOHGnpqbi\ntddeG/b64sWLkZOT4+jLExERkR2snEZERCQhTNxEREQSwsRNREQkIUzcREREEsLETUREJCFM3ERE\nRBLCxE1ERCQhTNxEREQSwsRNREQkIUzcREREEsLETUREJCFM3ERERBLCxE1ERCQhTNxEREQSwsRN\nREQkIUzcREREEsLETUREJCFM3ERERBLCxE1ERCQhTNxEREQSwsRNREQkIUzcREREEsLETUREJCFM\n3ERERBLCxE1ERCQhTNxEREQSwsRNREQkIUzcREREEsLETUREJCEKVwdARERkZegxYm9JLvSmZoSp\nNPjhintdHZLbYeImIiKXM/QY8cbFfShqugiLaAEAVHZU46VTb2J7YraLo3MvTNxEROQy1h72habL\nMJlN112vNza5ICr3xsRNREQu88bFfTirLx72ui4gbAqjkQYuTiMiIpe53Hpl2GshymB8b9FdUxiN\nNLDHTUREU2bo4jOL2XLdc1RyFRJC4rA9ZSvUykB0ocMFkbovJm4iIpoShh4jduX/Bq3dbQD6F58F\n+ajR3dtte0562Bz8YN59LopQGpi4iYhoSuwtybUlbasgXzUSQuOgNzUjXKXBnUl3uCg66WDiJiKi\nSTNwaLyx8/oV4hEB4diRtt0FkUkXEzcREU2KoUPjQ4Uog9nDdgATNxEROd1wSVulUEGrCrMNiwf6\nBrgoQuli4iYiIqexFVRpvgxT3/UFVVI0iRwanyAmbiIimrDREjbAoXFnmVDi3rVrF86cOQNBEPDE\nE08gPT3ddu2NN97Au+++C7lcjrS0NDz++OMTDpaIiNzT3pJcFDScHfZ6iDIYjy95lEPjTuBw4s7P\nz0dFRQVycnJQVlaGJ598Ejk5OQAAg8GAl19+GZ9++ikEQcCOHTtw9uxZzJ0712mBExGR+9Cbmu3+\nXCVXISUskfPZTuRw4s7Ly0NWVhYAID4+Hu3t7TAajQgICICvry98fX1hMBigUqnQ1dWF4OBgpwVN\nRESuZT3N63LrFUAEfGSD0wkT9uRxOHHr9XqkpaXZHoeGhkKv19sS98MPP4ysrCyoVCps3LgRsbGx\nTgmYiIhcZ7jTvEzm/uHwIF81V4xPMocTtyiK1z0WBAFA/1D5Cy+8gIMHD8Lf3x/f+c53cOnSJSQn\nJ08sWiIicqmR5rKDfNX46ZJHpjgi7+Nw4tbpdNDr9bbHDQ0NCA8PBwCUl5djxowZtuHxxYsX4/z5\n86Mmbq1W7Wg4ksD2SZcntw1g+6RuKtvX2me/mAoARIVETEosnn7/xsvhxJ2ZmYnnn38e27ZtQ3Fx\nMXQ6Hfz9/QEAUVFRKC8vR09PD3x8fFBUVIQ1a9aM+pqNjZ57AoxWq2b7JMqT2wawfVI32e0bepqX\nvzB4+FsmyKCUKZEQEoctsZudHosn3z9Hv5A4nLgXLFiA1NRUZGdnQy6X46mnnkJubi7UajWysrKw\nY8cO3HvvvVAoFFiwYAEWLVrk6FsREZGLDBwar+yoxtzwOVgYMXfQoSCcy55agjh0stqFPPVbFeDZ\n3xoBz26fJ7cNYPukbjLaN+hgEFPToIIqMeroKZ3H9uT7N+U9biIi8kxvXNyHs/piu9fCVZopjoaG\nYuImIqJBLrdeGfRYJsgQHRjJ87LdBBM3ERENNmQCVSlTcpuXG5G5OgAiInIv8SFxgx4nDHlMrsUe\nNxGRFxq6zSt7wOrwe1O22q5xeNz9MHETEXkZQ48Ru/J/g9bu/mIqlR3VEADbOdmBvgE8M9uNcaic\niMjL7C3JtSVtq+FO9yL3w8RNRORl7CVpbvOSDg6VExF5uKHz2UG+QYOuhyiDOY8tIUzcREQezN58\nNsuWShsTNxGRB7M3n93a3c592RLGxE1E5GEG1RrvbLruOuezpY2Jm4jIQ1gT9oWmyzCZTXafw/ls\n6WPiJiLyEAOP4BxIpVBBqwrjfLaHYOImIvIQw+3FTtEksqCKB2HiJiKSKEOPEa8fy0FNa4PdbV4q\nhQopmkQOjXsYJm4iIompNzbid4Uvoq27HeLXR3lxm5f3YOIm8hDtxh6cKdWjrLYN9c0mdHb3wUch\nQ6haiRidGnNnhSFGFwhBEFwdKk3Q7wpfvG6LF8BtXt6CiZtI4irqOvDhVxXIv9gAccA5yn6+cvT2\nWVBe245TlxqRe6Qc0doAbFgag4y0aZAxgUvKwC1ebd3tdp/DbV7egYmbSKI6Onuw97NSHCuqAwDM\niAjEirRpSIkNRWR4ABRyGURRREtHN0pr2nDyUiNOlzTi5fcv4JNT1dh+UxLio4Jd3Aoai3pjI3ad\neA69Yp/d6zJBhvnaNM5lewkmbiIJOlOqx8vvX4DB1IvYaWp8e80spM7UXDcMLggCNEF+WBrkh6Up\nOjS3d+Gvh8pwvLgeu14vwLdvmIXtG1Nd1AoajbWXXdhQBAssg67JBBnkghxqZQB+OPcB6ALCXRQl\nTTUmbiIJsYgi3j92FW9/cQUKhQzZaxOwbnE05LKxHfSnCfLDA7elYvW8SLzw7nns+7wM1fpO3Lch\nCT4K+SRHT+M13L5sAJivTcOOtO3QatVobOyY4sjIlXisJ5FEWCwiXvngInK/uAJNkBJPbF+E9Utj\nxpy0B5odG4p//+5SzI4JQd65a/jNvrPo6rE/DEuuM9y+bB9BwWFxL8bETSQBfWYLXnz3PI6eu4aZ\n09T42X1LEDtNPaHXDArwxY+3zcOy1Gm4UNGCZ986g+5es5MiJmcIG7LYTICAEGUwHl+6k9u8vBiH\nyoncnEUU8ccPLuDEhQYkRgfj0a3zoFI6539dH4Ucj39nCf7zT1/hxIUGvPD383j4W2kO9eLJ+bKT\n7oAAcF82DcLETeTm9n1eiuPn6xEfGYSd2+ZD6evcuWi5XIbvbZoDg6kXhaV6vHGwBP9482ynvgc5\nJtA3gKVK6Tr8Wk3kxj4rqMaBE1WYHuaPH22d5/SkbaWQy/DwHemIiQjEocJaHDlTOynvQ4MZeox4\nueh1/Ff+7/BS0esw9BhdHRJJAHvcRG6qtLoNb35yGWp/H/x42zwEqnwm9f1USgUe/lY6fvFKPl4/\nWIIZEYGImx40+i/SuNmO32y+DFNf//GblR3VEAD2sGlU7HETuaE2Qzf+39vnYBFF/OC2VIQHq6bk\nfbUhKty/ORVmswX/+3YRTN1cae5M1h72z/P+GwUNZ21J22q4VeREAzFxE7kZs8WC//37ebQZevAP\nN8QjZebUlrGcGx+GjRm
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fadb8f2ee80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R score: 0.226100740276\n"
     ]
    }
   ],
   "source": [
    "# do not change\n",
    "\n",
    "# predict from test\n",
    "preds = mlp.predict(X_test)\n",
    "\n",
    "plt.plot(x, y)\n",
    "plt.scatter(X_test, preds, color='g')\n",
    "plt.show()\n",
    "\n",
    "print('R score:',r2_score(y_test, preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [MLP documentation](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Licence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/).  \n",
    "\n",
    "© Óscar Araque, Universidad Politécnica de Madrid."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}