mirror of
https://github.com/gsi-upm/sitc
synced 2024-11-04 23:21:42 +00:00
563 lines
36 KiB
Plaintext
563 lines
36 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"![](files/images/EscUpmPolit_p.gif \"UPM\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Course Notes for Learning Intelligent Systems"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Department of Telematic Engineering Systems, Universidad Politécnica de Madrid, © Carlos A. Iglesias"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## [Introduction to Machine Learning](2_0_0_Intro_ML.ipynb)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Table of Contents\n",
|
|
"* [kNN Model](#kNN-Model)\n",
|
|
"* [Load data and preprocessing](#Load-data-and-preprocessing)\n",
|
|
"* [Train classifier](#Train-classifier)\n",
|
|
"* [Evaluating the algorithm](#Evaluating-the-algorithm)\n",
|
|
" * [Precision, recall and f-score](#Precision,-recall-and-f-score)\n",
|
|
"\t* [Confusion matrix](#Confusion-matrix)\n",
|
|
"\t* [K-Fold validation](#K-Fold-validation)\n",
|
|
"* [Tuning the algorithm](#Tuning-the-algorithm)\n",
|
|
"* [References](#References)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# kNN Model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The goal of this notebook is to learn how to train a model, make predictions with that model and evaluate these predictions.\n",
|
|
"\n",
|
|
"The notebook uses the [kNN (k nearest neighbors) algorithm](https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Loading data and preprocessing\n",
|
|
"\n",
|
|
"The first step is loading and preprocessing the data as explained in the previous notebooks."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# library for displaying plots\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"# display plots in the notebook \n",
|
|
"%matplotlib inline"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"## First, we repeat the load and preprocessing steps\n",
|
|
"\n",
|
|
"# Load data\n",
|
|
"from sklearn import datasets\n",
|
|
"iris = datasets.load_iris()\n",
|
|
"\n",
|
|
"# Training and test spliting\n",
|
|
"from sklearn.model_selection import train_test_split\n",
|
|
"\n",
|
|
"x_iris, y_iris = iris.data, iris.target\n",
|
|
"\n",
|
|
"# Test set will be the 25% taken randomly\n",
|
|
"x_train, x_test, y_train, y_test = train_test_split(x_iris, y_iris, test_size=0.25, random_state=33)\n",
|
|
"\n",
|
|
"# Preprocess: normalize\n",
|
|
"from sklearn import preprocessing\n",
|
|
"scaler = preprocessing.StandardScaler().fit(x_train)\n",
|
|
"x_train = scaler.transform(x_train)\n",
|
|
"x_test = scaler.transform(x_test)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"source": [
|
|
"## Train classifier"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The usual steps for creating a classifier are:\n",
|
|
"1. Create classifier object\n",
|
|
"2. Call *fit* to train the classifier\n",
|
|
"3. Call *predict* to obtain predictions\n",
|
|
"\n",
|
|
"Once the model is created, the most relevant methods are:\n",
|
|
"* model.fit(x_train, y_train): train the model\n",
|
|
"* model.predict(x): predict\n",
|
|
"* model.score(x, y): evaluate the prediction"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
|
|
" metric_params=None, n_jobs=1, n_neighbors=15, p=2,\n",
|
|
" weights='uniform')"
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from sklearn.neighbors import KNeighborsClassifier\n",
|
|
"import numpy as np\n",
|
|
"\n",
|
|
"# Create kNN model\n",
|
|
"model = KNeighborsClassifier(n_neighbors=15)\n",
|
|
"\n",
|
|
"# Train the model using the training sets\n",
|
|
"model.fit(x_train, y_train) "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Prediction [1 0 1 1 1 0 0 1 0 2 0 0 1 2 0 1 2 2 1 1 0 0 1 0 0 2 1 1 2 2 2 2 0 0 1 1 0\n",
|
|
" 1 2 1 2 0 2 0 1 0 2 1 0 2 2 0 0 2 0 0 0 2 2 0 1 0 1 0 1 1 1 1 1 0 1 0 1 2\n",
|
|
" 0 0 0 0 2 2 0 1 1 2 1 0 0 2 1 1 0 1 1 0 2 1 2 1 2 0 2 0 0 0 2 1 2 1 2 1 2\n",
|
|
" 0]\n",
|
|
"Expected [1 0 1 1 1 0 0 1 0 2 0 0 1 2 0 1 2 2 1 1 0 0 2 0 0 2 1 1 2 2 2 2 0 0 1 1 0\n",
|
|
" 1 2 1 2 0 2 0 1 0 2 1 0 2 2 0 0 2 0 0 0 2 2 0 1 0 1 0 1 1 1 1 1 0 1 0 1 2\n",
|
|
" 0 0 0 0 2 2 0 1 1 2 1 0 0 1 1 1 0 1 1 0 2 2 2 1 2 0 1 0 0 0 2 1 2 1 2 1 2\n",
|
|
" 0]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(\"Prediction \", model.predict(x_train))\n",
|
|
"print(\"Expected \", y_train)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Accuracy in training 0.964285714286\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Evaluate Accuracy in training\n",
|
|
"\n",
|
|
"from sklearn import metrics\n",
|
|
"y_train_pred = model.predict(x_train)\n",
|
|
"print(\"Accuracy in training\", metrics.accuracy_score(y_train, y_train_pred))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Accuracy in testing 0.921052631579\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Now we evaluate error in testing\n",
|
|
"y_test_pred = model.predict(x_test)\n",
|
|
"print(\"Accuracy in testing \", metrics.accuracy_score(y_test, y_test_pred))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now we are going to visualize the Nearest Neighbors classification. It will plot the decision boundaries for each class.\n",
|
|
"\n",
|
|
"We are going to import a function defined in the file [util_knn.py](files/util_knn.py) using the *magic command* **%run**."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"ename": "AttributeError",
|
|
"evalue": "module 'matplotlib.colors' has no attribute 'to_rgba'",
|
|
"output_type": "error",
|
|
"traceback": [
|
|
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
|
"\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)",
|
|
"\u001b[0;32m<ipython-input-7-626d889b2bd3>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0mget_ipython\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_line_magic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'run'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'util_knn.py'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mplot_classification_iris\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
|
|
"\u001b[0;32m~/GoogleDrive/cursos/sitc/sitc/ml1/util_knn.py\u001b[0m in \u001b[0;36mplot_classification_iris\u001b[0;34m()\u001b[0m\n\u001b[1;32m 49\u001b[0m % (n_neighbors, weights))\n\u001b[1;32m 50\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 51\u001b[0;31m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
|
|
"\u001b[0;32m~/anaconda3/lib/python3.5/site-packages/matplotlib/pyplot.py\u001b[0m in \u001b[0;36mshow\u001b[0;34m(*args, **kw)\u001b[0m\n\u001b[1;32m 242\u001b[0m \"\"\"\n\u001b[1;32m 243\u001b[0m \u001b[0;32mglobal\u001b[0m \u001b[0m_show\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 244\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_show\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 245\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 246\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/anaconda3/lib/python3.5/site-packages/ipykernel/pylab/backend_inline.py\u001b[0m in \u001b[0;36mshow\u001b[0;34m(close, block)\u001b[0m\n\u001b[1;32m 37\u001b[0m display(\n\u001b[1;32m 38\u001b[0m \u001b[0mfigure_manager\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcanvas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfigure\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 39\u001b[0;31m \u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0m_fetch_figure_metadata\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfigure_manager\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcanvas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfigure\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 40\u001b[0m )\n\u001b[1;32m 41\u001b[0m \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/anaconda3/lib/python3.5/site-packages/ipykernel/pylab/backend_inline.py\u001b[0m in \u001b[0;36m_fetch_figure_metadata\u001b[0;34m(fig)\u001b[0m\n\u001b[1;32m 172\u001b[0m \u001b[0;34m\"\"\"Get some metadata to help with displaying a figure.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 173\u001b[0m \u001b[0;31m# determine if a background is needed for legibility\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 174\u001b[0;31m \u001b[0;32mif\u001b[0m \u001b[0m_is_transparent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_facecolor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 175\u001b[0m \u001b[0;31m# the background is transparent\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 176\u001b[0m ticksLight = _is_light([label.get_color()\n",
|
|
"\u001b[0;32m~/anaconda3/lib/python3.5/site-packages/ipykernel/pylab/backend_inline.py\u001b[0m in \u001b[0;36m_is_transparent\u001b[0;34m(color)\u001b[0m\n\u001b[1;32m 193\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_is_transparent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcolor\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 194\u001b[0m \u001b[0;34m\"\"\"Determine transparency from alpha.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 195\u001b[0;31m \u001b[0mrgba\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcolors\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_rgba\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcolor\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 196\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mrgba\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0;36m.5\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;31mAttributeError\u001b[0m: module 'matplotlib.colors' has no attribute 'to_rgba'"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%run util_knn.py\n",
|
|
"plot_classification_iris()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Evaluating the algorithm"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Precision, recall and f-score"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"For evaluating classification algorithms, we usually calculate three metrics: precision, recall and F1-score\n",
|
|
"\n",
|
|
"* **Precision**: This computes the proportion of instances predicted as positives that were correctly evaluated (it measures how right our classifier is when it says that an instance is positive).\n",
|
|
"* **Recall**: This counts the proportion of positive instances that were correctly evaluated (measuring how right our classifier is when faced with a positive instance).\n",
|
|
"* **F1-score**: This is the harmonic mean of precision and recall, and tries to combine both in a single number."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
" precision recall f1-score support\n",
|
|
"\n",
|
|
" setosa 1.00 1.00 1.00 8\n",
|
|
" versicolor 0.79 1.00 0.88 11\n",
|
|
" virginica 1.00 0.84 0.91 19\n",
|
|
"\n",
|
|
"avg / total 0.94 0.92 0.92 38\n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(metrics.classification_report(y_test, y_test_pred, target_names=iris.target_names))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Confusion matrix"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Another useful metric is the confusion matrix"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[[ 8 0 0]\n",
|
|
" [ 0 11 0]\n",
|
|
" [ 0 3 16]]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(metrics.confusion_matrix(y_test, y_test_pred))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We see we classify well all the 'setosa' and 'versicolor' samples. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### K-Fold validation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In order to avoid bias in the training and testing dataset partition, it is recommended to use **k-fold validation**."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[ 0.93333333 0.8 1. 0.93333333 0.93333333 0.93333333\n",
|
|
" 1. 1. 0.86666667 1. ]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from sklearn.model_selection import cross_val_score, KFold\n",
|
|
"from sklearn.pipeline import Pipeline\n",
|
|
"from sklearn.preprocessing import StandardScaler\n",
|
|
"\n",
|
|
"# create a composite estimator made by a pipeline of preprocessing and the KNN model\n",
|
|
"model = Pipeline([\n",
|
|
" ('scaler', StandardScaler()),\n",
|
|
" ('kNN', KNeighborsClassifier())\n",
|
|
"])\n",
|
|
"\n",
|
|
"# create a k-fold cross validation iterator of k=10 folds\n",
|
|
"cv = KFold(10, shuffle=True, random_state=33)\n",
|
|
"\n",
|
|
"# by default the score used is the one returned by score method of the estimator (accuracy)\n",
|
|
"scores = cross_val_score(model, x_iris, y_iris, cv=cv)\n",
|
|
"print(scores)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"source": [
|
|
"We get an array of k scores. We can calculate the mean and the standard error to obtain a final figure"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Mean score: 0.940 (+/- 0.021)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from scipy.stats import sem\n",
|
|
"def mean_score(scores):\n",
|
|
" return (\"Mean score: {0:.3f} (+/- {1:.3f})\").format(np.mean(scores), sem(scores))\n",
|
|
"print(mean_score(scores))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"So, we get an average accuracy of 0.940."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Tuning the algorithm"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We are going to tune the algorithm, and calculate which is the best value for the k parameter."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 18,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<matplotlib.text.Text at 0x7fc7cb526160>"
|
|
]
|
|
},
|
|
"execution_count": 18,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEPCAYAAABRHfM8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm4HNV55/Hv72pDC0JCQqDtSmKRWMxiMGIXLWwHOWFC\nRjxJYBJjMomTzBhnJjNmgAwJV3HsQB4ncWJiJ3G84OCJ7NiTmJCxjbF0EQZsZJBkI0sIY+lqRUhC\nQqAFLfedP06VbqvVfbu6u6rX9/M8/ai7urrr3KPq81adVWaGc845V05XoxPgnHOuNXjAcM45l4gH\nDOecc4l4wHDOOZeIBwznnHOJeMBwzjmXSOYBQ9ICSWslrZN0d5H3uyU9IWmVpCWSpuS9d1TSC5JW\nSPrXrNPqnHOuNGU5DkNSF7AOeDewFVgO3Gpma/P2+SrwqJk9IikH/Gczuz16b6+Zjc0sgc455xLL\n+g5jLvCymfWZ2WFgMXBzwT7nA0sBzKy34H1lnD7nnHMJZR0wpgKb8l5vjrblWwksBJC0EBgjaXz0\n3ghJz0l6RlJhoHHOOVdHzdDofReQk/Q8cB2wBTgavTfDzOYCvwZ8UtKsBqXROec63tCMv38L0J33\nelq07Rgz2wbcAiBpNHCLme3New8zWy+pF3gnsD7/85J8MiznnKuCmVVU7Z/1HcZy4GxJMyQNB24F\nHs3fQdIESXGi7wU+H20fF30GSROBq4GfFDuImfkjpcf999/f8DS008Pz0/OzWR/VyDRgmNlR4E7g\ncWA1sNjM1khaJOmmaLcc8JKktcAk4GPR9vOAH0paAXwX+FPL613lnHOuvrKuksLMvgXMKdh2f97z\nrwNfL/K5Z4GLsk6fc865ZJqh0ds1kVwu1+gktBXPz3R5fjZWpgP36kGS1fI3bNwIP/0p3HBDioly\nLW/XLnj9dTjnnEanpPFeew3eegvOPLPRKXFpkoQ1WaN303vpJfiTP2l0KlyzefhhuO++RqeiOfz9\n38NHP9roVLhm0PEBY+ZM2LCh0alwzWb9+vBwnhduQMcHjO5u2LIFjhxpdEpcM/FCcoDnhYt1fMAY\nMQImTYLNmxudEtdM1q+HnTtD3X2nW78+/D4OH250SlyjdXzAAJg1y6+g3ACzUE05daqfF0eOhDvw\n008PHURcZ/OAgQcMd7wdO+Ckk+Cii7x9a/PmECxmz/a8cB4wAA8Y7njr14dzws8Lzwt3PA8YeE8p\ndzwvJAd4Xrh8HjDwH4M7nheSAzwvXD4PGPiPwR3PC8kBnhcunwcMQm+YnTvh4MFGp8Q1g8JCssVn\nz6mJBwyXzwMGMGQITJ8OfX2NTolrBhs2hHatceNAgt27G52ixonzYvJk2LMHDhxodIpcI3nAiPgV\nlAM4ehQ2bQqFpNTZ58XBg+HOe+pU6OoKsyJ455DO5gEj4j2lHMDWrXDqqWEcBnR2wOjrC3feQ4aE\n152cFy7IPGBIWiBpraR1ku4u8n63pCckrZK0RNKUgvdPlrRJ0l9nmU7/MTgYqLOPdfJ54XnhCmUa\nMCR1AQ8BNwIXALdJOrdgt08AXzSzi4E/Bh4oeP+jwJNZphP8x+ACLyQHeF64QlnfYcwFXjazPjM7\nDCwGbi7Y53xgKYCZ9ea/L+kywjrfj2ecTv8xOMALyXyeF65Q1gFjKrAp7/XmaFu+lcBCAEkLgTGS\nxksS4e7jI0BFq0JVw38MDgZ6BcVmzuzc82L9+hPzwtv5OtvQRicAuAt4SNIdwDJgC3AU+K/Av5vZ\n1hA7SgeNnp6eY89zuVxV6/5OmgT798Obb8LJJ1f8cdcm1q+H228feD1zZmj8NQu9pjrJhg1+h9FO\nent76e3trek7Ml3TW9KVQI+ZLYhe3wOYmT1YYv/RwBoz65b0CHAt0A+cDAwDPm1mf1DwmZrW9M53\n/vnwla/AhRem8nWuBXV3w5NPHl9QTpoEq1aFsQidZOJEWL06zFYLIWiefHKY7vyUUxqbNle7ZlzT\nezlwtqQZkoYDtwKP5u8gaUJU/QRwL/B5ADP7dTObaWZnEqqlvlQYLNLmV1Cd7dAh2L49dCXN14nn\nxZtvhkF6kyYNbOv0cSku44BhZkeBOwmN1quBxWa2RtIiSTdFu+WAlyStJTRwfyzLNA3GfwydbeNG\nmDIFhhZU1HbieRG3XxRWw3ViXrgBmbdhmNm3gDkF2+7Pe/514OtlvuNh4OFMEpjHfwydrbCRN9aJ\n50VhD6lYJ3cCcD7S+zidWDC4AYWNvLFOLCQHC57eU6pzecDI4wGjs5W6qu7EQrJU8PTfSGfzgJEn\n7mfeydNZd7LBAkanFZKeF64YDxh5xo8Ps3K+/nqjU+IaoVQh2d0dupIeOVL/NDVKubstv6jqTB4w\nCvgVVOcqVUiOGBG6l27eXP80NYJZ6bwYOzbkx44d9U+XazwPGAU8YHSm/fvhjTfgjDOKv99J58Wu\nXaFr8bhxxd/3KUI6lweMAp1UMLgBGzbAjBmhSrKYTuopVaqHVMx/I53LA0YB/zF0plJVMLFO6ilV\nqodUzH8jncsDRoFOupJ0A5IEjE45LzwvXCkeMAp00pWkG+CF5ADPC1eKB4wC8XTW/f2NTomrJy8k\nB3heuFI8YBQYPTp0HXz11UanxNVT4cJJhaZOhZ074eDBeqWoccoFjBkzYNMmv6jqRB4wivArqM5T\nrpAcMiRMe97XV780NUJ/f/gbBwueI0eGQa5bt9YtWa5JeMAowgNGZ9mzJ4zinjBh8P06YfzBq6+G\nxZFGjRp8P/+NdCYPGEV4T6nOEt9dlFuCtRMKyXJ3WrFOyAt3Ig8YRXhPqc7iheQAzws3mMwDhqQF\nktZKWifp7iLvd0t6QtIqSUskTcnb/rykFyT9WNLvZJ3WmP8YOosXkgM8L9xgMg0YkrqAh4AbgQuA\n2ySdW7DbJ4AvmtnFwB8DD0TbtwFXmtmlwBXAPZJKzPSTLv8xdJZyPaRinXBeJA0YndCe406U9R3G\nXOBlM+szs8PAYuDmgn3OB5YCmFlv/L6ZHY4+AzASKFPDnJ7u7tADpJOms+5kflU9wPPCDSbrgDEV\n2JT3enO0Ld9KYCGApIXAGEnjo9fTJK0C+oAHzawuoyOGD4fTTw99zV37S1pITpoUZrV9883s09Qo\nSe+2pk8PPaoOHy6/r2sfQxudAOAu4CFJdwDLgC3AUQAz2wxcHFVFfUPS18zshJn4e3p6jj3P5XLk\ncrmaExX3lEpSkLjWZVZ+sr2YNFAVc+GFGSesAY4cCXfW3d3l9x02DCZPho0b4ayzsk+bq11vby+9\nvb01fUfWAWMLkH/6TYu2HWNm24BbACSNBm4xs70F+7wq6UXgOuD/Fh4kP2CkxXtKdYbXXgsD0U4+\nOdn+cVVMOwaMTZvCnfXw4cn2j/PCA0ZrKLyYXrRoUcXfkXWV1HLgbEkzJA0HbgUezd9B0gTpWA/4\ne4HPR9unSjopej4euBZ4KeP0HuN1tJ2h0rvIdj4vPC9cOZkGDDM7CtwJPA6sBhab2RpJiyTdFO2W\nA16StBaYBHws2n4e8ANJKwiN4n9mZquzTG8+/zF0hqR19rF2Pi8qDRjeU6rzZN6GYWbfAuYUbLs/\n7/nXga8X+dwTwMVZp6+Udi4Y3IBqrqqXLcsuPY1UTV5885vZpcc1Hx/pXYIHjM7gV9UD/G7LleMB\no4QpU2DXLjhwoNEpcVmqtt7eLLs0NYq3YbhyPGCUEE9nvXFjo1PislRpITl+PHR1weuvZ5emRqk0\nLyZPhjfeCGNTXGfwgDEIv4Jqb0ePhq6kM2ZU9rl2PC8OHAhBcMqU5J/p6gpjNtq1is6dyAPGINqx\nYHADtm4Na2CcdFJln2vH86KvL9xRDxlS2efauU3HncgDxiDasWBwA6odyd+O54XnhUvCA8Yg/MfQ\n3qotJNvxqrrSHlIx/410Fg8Yg/CV99qbX1UP8LxwSXjAGITPJ9XevJAc4HnhkvCAMYhJk0LvkXae\nzrqT1VIl1dcH/f2pJ6lhPGC4JDxgDCKeztp/EO2p2nr70aNh7NiwHkS7qDZgTJgQ1sTYsyf9NLnm\n4wGjDL+Cak+HDsH27aEraTXa6bzYuxcOHoTTTqv8s5JX3XYSDxhltFPB4AZs3BgGqQ2tcvrNduop\nFd9pHVtkoEL+G+kcHjDK8Cqp9lTraortVEh6XrikPGCU4bfb7ckLyQGeFy4pDxhl+I+hPVXb4B1r\np/Oi1oDRTtVzbnCZL6AkaQHwSUJw+pyZPVjwfjdhWdbTgF3Ar5vZVkkXA58BTgaOAh83s69mnd5C\n+dNZV1vH6ypz8GDl8ztVav16+IVfqP7z9QoY9cqLvKWeK1avvHj99eaYGXfSpOTrnmelHudFMZkG\nDEldwEPAu4GtwHJJ3zCztXm7fQL4opk9IikHPADcDuwH3m9mr0iaDDwv6VtmtjfLNBcaNy40jO7a\nBRMn1vPInam/H845Bx5/HM47L7vj1HpV3d0NW7bAkSPVN5yXc+RImEn3+9+vLa3lpFEltWFDthdV\nBw+GO5mxY7P5/qT274fbb4dPfrJxadi7N+TFli0wcmR9j531HcZc4GUz6wOQtBi4GcgPGOcDvw9g\nZr2SvhE9fznewcy2SXqNcBdS14ABA1dQHjCyt3o1bN4M3/lOcweM4cPh9NNDWmup2hrM88/Da6/B\nd78Lv/Vb2RzDrPbqubFjYcQI2LEjXH1n4dln4fzzQ/BspOXL4Y47GpuGp56C3bvD/1uWv5Fism7D\nmApsynu9OdqWbyWwEEDSQmCMpPH5O0iaCwwzs1cyTGtJ3lOqfpYuDYXO0qXZHWPfvnCVdsYZtX1P\n1lUx9ciLXbvCHdK4cbV9Tz3y4oYbsvv+pN75znBlv31749IQnw+NaDfKvA0jgbuAhyTdASwDthDa\nLACIqqO+BLy/1Bf09PQce57L5cjVUiFbhPeUqp8lS+Cee+CjHw0LHFW6PkMSGzaEqp6uGi+X4kJy\n/vxUknWCJUvg3nvhwQezq+6p9U4rFufFFVfU/l3FLFkCeT/zhhk6FK6/PhTat97amDQsWQIXXFB5\ngO7t7aW3t7emY2cdMLYA3Xmvp0XbjjGzbcAtAJJGA7fE7RSSTgYeA+41s+WlDtKT8Zk0a1aoKnHZ\nOnoUnnwS/vZv4bOfhZUr4bLL0j9OrVUwsSyvqt9+O1TDfPWrob587dpsqh/SChhZ9pR6661wLlx9\ndTbfX6n580Oh3YiA8frr8NOfwl13VX7uFV5ML1q0qOLjZ10ltRw4W9IMScOBW4FH83eQNEE6du10\nL6HHFJKGAf8KPGxm/5JxOgfVTl0om9nKlWH09RlnhOqHrKpi0r6qzsJzz8G554aqok7Pi6efDhcO\no0Zl8/2VyvL/o5wnn4RrroHZsxtTJmUaMMzsKHAn8DiwGlhsZmskLZJ0U7RbDnhJ0lpgEvCxaPuv\nANcCd0haIekFSRdlmd5SPGDUx5IlA9U78VVcFlqhkPS8GLBkSXO0X8Te8Y4w2eLGjfU/dnxeNKpM\nynzgnpl9y8zmmNk5ZvZAtO1+M3ssev51M5ttZuea2W+b2eFo+5fNbISZXWpm74z+/VHW6S2mHaez\nbkb5BUMuB9/7XpgJNW1pVsPUo5CcPx96e7M5/9avb/7qufzg2Qy6usL52Yi7jPi8aNRgSR/pncCo\nUXDKKbBtW6NT0r4OHw5VD9dfH15PmABnnQU//GH6x0orYEydGnoZHTxY+3fl278/dKm99trwetq0\nkB8//nG6x4FQ6KQVPDdtCu1QadqzJ7TfZNWYXq1GVEtt3w5bt4aeWqedFtq59tZ5kIEHjIS8p1S2\nli8PAWLChIFtWVXFpBUwhgwJ06P39dX+XfmeeQYuvhjGjBnYlkVe9PeHtKdxh3HSSXDqqaFAS9Oy\nZXDVVWGcRzO54Ybw/2FWv2MuXQrz5oXzrlFr9XjASMjbMbJVrJ46/lGmac+ecBV86qnpfF8W50W9\n8mLbttConlZjchbVJM1WHRWbPTuMxH+ljiPDCs+LRpRJHjAS8oCRrWIDs+bNC72F0qzyie8u0hrT\nkMV5USwvcrkwwvfIkfSOk9adVqxeedEMpPpXSxXmhQeMJuYBIzsHD8IPfgDXXXf89rFj058OotkL\nyb17Q1vFVVcdv33SpFD99cIL6R2r2fNix45QZZbFWJw0ZHHXV8rGjeHu+IILBrZ5wGhiHjCy8+yz\noatisYnl0v5RNnsh+dRTMHdu8ZlIOy0ventDw39WkzvWav78cNVfj3aMpUvD8fJnJ2hETykPGAn5\nfFLZGazaIe3b/rQLybR/tPXMi7RGvMfSDhjNWh0VmzUrzBa7Zk32xyqWF36H0cS6u0MPkCzGBXS6\nwRo2r7kGVqwIEwamodmvqgfLi3nzQg+qQ4fSOVYr5EUzBwyoT7WUWfHzIn+tnnrxgJHQ8OFhyorN\nmxudkvYSzxN0zTXF3x81Ci69NAziS0PaV9WTJsGBA/Dmm7V/VzxP0OWXF39//HiYMye096Qh7YAx\nbRq8+mo6AW3LltCGcVFD5nZILstR+LFXXgk9+2bPPn77KaeEcmnnzmyPn88DRgW8HSN9SeYJSusq\nLl77Ic1CMs3+8PE8QYOt5pZWtdThw6FbbXd3+X2TGjYszAW2aVP5fctZujT0DKt1RuGszZ8f/t+y\nnAUiro4q1rOv3mVSk/93NBcPGOlL0s8+rULytddCnfPJJ9f+XfnSOi+S5EVaV7SbNoU75mHDav+u\nfGnlRbO3X8SmTg2jrletyu4Yg50X9W5b9YBRAQ8Y6UtSMFxxRWhY3LOntmOlXQUTSzNglMuLa68N\n06UcOFDbsVohL5pxwF4xWVZLxe0Xpc6Les9AUTZgSPpw4Qp4ncp7SqVrz54QCMrNEzRiBFx5ZZgm\nohZZFZJp9JTKnydoMCefHKYNeeaZ2o6XdltOLI2AsX59CIj1Xn60WlkO4PvJT2D06NL/V81YJXU6\nsFzSVyUtyFu7ouP4HUa6li0LgSDJPEFp/Cib+ao6f56gctK4om32vChVZ9+M4lH4WfSgjMdflNJ0\nAcPM7gPOAT4H3AG8LOnjks7KOG1NxycgTFcl9dRpFJLNfFVdSRfSNDoBNPPdVitVR0Fow5gxI8ww\nnLZy50XTBQwAMzPg1ehxBBgPfE3Sn5X7bHRXslbSOkl3F3m/W9ITklZJWiJpSt5735S0W9KjhZ9r\nhClTQtfHWuuPXVBJIfmud4WCaMeO6o+X9VV1Lf3hKwmeV10Vpg+ppStvs95hlKuzb1ZZVEv194fR\n7oMFz5kzw7Qh9VqrJ0kbxn+T9DzwZ8DTwIVm9l+Ay4jW4h7ks13AQ8CNwAXAbZLOLdjtE8AXzexi\n4I+BB/Le+zPg1xP+LZnLajrrTrRjRwgASecJGjo0zDX15JPVHzOrQnLcuJC+Xbuq+3yxeYIGM3Jk\nGKvx1FPVHQ+yy4vJk+GNN8KaHtVYty703DrzzHTTlbUsBvCtWhXG+UyZUnqfkSPD+Zf2tPKlJLnD\nOBVYaGY3mtk/562I1w/cNPhHmQu8bGZ90ecWAzcX7HM+sDT6zt78981sKfBWkj+kXrwdIx1PPhkC\nQCXzBNVSLXX0aOhKOmNGdZ8vp5bzotg8QeXUUkAdOAC7dw9eEFWrqyuM7ai2WiqujmqV9ovYvHlh\nksy3307vO5PeadWzqjzJKfpN4PX4haSxkq4AMLNys6hMBfKH8WyOtuVbCSyMvnshMKaZe2V5T6l0\nVFPtUEshuWULTJxYfFK/NNRSd1/NmINaqkD6+sKdclaD4moJnq1YHQXhKv/cc9MbhQ+VBYx6lUlJ\nTpnPcPxV/lvRtrTcBeSiaq/rgC1Aygs9psfvMNJRTcPmxReHqqxqbr+zqoKJVXtelJonqJzLL4eX\nXw5tapVq1rxIUmffzNKsljp8OEyHk8uV37eeZVKSCgFFjd5AqIqSlLQiYQuQP/nAtGjbMWa2jagt\nRNJo4BYzq2il2p6enmPPc7kcuSS5XKVZs9Jdk6ATbd0aCv6LL67sc11dYc3vpUvh136tss9m1UMq\nNmsWrF5d+ed+9rPi8wSVM3w4XH116Jr8S79U2WezDhjV3m29+GK4Up8+Pe0U1cf8+fCnfwp5xVHV\nnn8+5OPEieX3nTUrTLFTTm9vL729vTWlK0nB/zNJv8fAXcV/BX6W8PuXA2dLmgFsA24FbsvfQdIE\n4PUoKN0LfL7gOxQ9SupJ438oIb/DqN3SpaHgr6ZKJL6KqzRg1OOq+rHHKv9cLXX2cZtOswWMWbPC\nSomVatXqqNi114aCfv/+2pe9rSQvZs2CRx4pv1/hxfSiRYsqTleSn+zvAlcT7gw2A1cAv53ky83s\nKHAn8DiwGlhsZmskLZIUN5jngJckrQUmAR+LPy9pGfAV4AZJGyW9N9FflSEPGLWrpWCotu6+Wath\nas2LaqpAmjUvWmX+qFLGjIFLLkl2tV9OJXlRz3ZVWT0nU8+AdFyNWebMwomxbVvxFeJceWeeGa7G\nzz+/8s+aha6bzz5bWaE3bx4sWpRd/fj+/XDqqeHfpHdO8d/y/e9XV1125EgYNLZ2LZx+evLPXXYZ\nfOYzYWW/LOzcCeecE3piJXXkSKh+eemlyv6WZvOHfxiqGD/+8eq/4+23Q15s3hymMC/n8OEwfci+\nfZVNJikJM6vo3jbJOIyTJH1I0qclfT5+VHKQdpLmdNadaP36UKhWO0+QNLA0ZqXHzfKqetSo8ON+\n9dXkn1mzJvSjr7ZtJR6bUmm1dNbtORMmhABQyWSRK1aE9TRaOVhAOg3f3/9++H0kCRYQgsTkyelM\nK19OkmuhfwTOIAy+e5LQcJ3CcjGty6ulqhePOailn32l1VKHDoWpzadNq/6YSVR6XqRRZ19pAbV3\nb7iCPe202o47GKnysQGtXh0Vu+qq0Hi/t6JuO8erJi/qVSYlCRhnm9kfAvvM7GHgFwjtGB3L55Sq\nXhoFQ9zYm7QmcuPGMEitkkGC1aj0R5tGXlQaPNevD3cXWQ+Mq/QuvNXmjyrlpJNCVV8to/CryYtm\nChjxHIx7JL0DOIXQON2x/A6jOtWOOSh01llhmpZ165Ltn3V1VKyS8yKtMQfveEdoK0haHdGMeXHo\nUJiu/frrs01TvdRSLbV/f+i2f+21lX2umQLG30cjr+8DHgV+AjyYaaqanAeM6qxbFwr6s2qc51iq\n7Mq6GQvJVatCtVCt03N0dYXBXa2cF8uXh0byU0/NNk31UkvAePrpsCbK6NGVfS6NWYKTGDRgRJMH\n7jWz3Wa2zMzONLNJZvZ32SeteXnAqE6a6xxUMq9UMxaSadbZt3rwbJfqqNjll8Mrr1Q3GWW1edEU\ndxjRBIP/K/tktJa4frbFeyTXXZoFQ9xTKsm0zvUqJCu5yks7L5K26WTdQypWacBohwbv2LBhcM01\n1c2sXG1eNEXAiDwh6SOSpks6NX5knrImNm5cOCl27mx0SlpHf3/51cMq0d0d/h+STMdRr0KyuztM\ne3LkyOD7HTkSGkXTmsFmzpzQF/9nCeZfqHfwLBfEDhwIVVLXXZd9muqpmmqpN94IS7JeeWXlx5sy\nJbRlZb1WT5KA8avAh4BlwPPR44dZJqoVeE+pyqxeHQr47u7y+yaVtFqqXoXk8OFhHEG5Buh4nqC0\nurYmbdMxq19ejB0begyVW/Dq2WfhwgvDWuXtpJqxQk89FXpYVTOjcq3Tyic+TrkdzGxWkUeLLW+S\nPm/HqEwW9dRJruL27Qt94s84I91jl5LkvMgiL5IEz507Q1BLOiCsVknzop2qo2LvfGe426xkIGet\neVGPAcVJRnrfXuyRbbKanweMymQxMGv+/DBb69FBJsPfsCEsmpTV2g+FGlVIxsFzsCqget1dxJLk\nRbsM2Cs0ZMjAzMpJ1ZoX9aj1SPIzujzvcR3QA/xihmlqCR4wkjt6NDQApn1Vffrpoe52xYrS+zRb\nIfn222Hqh3nz0j/uyJFhXqlSmi0v3nwzdC+++ur6pameKqmW2rUr9Kx617uqP149yqQkVVIfznt8\nELgUGJNtspqfzyeV3IoVoWDPYp6gctVS9S4ky/WU+sEPwsps48alf+xy1VL1avyPlSvAvve9UECO\nHFm/NNVTJQ3fvb1hsF4lkwcWaoqAUcQ+oI4/webkdxjJZVntUK6xt9kKySzr7FsteLZrdVTsggtC\n+9nGjeX3TSMvmiJgSPo3SY9Gj8eAl4B/yTZZzW/mzHAiJBkH0OmyLCSvvz6Mjj10qPj7zVYNk2Uh\nOX9+qPordU42W16024C9QpWMwk8jL5oiYACfAP48evwpMM/M7sk0VS1g1KhQrbBtW6NT0twOHw4F\nelbzBJ16Kpx9dujLX0y9C8kpU8I628X6w+/fH7rUXnNNNseeOjVMLf6jHxV/vxF3GJs2Fe+UsHt3\nWPviijafxjRJtdS2baE31SWX1HasiRNDG9kbb9T2PYNJEjA2Aj8wsyfN7Glgl6SZSQ8gaYGktZLW\nSbq7yPvdkp6QtErSEklT8t77QPS5l5qxZ5ZXS5W3fHko0LOcJ2iwaql6F5JDhoQ1qfv6Tnzv6adD\noTAmwxbAUgVUf3+4I65n9dxJJ4X/961bT3xv2bLQ2D18eP3S0whJeq/19oYLqiFDajtWNdPKVypJ\nwPhnIP8m92i0raxoLqqHCGtpXADcJuncgt0+AXzRzC4G/hh4IPrseOCPCL2zrgDul1SnHuTJeMAo\nrx797Es19u7eHQrKek9qV+q8qEedfanguXUrjB9f/wbmUnnR7tVRsXPOCefgK6+U3ifNvMi6TEoS\nMIaa2bEa4uh50uuCucDLZtZnZoeBxcDNBfucDyyNvrs37/0bgcfN7A0z20NYF3xBwuPWhQeM8upR\nMFx3HTz3HBw8ePz2+O4i67UfCpVq7K1HXuRy4eq9cHqSejf+xwYLGO3c4B2LR+EPVi2VZl40Q8DY\nIenYuAtJNwNJZ1GaCuRPlLA52pZvJbAw+u6FwJjo7qLws1uKfLahvGvt4A4erM88QWPHhuklnn32\n+O3NVEju3RtWYrvqqmyPfdppYaDi888fv73eVXOxYsHztddC28all9Y/PY0wWMDo64O33go9qtLQ\nDAHjd4EkmwQ8AAAS0klEQVQ/kLRR0kbgbuB3UkzDXUBO0vOEgYFbCNVeTc/nkxrcs8+GBX7Gjs3+\nWMWqpRpVSBb70T71VGjgrWaeoEoVq5Zqprzo7Q0XEVmvgNgs4gF8xdoxli4Nd4Vp3QVnHTDK/peZ\n2SvAlZLGRK/fquD7twD5081Ni7blf/824BYASaOBW8xsr6QtQK7gs0WbNnt6eo49z+Vy5NKaBrQM\nr5IaXD3rqW+4Ae6/Hz760YFt69eHOuR6K3Ze1DMv5s+Hv/kbuCevL+P69dn1zhrMrFnwpS8dv61T\nqqNiM2eGBZF+8pMT7yTSzovBaj16e3vp7e2t7QBmNugD+DgwLu/1eOBPyn0u2ncI8FNgBqHdYyVw\nXsE+EwBFz/8E6Mk7ziuEJWHj5+OKHMMa5dAhs+HDw7/uRNdcY/ad79TnWPv2mY0ebfbmmwPbfv7n\nzb7xjfocP9/27Wannnr8tksuMXv66focf/duszFjzA4eHNh2/fVmTzxRn+Pn+9nPzKZPP37b7Nlm\nK1fWPy2N9Ju/afbXf338tv5+s2nTzNatS+84e/aE30F/f/l9o7KzbDme/0hSJfU+C43OcYDZDfx8\nwmB0FLiT0GC9GlhsZmskLZJ0U7RbDnhJ0lrCWuEfyzvORwlTqf8AWJSfjmYwbFiYBTXpesqd5K23\nYOXK+s0TNGoUXHZZmG4i1qhqmNNOC+03e/eG1/E8QZdfXp/jjxsXph957rmBbY3Ki+nTYfv2gYGV\nmzeH/LjwwvqnpZGKzSv105+Gaqqzz07vOKecEroqZ7VWT5KAMUTSiPiFpJHAiEH2P46ZfcvM5pjZ\nOWb2QLTtfjN7LHr+dTObbWbnmtlvW+hNFX/2i9HnZpvZl0odo5G8Wqq4730vFOCjRtXvmPmNi2aN\na/SWjm/sffLJUB1UyzxBlcpv0zl8OAwOmz69fsePDR0aBjPG02PEdfb1mj24WcyfH9pu8kfhx9VR\naffiy7JMSvLf9mXgu5J+U9JvAd8BHs4mOa3He0oV14h5gvIbe197LQSrRi3Mk/+jbUSdfX7w3LQJ\nJk+ub8DKlx88233+qFLiyTdXrRrYllVeNDRgmNmDhLaF84A5wLcJbRIO7ylVSiMKyblzw/Teu3c3\nrgomlv+jbUQhee21oWvt/v3Nkxdm8N3vdsaAvWLy7/rMsusI0eg7DIDtgAG/DNwArMkmOa3Hq6RO\ntGdPKLjnzq3vcUeMCOMcli1rnkLy1VfDKOta5wmq1JgxcPHF8MwzzZMX69eHtoxzC+d66BD5d32r\nV4e73xkZXHpnWetRMmBImi3p/qgx+lOEOaVkZvPN7KFsktN6PGCcaNmyUHCPSNzSlZ64WqpZCsml\nS9OZJ6gacQHVTHmRRZ19q8jlQtve4cPZ3nVmWesx2B3GWsLdxE1mdq2ZfYoWGVBXTx4wTtTIeYLi\n2/5mKiQbmRfNFDw7Zf6oUiZODFf/zz+fbV40qkpqIbANWCrps5LeDXTotUFpU6aEOvNi01l3qkYO\nzLrssjDdwnPPNbaQjKsFGpkXV10FP/5xmO68WQJGJzZ457vhBnjiiWyWLI5luVZPyYBhZv9qZrcC\n5xJGWP93YJKkz0j6ufST0pq6ukJ3RW/4DnbsCCfrZZc15vhDh4b1sn/0o8Z0qY2NGxd6Je3dm948\nQZUaOTK0I61e3di8OOOMsEbDiBGNDVzN4IYb4DOfCXkyeXI2xxg5MsxMXGxa+Vol6SW1z8z+j5n9\nB8L0HCsI80m5iPeUGhCvTdzIeYLiK7csGhQrMWtW48cczJ8fBnJNmVJ+36x0dYX/i/nzO7f9IjZv\nXugIkXXVXFYN3xX9rKPR138fPVzknHPgN34jjLLsdLt2wX33NTYN731v+MHUY6K/wcyZ0/gqmJ/7\nOVi8uPED5ebMCf8vne6UU8IklFnnRXwRm/ZM0fEcTi1LkjX6b9i3L0x54IIzz2zcILHY3r31mSV3\nMPv2haDViB5S+ZohL956KwykbHTgagZ794YutVnebf3v/x2qAP/oj0rvIwkzqygVHTLBcLZGjw5X\nUK55NLqAhHBeNINmyIssl6VtNfX4/5g1KywJnDaP984512ay6lrrAcM559pMVgHD2zCcc67NHD4c\nqkT37SvdnlhNG4bfYTjnXJsZNiyM80h7rR4PGM4514ayqJbKPGBIWiBpraR1kk4Y8CdpuqQlkl6Q\ntFLS+6LtwyR9XtKPJK2QdH3WaXXOuXaRRcDItFutpC7gIeDdwFZguaRvmNnavN3uA75iZn8n6Tzg\n/wGzgA8S1py9SNJpwDeBd2WZXuecaxeteIcxF3jZzPqipVcXAzcX7NMPxD2TxwFboufnA0sAzGwH\nsEeSBwznnEugFQPGVCC/2WVztC3fIuD9kjYBjwEfjravAn5R0hBJs4DLgAasSuycc60ni/mkmmGk\n923AF8zsLyVdCTwCXAB8nrAs7HKgD3iaEutx9PT0HHuey+XI5XLZptg555pc4aSovb299Pb21vSd\nmY7DiAJAj5ktiF7fQ2iXeDBvnxeBG81sS/T6FeAKM9tZ8F1PA79Z0P7h4zCcc66I/v4wf9fu3WHK\n80LNOA5jOXC2pBmShgO3Ao8W7NMHvAcgavQeYWY7JY2UNCra/l7gcGGwcM45V1xXF3R3p7v0QqZV\nUmZ2VNKdwOOE4PQ5M1sjaRGw3MweAz4CfFbS7xMawD8QfXwS8G1JRwkN4e/PMq3OOddu4obv885L\n5/t8ahDnnGtTv/u7cOGF8KEPnfheM1ZJOeeca5C0e0p5wHDOuTaV9vLRHjCcc65NpT14zwOGc861\nKQ8YzjnnEpk4EQ4dgjfeSOf7PGA451ybktJt+PaA4ZxzbSzNaikPGM4518bS7CnlAcM559qY32E4\n55xLxAOGc865RNIMGD6XlHPOtbE33oApU+Ctt0KvqZjPJeWcc+44p5wCI0bAjh21f5cHDOeca3Np\n9ZTygOGcc20urXYMDxjOOdfmWiZgSFogaa2kdZLuLvL+dElLJL0gaaWk90Xbh0r6oqQfSVodrQfu\nnHOuQi0RMCR1AQ8BNwIXALdJOrdgt/uAr5jZpcBtwKej7b8MDDezi4B3Ab8jqTvL9DrnXDtKaz6p\nrO8w5gIvm1mfmR0GFgM3F+zTD4yNno8jrN8NYMBoSUOAUcDbwN6M0+ucc22nVRq9pwKb8l5vjrbl\nWwS8X9Im4DHgw9H2rwH7gW3ABuATZrYn09Q651wbmjkTNm6E/v7avmdoKqmpzW3AF8zsLyVdCTxC\nqL66AjgCnAFMAJ6S9ISZbSj8gp6enmPPc7kcuVwu+1Q751yLGDkSRo3q5SMf6WXs2PL7l5LpSO8o\nAPSY2YLo9T2AmdmDefu8CNxoZlui1z8FrgR6gGfN7MvR9s8B3zSzrxUcw0d6O+dcGVdfDQ8+CNdd\nF14340jv5cDZkmZIGg7cCjxasE8f8B4ASecBJ5nZTmAjcEO0fTQhiKzNOL3OOdeW0ugplWnAMLOj\nwJ3A48BqYLGZrZG0SNJN0W4fAT4oaSXwZeAD0fa/AU6O7kB+AHzOzF7MMr3OOdeu0ugp5ZMPOudc\nB/iHf4Cnn4YvfCG8bsYqKeecc02g6auknHPONYc0AoZXSTnnXAc4fBjGjAnrYgwb5lVSzjnnShg2\nDCZPDgP4quUBwznnOkStPaU8YDjnXIeodU4pDxjOOdcham349oDhnHMdwgOGc865RDxgOOecS6TW\nRm8fh+Gccx2ivx9GjYLXX4fRo30chnPOuRK6uqC7G/r6qvx8uslxzjnXzGppx/CA4ZxzHcQDhnPO\nuUQ8YDjnnEuklp5SmQcMSQskrZW0TtLdRd6fLmmJpBckrZQUr//9nyStiLavkHRU0kVZp9c559pZ\nLXcYmXarldQFrAPeDWwlrPF9q5mtzdvn74AXzOzvojW9/5+ZzSr4nncA/2Jm5xQ5hnerdc65hHbs\ngDlzYPfu5utWOxd42cz6zOwwsBi4uWCffmBs9HwcsKXI99wWfdY551wNJk6EQ4eq+2zWAWMqsCnv\n9eZoW75FwPslbQIeAz5c5Ht+FfinTFLonHMdRArVUtUYmm5SqnIb8AUz+0tJVwKPABfEb0qaC+wz\ns5+U+oKenp5jz3O5HLlcLrPEOudcK+rt7aW3txeo/g4j6zaMK4EeM4sbsu8BzMwezNvnReBGM9sS\nvX4FuMLMdkav/wJ4zcweKHEMb8NwzrkK/N7vwac+1XxtGMuBsyXNkDQcuBV4tGCfPuA9AFGj94i8\nYCHgV/D2C+ecS021VVKZBgwzOwrcCTwOrAYWm9kaSYsk3RTt9hHgg5JWAl8GPpD3FfOAjWa2Ict0\nOudcJ6k2YPhstc4512G2boWpUyuvkvKA4ZxzHUhqvjYM55xzbcIDhnPOuUQ8YDjnnEvEA4ZzzrlE\nPGA455xLxAOGc865RDxgOOecS8QDhnPOuUQ8YDjnnEvEA4ZzzrlEPGA455xLxAOGc865RDxgOOec\nS8QDhnPOuUQyDxiSFkhaK2mdpLuLvD9d0hJJL0haKel9ee9dJOkZSS9KWhWt2uecc64BMg0YkrqA\nh4AbgQuA2ySdW7DbfcBXzOxS4Dbg09FnhwD/CPy2mb0DyAGHs0yv49gi8S4dnp/p8vxsrKzvMOYC\nL5tZn5kdJqzNfXPBPv3A2Oj5OGBL9PzngFVm9iKAme32lZKy5z/IdHl+psvzs7GyDhhTgU15rzdH\n2/ItAt4vaRPwGPDhaPtsAEnfkvRDSXdlnFbnnHODaIZG79uAL5jZdOAXgEei7UOBa6L3rwP+o6T5\njUmic865TNf0lnQl0GNmC6LX9wBmZg/m7fMicKOZbYlevwJcAbwbWGBmvxFtvw84YGZ/XnAMr6Zy\nzrkqVLqm99CsEhJZDpwtaQawDbiVcMeQrw94D/CwpPOAEWa2U9K3gbsknQQcAa4H/qLwAJX+wc45\n56qTacAws6OS7gQeJ1R/fc7M1khaBCw3s8eAjwCflfT7hAbwD0Sf3SPpL4AfRtv/3cy+mWV6nXPO\nlZZplZRzzrn20QyN3lUrNyjQVUbShmiA5ApJzzU6Pa1G0uckbZf0o7xt4yU9LuklSd+WdEoj09gq\nSuTl/ZI2R4N8X5C0oJFpbCWSpkUDpFdL+rGk34u2V3R+tmzASDgo0FWmH8iZ2TvNbG6jE9OCvkA4\nH/PdAzxhZnOAJcC9dU9VayqWlwB/YWaXRo9v1TtRLewI8D/M7ALgKuBDUXlZ0fnZsgGDZIMCXWVE\na58TDWVm3wN2F2y+GXg4ev4w8Et1TVSLKpGXEM5RVyEze9XMVkbP3wLWANOo8Pxs5cIhyaBAVxkD\nvi1puaQPNjoxbWKSmW2H8KMFJjU4Pa3uQ9Gcc//g1XvVkTQTuAT4PnB6JednKwcMl75rzOxdwM8T\nfpjXNjpBbch7mVTv08BZZnYJ8CpFutm7wUkaA3wN+G/RnUbh+Tjo+dnKAWML0J33ehoD81C5KpjZ\ntujfHcC/EKr9XG22SzodQNIZwGsNTk/LMrMdefPJfRa4vJHpaTWShhKCxT+a2TeizRWdn60cMI4N\nCoymPb8VeLTBaWpZkkZFVx9IGk2Y/PHFxqaqJYnj69kfBe6Inn8A+EbhB1xJx+VlVKDFFuLnZ6U+\nD/zEzP4qb1tF52dLj8OIutX9FQODAh9ocJJalqRZhLsKIwzo/LLnZ2Uk/R/CNPwTgO3A/cC/Av8M\nTCfMavArZranUWlsFSXycj6h7r0f2AD8Tlz/7gYn6RpgGfBjwm/cgD8AngO+SsLzs6UDhnPOufpp\n5Sop55xzdeQBwznnXCIeMJxzziXiAcM551wiHjCcc84l4gHDOedcIh4wnCsiGhD642b/TufqyQOG\nc6VlMUjJBz65luUBw7kyJJ0ZLdhzWcH2f5L0vrzXX5C0MLqTWCbph9HjyiLf+QFJn8p7/W+S5kXP\n3yvpmeizX5E0Ksu/z7mkPGA4NwhJswkTtt1uZs8XvP0V4Fej/YYBNwD/TpjK4j3RzL+3Ap+iuBPu\nNiRNAO4D3h19/nngf6bwpzhXs6GNToBzTWwSYS6ohWa2tsj73wQ+GQWL9wHLzOxtSWOBhyRdAhwF\nzqngmFcC5wNPSxIwDHi2lj/CubR4wHCutDeAjcB1wAkBIwoOvcACwp3GP0Vv/T7wqpldJGkIcKDI\ndx/h+Dv8k6J/BTxuZr+Wyl/gXIq8Ssq50t4G/iNwu6TbSuzzVeA3gGuBeI3pU4Bt0fPbgSF5+8fT\ndW8ALlEwnYG1R74PXCPpLDg27XwldyjOZcYDhnODMLMDwE3Af5d0U5FdHgfmAd8xsyPRtk8Dd0ha\nAcwG9uV/ZfS9TxOCxmrgk4S2CsxsJ2F9gn+StAp4BpiT7l/lXHV8enPnnHOJ+B2Gc865RDxgOOec\nS8QDhnPOuUQ8YDjnnEvEA4ZzzrlEPGA455xLxAOGc865RDxgOOecS+T/A4634oZ/CW4QAAAAAElF\nTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x7fc7cb726ac8>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"k_range = range(1, 21)\n",
|
|
"accuracy = []\n",
|
|
"for k in k_range:\n",
|
|
" m = KNeighborsClassifier(k)\n",
|
|
" m.fit(x_train, y_train)\n",
|
|
" y_test_pred = m.predict(x_test)\n",
|
|
" accuracy.append(metrics.accuracy_score(y_test, y_test_pred))\n",
|
|
"plt.plot(k_range, accuracy)\n",
|
|
"plt.xlabel('k value')\n",
|
|
"plt.ylabel('Accuracy')\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The result is very dependent of the input data. Execute again the train_test_split and test again how the result changes with k."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## References"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"* [KNeighborsClassifier API scikit-learn](http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html)\n",
|
|
"* [Learning scikit-learn: Machine Learning in Python](http://proquest.safaribooksonline.com/book/programming/python/9781783281930/1dot-machine-learning-a-gentle-introduction/ch01s02_html), Raúl Garreta; Guillermo Moncecchi, Packt Publishing, 2013.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Licence\n",
|
|
"The notebook is freely licensed under under the [Creative Commons Attribution Share-Alike license](https://creativecommons.org/licenses/by/2.0/). \n",
|
|
"\n",
|
|
"© Carlos A. Iglesias, Universidad Politécnica de Madrid."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.5.6"
|
|
},
|
|
"latex_envs": {
|
|
"LaTeX_envs_menu_present": true,
|
|
"autocomplete": true,
|
|
"bibliofile": "biblio.bib",
|
|
"cite_by": "apalike",
|
|
"current_citInitial": 1,
|
|
"eqLabelWithNumbers": true,
|
|
"eqNumInitial": 1,
|
|
"hotkeys": {
|
|
"equation": "Ctrl-E",
|
|
"itemize": "Ctrl-I"
|
|
},
|
|
"labels_anchors": false,
|
|
"latex_user_defs": false,
|
|
"report_style_numbering": false,
|
|
"user_envs_cfg": false
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 1
|
|
}
|