{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## COSC102 Topic 1 - Nearest Neighbors Classification Demo\n",
    "\n",
    "Sample usage of Nearest Neighbors classification.\n",
    "It will plot the decision boundaries for each class by using a 'mesh' \n",
    "(a grid of points) on a matplotlib figure.\n",
    "\n",
    "First we will import Python libraries that we need to access the data, create plots and run the nearest-neighbours machine learning algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.colors import ListedColormap\n",
    "from sklearn import neighbors, datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This a parameter for the KNN algorithm - we can play with this to achieve different results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_neighbors = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we import some data to play with - the iris dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = datasets.load_iris()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This specifies the data columns from the iris dataset that we will use. We must select exactly 2 columns for this script to work.\n",
    "\n",
    "* Here, you can alter the columns (0-3) to use.\n",
    "* Try ```features_to_use = [2,3]```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "features_to_use =[0,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This specifies the how the algorithm uses the distances between the points to classify points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "weights = ['uniform', 'distance']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We only take the first two features. We could avoid this ugly slicing by using a two-dim dataset. The measurements sit within the 'data' attribute of the 'iris' object.\n",
    "\n",
    "The species names sit in the 'target' attribute of the 'iris' object. We can change the columns that we are using by updating the indexes Try [:, 2:4:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = iris.data[:, features_to_use]\n",
    "y = iris.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the step size in the 'mesh' that will be used to demonstrate the boundaries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = .02 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create color maps for the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmap_light = ListedColormap(['orange', 'cyan', 'cornflowerblue'])\n",
    "cmap_bold = ListedColormap(['darkorange', 'c', 'darkblue'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create an instance of Neighbours Classifier and fit the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "                     metric_params=None, n_jobs=None, n_neighbors=10, p=2,\n",
       "                     weights='uniform')"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = neighbors.KNeighborsClassifier(n_neighbors, weights=weights[0])\n",
    "clf.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the decision boundary. For that, we will assign a color to each point in the mesh [x_min, x_max] * [y_min, y_max]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n",
    "y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n",
    "xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n",
    "                     np.arange(y_min, y_max, h))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we use our algoithm to classify each point the 'mesh' that will be used to demonstrate the decision boundaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Put the result into a color plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "Z = Z.reshape(xx.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a figure to plot the results.\n",
    "\n",
    "Plot also the training points to see where they sit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 25872 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "plt.pcolormesh(xx, yy, Z, cmap=cmap_light)\n",
    "\n",
    "# Plot also the training points to see where they sit\n",
    "plt.scatter(X[:, 0], X[:, 1], c=y, cmap=cmap_bold,\n",
    "            edgecolor='k', s=20)\n",
    "plt.xlim(xx.min(), xx.max())\n",
    "plt.ylim(yy.min(), yy.max())\n",
    "\n",
    "# Set the title of the plot\n",
    "plt.title(\"3-Class classification (k = %i, weights = '%s')\"\n",
    "          % (n_neighbors, weights[0]))\n",
    "\n",
    "# Set the labels on the plot\n",
    "plt.xlabel(iris.feature_names[features_to_use[0]])\n",
    "plt.ylabel(iris.feature_names[features_to_use[1]])\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
