{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Visualization - Plots\n",
    "\n",
    "In this notebook we will work through a variety of commonly used plots and implement them using matlibplot. Feel free to modify the code and re-run to see the results.\n",
    "\n",
    "The first thing we need to do is load some data sets that we will use in our plotting demos. We will make of the *iris* dataset (loaded from Scikit-learn) and some real-world data collected using an inertial measurement unit (IMU) from a livestock monitoring experiment. The IMU data includes timeseries data for an 3-axis accelerometer and gyroscope sensor sampled at 20Hz that was mounted on a collar warn by a Merino sheep. The sample provided in imu_dataset.csv is a small segment that was captured when the animal was ruminating (essentially chewing). This produces a distinctive cyclic pattern in the timeseries data.\n",
    "\n",
    "To start we will import the python modules that we need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from matplotlib.colors import ListedColormap\n",
    "from sklearn import datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the Data\n",
    "\n",
    "Next we will import the iris dataset and extract the items we need for the examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = datasets.load_iris()\n",
    "X = iris.data\n",
    "tgt = iris.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can import our IMU data. We will start by creating some empty numpy arrays to store the data. We will create an array for the accelerometer sensor and a second array for the gyroscope data. Each array will have three columns (one for each axis of the sensor). We will follow the standard patter for importing data - open the file, read the data line by line using the cdv functions.\n",
    "\n",
    "We will iteratively add the rows of data to the ax_set and gx_set arrays using the `append` function provided by the numpy array object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create empty 3D np.arrays to store the accelerometer and gyroscope data \n",
    "ax_set = np.empty((0,3), float)\n",
    "gx_set = np.empty((0,3), float)\n",
    "\n",
    "# Read the data line-by-line into the 3D arrays\n",
    "with open('imu_dataset.csv', newline='') as f:\n",
    "    reader = csv.reader(f)\n",
    "    for row in reader:\n",
    "        #We are only interested in the accelerometer data\n",
    "        temp_Arr = np.array([float(row[1]), float(row[2]),float(row[3]),\n",
    "                             float(row[4]), float(row[5]),float(row[6])])\n",
    "        ax_set = np.append(ax_set,[temp_Arr[0:3]], axis = 0);\n",
    "        gx_set = np.append(gx_set, [temp_Arr[3:6]], axis = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Line Plot\n",
    "\n",
    "Now we can plot our data using a simple line plot. We will only plot the slice of data that lies in the range `segment_to_view_start` through to `segment_to_view_end`. This is placed in the index for the 'row' dimension. columns 0,1 and 2 correspond to the data x, y and z axis of the sensors.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Angle')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 526952 characters]\n",
      "text/plain": [
       "<Figure size 2160x1440 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# The segment that we want to analyse - the whole segment contains 5000 samples (250 seconds)\n",
    "# Try changing this to 1000, 1850 to view a single 'cycle'\n",
    "segment_to_view_start = 1000\n",
    "segment_to_view_end = 1850\n",
    "\n",
    "# Plot the acceleration data\n",
    "fig, ax = plt.subplots(nrows=2,figsize=(30,20))\n",
    "ax[0].set_title('Accelerometer Data')      \n",
    "ax[0].plot(ax_set[segment_to_view_start:segment_to_view_end,0], 'r-')\n",
    "ax[0].plot(ax_set[segment_to_view_start:segment_to_view_end,1], 'g-')\n",
    "ax[0].plot(ax_set[segment_to_view_start:segment_to_view_end,2], 'b-')\n",
    "ax[0].set_xlabel('Time')\n",
    "ax[0].set_ylabel('Acceleration')\n",
    "\n",
    "# Plot the gyroscope data\n",
    "ax[1].set_title('Gyroscope Data')\n",
    "ax[1].plot(gx_set[segment_to_view_start:segment_to_view_end,0], 'r-')\n",
    "ax[1].plot(gx_set[segment_to_view_start:segment_to_view_end,1], 'g-')\n",
    "ax[1].plot(gx_set[segment_to_view_start:segment_to_view_end,2], 'b-')\n",
    "ax[1].set_xlabel('Time')\n",
    "ax[1].set_ylabel('Angle')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scatter Plots\n",
    "The next part of this demostration will involve creating a series of scatter plots for the iris dataset. We will create a colour map that will be used to set the colours of the points based on their species."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x1f1b29660c8>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 25204 characters]\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cmap_bold = ListedColormap(['red', 'green', 'blue'])\n",
    "fig, ax = plt.subplots(figsize=(5, 5))\n",
    "ax.scatter(X[:, 0], X[:, 1], c=tgt, cmap=cmap_bold,\n",
    "                              edgecolor='k', s=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next example we use the 'scatter' function to draw a scatter plot for each combination of the 4 columns in the iris dataset. We use two 'nested' loops to draw a scatter plot for each combination of the columns, creating 4x4 grid of scatter plots. This is an effective way to review multi-dimensional data.\n",
    "\n",
    "This example makes use of the `range( ... )` function to create the lists that we iterate through to generate the plots. We make use of the `suptitle( ... )` function to place a title at the top of the figure (i.e. above the axes within the subplots).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 217188 characters]\n",
      "text/plain": [
       "<Figure size 864x864 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cmap_bold = ListedColormap(['red', 'green', 'blue'])\n",
    "\n",
    "fig, axs = plt.subplots(4, 4, figsize=(12, 12))\n",
    "\n",
    "for x in range(0,4):\n",
    "    for y in range(0,4):\n",
    "        if x == 0:\n",
    "            axs[y, x].set_ylabel(iris.feature_names[y])\n",
    "        if y == 3:\n",
    "            axs[y, x].set_xlabel(iris.feature_names[x])\n",
    "        if x != y:\n",
    "            axs[y, x].scatter(X[:, x], X[:, y], c=tgt, cmap=cmap_bold,\n",
    "                              edgecolor='k', s=20)\n",
    "fig.suptitle('Iris Data (red=setosa, green=versicolor,blue=virginica',fontsize=24)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bar Charts\n",
    "\n",
    "This is a really quick demo of the `barh` function to draw an horizontal bar chart. In this example, we create a mock categorical dataset for the demonstration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 9064 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Fixing random state for reproducibility\n",
    "np.random.seed(19680801)\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "# Example mock data for the demo\n",
    "people = ('Tom', 'Dick', 'Harry', 'Slim', 'Jim')\n",
    "y_pos = np.arange(len(people))\n",
    "performance = 3 + 10 * np.random.rand(len(people))\n",
    "error = np.random.rand(len(people))\n",
    "\n",
    "ax.barh(y_pos, performance, xerr=error, align='center')\n",
    "ax.set_yticks(y_pos)\n",
    "ax.set_yticklabels(people)\n",
    "ax.invert_yaxis()  # labels read top-to-bottom\n",
    "ax.set_xlabel('Performance')\n",
    "ax.set_title('How fast do you want to go today?')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Histogram\n",
    "\n",
    "In this demonstration we will create a simple histograms of our accelerometer data imported above. In this first example, we will plot a histogram of the x-axis from the accelerometer data (i.e. the data in column 0). We will divide our data into  100 bins for the plow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 6320 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# the histogram of the data\n",
    "fig, ax = plt.subplots()\n",
    "nbins = 100\n",
    "n, bins, patches = ax.hist(ax_set[:,1], nbins)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this next example, we can create a multi-histogram displaying the distributions of the x,y and z axes respectively (columns 0, 1 and 2)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 6424 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# the histogram of the data\n",
    "fig, ax = plt.subplots()\n",
    "n, bins, patches = ax.hist(ax_set, nbins)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Box Plots\n",
    "\n",
    "In this example we will create a simple box plot for the accelerometer data plotted in the histograms in the previous example. Recall that box plots provide a summary of the data's distribution by ploting the min, max, median, UQ and LQ along with points identified as outliers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Multi Box Plot')"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 16916 characters]\n",
      "text/plain": [
       "<Figure size 288x864 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axs = plt.subplots(nrows=2, figsize=(4, 12))\n",
    "\n",
    "# basic plot\n",
    "axs[0].boxplot(ax_set[:,0])\n",
    "axs[0].set_title('Basic Plot')\n",
    "\n",
    "# Plot multiple series - in this examplewe turn the outlier plot off by setting sym=''\n",
    "axs[1].boxplot(ax_set,sym='')\n",
    "axs[1].set_title('Multi Box Plot')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2D Histogram (Heat Map)\n",
    "\n",
    "In this example, we will demonstrate the creation of a 2-dimensional histgram of the x and y acceleraometer data. We will first construct a scatter plot and then construct the corresponding 2D histogram to deomonstrate the concept. Notice that it is difficult to view the structure of the data in the scatter plot due to the large number of data points. The 2D histogram provides an effective way to analyse large datasets like this by revealing some of the underlying structure.\n",
    "\n",
    "We use the `colorbar( ... )` function to include the colour bar that corresponds to the counts that fall within each 2D bin.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Ay')"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 106628 characters]\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Scatter\n",
    "fig, ax = plt.subplots(figsize=(8, 8))\n",
    "ax.scatter(ax_set[:,0], ax_set[:,1], edgecolor='k')\n",
    "ax.set_title('Scatter Plot')\n",
    "ax.set_xlabel('Ax')\n",
    "ax.set_ylabel('Ay')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 15636 characters]\n",
      "text/plain": [
       "<Figure size 720x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10, 8))\n",
    "ax.set_title('Ax Vs Ay')\n",
    "hs = ax.hist2d(ax_set[:,0], ax_set[:,1], bins=40, cmap='jet')\n",
    "cb = fig.colorbar(hs[3], ax=ax)\n",
    "cb.set_label('counts')\n",
    "ax.set_xlabel('Ax')\n",
    "ax.set_ylabel('Ay')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can even use hexagonal bins with the `hexbin( ... )` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Ay')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 112432 characters]\n",
      "text/plain": [
       "<Figure size 720x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10, 8))\n",
    "hb = ax.hexbin(ax_set[:,0], ax_set[:,1], gridsize=40, cmap='jet')\n",
    "ax.set_title(\"Ax vs Ay Hexagon binning\")\n",
    "cb = fig.colorbar(hb, ax=ax)\n",
    "cb.set_label('counts')\n",
    "ax.set_xlabel('Ax')\n",
    "ax.set_ylabel('Ay')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
