{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise answers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 4\n",
    "\n",
    "### TRY THIS: VARIABLES AND EXPRESSIONS\n",
    "\n",
    "In the Python shell create some variables. What happens when you try to put spaces, dashes or other non alphanumeric characters in the variable name? Play around with a few complex expressions, e.g., x = 2 + 4 * 5 – 6 / 3. Use parentheses to group the numbers in different ways and see how that changes the result compared to the original ungrouped expression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.14"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> x = 3\n",
    ">>> y = 3.14\n",
    ">>> y\n",
    "3.14"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> x\n",
    "3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-10-45de2e2d3a3b>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-10-45de2e2d3a3b>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    big var = 12\u001b[0m\n\u001b[0m          ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    ">>> big var = 12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'big' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-9-23c727abea60>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mbig\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m: name 'big' is not defined"
     ]
    }
   ],
   "source": [
    ">>> big-var"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'big' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-11-3e24f3328846>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mbig\u001b[0m\u001b[0;34m&\u001b[0m\u001b[0mvar\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m: name 'big' is not defined"
     ]
    }
   ],
   "source": [
    ">>> big&var\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20.0"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> x = 2 + 4 * 5 - 6 /3\n",
    ">>> x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "28.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> x = (2 + 4) * 5 - 6 /3\n",
    ">>> x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-2.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> x = (2 + 4) * (5 - 6) /3\n",
    ">>> x\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS : MANIPULATING STRINGS AND NUMBERS\n",
    "In the Python shell create some string and number variables (integers, floats, and complex numbers). Experiment a bit with what happens when you do operations with them, including across types – for example, can you multiply a string by an integer? By a float or complex number?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> i = 3\n",
    ">>> f = 3.14"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-23-13d2677862a7>, line 3)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-23-13d2677862a7>\"\u001b[0;36m, line \u001b[0;32m3\u001b[0m\n\u001b[0;31m    c = 3j2\u001b[0m\n\u001b[0m          ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    ">>> c = 3j2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-17-b3df3fec8294>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-17-b3df3fec8294>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    c = 3J2\u001b[0m\n\u001b[0m          ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    ">>> c = 3J2\n",
    "  File \"<stdin>\", line 1\n",
    "    c = 3J2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3+2j)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> c = 3 + 2j\n",
    ">>> c\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> s = 'hello'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "can't multiply sequence by non-int of type 'float'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-27-b82dfe1eab71>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0ms\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: can't multiply sequence by non-int of type 'float'"
     ]
    }
   ],
   "source": [
    ">>> s * f"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hellohellohello'"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> s * i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "can't multiply sequence by non-int of type 'complex'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-29-419e81d58692>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0ms\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mc\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: can't multiply sequence by non-int of type 'complex'"
     ]
    }
   ],
   "source": [
    ">>> s * c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9+6j)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> c * i\n",
    "(9+6j)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9.42+6.28j)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> c * f\n",
    "(9.42+6.28j)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Also load the math module and try out a few of the functions, then load the cmath module and do the same. What happens if you try to use one of those functions on an integer or float after loading the cmath module? How might you get the math module functions back?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.0"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> from math import sqrt\n",
    ">>> sqrt(16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4+0j)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> from cmath import sqrt\n",
    ">>> sqrt(16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To recoonect the first `sqrt` to our current namespace, we can re-import it. Note that this does **not** reload the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.0"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> from math import sqrt\n",
    ">>> sqrt(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS : GETTING INPUT\n",
    "Experiment with the input() function to get string and integer input. Using code similar to the code above, what is the effect of not using int() around the call to input()for integer input? Can you modify that code to accept a float, say 28.5? What happens if you deliberately enter the “wrong” type of value? i.e, a float where an int is expected or a string where a number is expected, and vice versa?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "int?3\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'3'"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> x = input(\"int?\")\n",
    "#int?3\n",
    ">>> x\n",
    "'3'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "float?3.5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3.5"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> y = float(input(\"float?\")) \n",
    "#float?3.5\n",
    ">>> y\n",
    "3.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "int?3.5\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "invalid literal for int() with base 10: '3.5'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-41-324f008217ab>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mz\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"int?\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0;31m#int?3.5\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: invalid literal for int() with base 10: '3.5'"
     ]
    }
   ],
   "source": [
    ">>> z = int(input(\"int?\"))\n",
    "#int?3.5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK : PYTHONIC STYLE\n",
    "Which of the following variable and function names do you think are not good Pythonic style? Why? \n",
    "\n",
    "`bar(, varName, VERYLONGVARNAME, foobar, longvarname, foo_bar(), really_very_long_var_name`\n",
    "\n",
    "* `bar(` - not good, not legal, includes symbol\n",
    "* `varName` – not good, mixed case\n",
    "* `VERYLONGVARNAME` – not good, long, all caps, hard to read,\n",
    "* `foobar` - good\n",
    "* `longvarname` – good, although underscores to separate words would be better\n",
    "* `foo_bar()` - good\n",
    "* `really_very_long_var_name` – long, but good if all of the words are needed, say to distinguish between similar variables.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: LEN()\n",
    "\n",
    "What would len() return for each of the following: [0]; []; [[1, 3, [4, 5], 6], 7 s]?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* `len([0])` - 1; \n",
    "* `len([])` - 0; \n",
    "* `len([[1, 3, [4, 5], 6], 7 s])` - 2 ([1, 3, [4, 5], 6] is a list and a single item in the list before the second item, 7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: LIST SLICES AND INDEXES\n",
    "Using what you know about the len() function and list slices, how would you combine the two to get the second half of a list when you don’t know what size it is. Experiment in the Python shell to confirm that your solution works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 5, 6]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> my_list = [1, 2, 3, 4, 5, 6]\n",
    ">>> last_half = my_list[len(my_list)//2:]\n",
    ">>> last_half"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`len(my_list) // 2` is the half way point, slice from there to the end."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: MODIFYING LISTS\n",
    "Suppose you have a list 10 items long. How might you move last 3 items from the end of the list to the beginning, keeping them in the same order?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 5, 6, 1, 2, 3]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> my_list = my_list[-3:] + my_list[:-3]\n",
    ">>> my_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: SORTING LISTS\n",
    "\n",
    "Suppose you have a list where each element is in turn a list: [[1, 2, 3], [2, 1, 3], [4, 0, 1]]. If you wanted to sort this list by the second element in each list, so that the result would be [[4, 0, 1], [2, 1, 3], [1, 2, 3]], what function would you write to pass as the key value to the sort() method?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[4, 0, 1], [2, 1, 3], [1, 2, 3]]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "the_list =  [[1, 2, 3], [2, 1, 3], [4, 0, 1]]\n",
    "the_list.sort(key=lambda x: x[1])\n",
    "the_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[4, 0, 1], [2, 1, 3], [1, 2, 3]]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "the_list =  [[1, 2, 3], [2, 1, 3], [4, 0, 1]]\n",
    "def key_func(x):\n",
    "    return x[1] \n",
    "the_list.sort(key=key_func)\n",
    "the_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: LIST OPERATIONS\n",
    "What would be the result of len([[1,2]] * 3)? \n",
    "```\n",
    "3\n",
    "```\n",
    "\n",
    "What are two differences between using the in operator and a list’s index() method?\n",
    "\n",
    "* index gives position, in gives true/false answer\n",
    "* index gives error if element is not in list\n",
    "\n",
    "Which of the following with raise an exception? min([\"a\", \"b”, \"c\"]); max([1, 2, \"three\"]); [1, 2, 3].count(\"one\")\n",
    "\n",
    "max([1, 2, \"three\"]) - strings and int's can't be comparied, so it's impossible to get a max value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: LIST OPERATIONS\n",
    "If you have a list x write the code to safely remove an item if and only if that value is in the list.\n",
    "````\n",
    "if element in x:\n",
    "    x.remove(element)\n",
    "```\n",
    "Modify that code to remove the element only in the item occurs in the list more than once.\n",
    "```\n",
    "if x.count(element) > 1:\n",
    "    x.remove(element)\n",
    "```\n",
    "Note: this will only remove the first occurrence of element."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: LIST COPIES\n",
    "Suppose you have the following list – x = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] What code could you use to get a copy y of that list where you could change its elements without the side effect of changing the contents of x?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1, 2, 3], [4, 5, 6], [7, 8, 9]]\n",
      "[[1, 10, 3], [4, 5, 6], [7, 8, 9]]\n",
      "[[1, 10, 3], [4, 5, 6], [7, 8, 9]]\n",
      "[[1, 2, 3], [4, 5, 6], [7, 8, 9]]\n",
      "[[1, 10, 3], [4, 5, 6], [7, 8, 9]]\n"
     ]
    }
   ],
   "source": [
    "import copy\n",
    "x = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]\n",
    "y = x[:]\n",
    "\n",
    "y[0][1] = 10\n",
    "print(x)\n",
    "print(y)\n",
    "x = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]\n",
    "copy_x = copy.deepcopy(x)\n",
    "copy_x[0][1] = 10\n",
    "print(x)\n",
    "print(copy_x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: TUPLES\n",
    "Explain why the following operations are not legal for the tuple x = (1, 2, 3, 4)\n",
    "```\n",
    "x.append(1)\n",
    "x[1] = \"hello\"\n",
    "del x[2]\n",
    "```\n",
    "All of the above operations change the object in place and tuples can't be changed.\n",
    "\n",
    "If you had a tuple x = (3, 1, 4, 2) how might you end up with a sorted version of x?\n",
    "```\n",
    "x = sorted(x)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3, 4]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = (3, 1, 4, 2)\n",
    "x = sorted(x)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: SETS\n",
    "If you were to construct a set from the following list, how many elements would it have? [1, 2, 5, 1, 0, 2, 3, 1, 1, (1, 2, 3)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(set([1, 2, 5, 1, 0, 2, 3, 1, 1, (1, 2, 3)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB 5: EXAMINING A LIST\n",
    "In this lab the task is to read a set of temperature data (in fact the monthly high temperatures at Heathrow airport for 1948-2016) from a file and then find some basic information – the highest and lowest temperatures, the mean (average) temperature, and the median temperature (the temperature in the middle if all of the temperatures are sorted). \n",
    "\n",
    "The temperature data is in the file `lab_05.txt` in the source code directory for this chapter. Since we have not yet discussed reading files, the code to read the files into a list is provided below:\n",
    "````\n",
    "with open('lab_05.txt') as infile:\n",
    "     for row in infile:\n",
    "        temperatures.append(float(row.strip())\n",
    "```\n",
    "As mentioned above you should find the highest and lowest temperature, the average, and the median. You will probably want to use the `min()`, `max()`, `sum()`, `len()`, and `sort()` functions/methods. \n",
    "\n",
    "**Bonus:** Determine how many unique temperatures are in the list. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "temperatures = []\n",
    "with open('lab_05.txt') as infile:\n",
    "     for row in infile:\n",
    "        temperatures.append(float(row.strip()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "max = 28.2\n",
      "min = 0.8\n",
      "mean = 14.848309178743966\n",
      "median = 14.7\n"
     ]
    }
   ],
   "source": [
    "max_temp = max(temperatures)\n",
    "min_temp = min(temperatures)\n",
    "mean_temp = sum(temperatures)/len(temperatures)\n",
    "# we'll need to sort to get the median temp\n",
    "temperatures.sort()\n",
    "median_temp = temperatures[len(temperatures)//2]\n",
    "print(\"max = {}\".format(max_temp))\n",
    "print(\"min = {}\".format(min_temp))\n",
    "print(\"mean = {}\".format(mean_temp))\n",
    "print(\"median = {}\".format(median_temp))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of temps - 828\n",
      "number of unique temps - 217\n"
     ]
    }
   ],
   "source": [
    "unique_temps = len(set(temperatures))\n",
    "\n",
    "print(\"number of temps - {}\".format(len(temperatures)))\n",
    "print(\"number of unique temps - {}\".format(unique_temps))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: SPLIT AND JOIN\n",
    "How could you use split and join to change all of the whitespace in string x to dashes? E.g., \"this is a test\" to \"this-is-a-test\".\n",
    "```\n",
    ">>> x = \"this is a test\" \n",
    ">>> \"-\".join(x.split())\n",
    "'this-is-a-test'\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: STRINGS TO NUMBERS\n",
    "Which of the following will not be converted to numbers and why?\n",
    "1.\tint('a1')\n",
    "2.\tint('12G', 16)\n",
    "3.\tfloat(\"12345678901234567890\")\n",
    "4.\tint(\"12*2\")\n",
    "\n",
    "Only #3 will convert - all of the others have a character that would not be allowed for conversion to an int."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "invalid literal for int() with base 10: '12*2'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-93-f1de953f5e64>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"12*2\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m: invalid literal for int() with base 10: '12*2'"
     ]
    }
   ],
   "source": [
    "int(\"12*2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: STRIP\n",
    "If the string x equals \"(name, date),\\n\" which of the following would return a string containing \"name, date\"?\n",
    "1.\tx.rstrip(\"),\")\n",
    "2.\tx.strip(\"),\\n\")\n",
    "3.\tx.strip(\"\\n)(,\")\n",
    "\n",
    "#3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: STRING SEARCHING\n",
    "If you wanted to check to see if a line ended with the string \"rejected\" what string method would you use? Would there be any other ways you could get the same result?\n",
    "\n",
    "`endswith('rejected')` \n",
    "\n",
    "You could also do `line[:-8] == rejected` but that would not be as clear or Pythonic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if line.endswith('rejected')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: MODIFYING STRINGS\n",
    "What would be a quick way change all punctuationin a string to spaces?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'This is text  with  punctuation  Right '"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    ">>> punct = str.maketrans(\"!.,:;-?\", \"       \")\n",
    ">>> x = \"This is text, with: punctuation! Right?\"\n",
    ">>> x.translate(punct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: STRING OPERATIONS\n",
    "Suppose you have a list of strings where some (but not necessarily all) of the strings begin and end with the double quote character:\n",
    "```\n",
    "x = ['\"abc\"', 'def', '\"ghi\"', '\"klm\"', 'nop']\n",
    "```\n",
    "What code would you use on each element to remove just the double quotes?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "abc\n",
      "def\n",
      "ghi\n",
      "klm\n",
      "nop\n"
     ]
    }
   ],
   "source": [
    ">>> x = ['\"abc\"', 'def', '\"ghi\"', '\"klm\"', 'nop']\n",
    ">>> for item in x:\n",
    "...     print(item.strip('\"'))\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What code could you use to find the position of the last “p” in “Mississippi”? Once you found its position, what code would you use to remove just that letter?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mississipi\n"
     ]
    }
   ],
   "source": [
    ">>> state = \"Mississippi\"\n",
    ">>> pos = state.rfind(\"p\")\n",
    "\n",
    ">>> state = state[:pos] + state[pos+1:]\n",
    ">>> print(state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: THE FORMAT() METHOD\n",
    "What will be in x when the following snippets of code are executed?\n",
    "```\n",
    "x = \"{1:{0}}\".format(3, 4)\n",
    "'  4'\n",
    "x = \"{0:$>5}\".format(3)\n",
    "'$$$$3'\n",
    "x = \"{a:{b}}\".format(a=1, b=5)\n",
    "'    1'\n",
    "x = \"{a:{b}}:{0:$>5}\".format(3, 4, a=1, b=5, c=10)\n",
    "'    1:$$$$3'\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: FORMATTING STRINGS WITH %\n",
    "What would be in the variable x after the following snippets of code have executed?\n",
    "```\n",
    "x = \"%.2f\" % 1.1111\n",
    "x will contain '1.11'\n",
    "x = \"%(a).2f\" % {'a':1.1111}\n",
    "x will contain '1.11'\n",
    "x = \"%(a).08f\" % {'a':1.1111}\n",
    "x will contain '1.11110000'\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: BYTES\n",
    "For which of the following kinds of data would you want to use a string? For which could you use bytes? \n",
    "\n",
    "1) Data file storing binary data; \n",
    "\n",
    "   Since the data is binary, we will be more concerned with the contents as numbers rather than text. Therefore it would make sense to use bytes.\n",
    "   \n",
    "2) Text in a language with accented characters; \n",
    "\n",
    "   String – Python 3 strings are Unicode so can handle accented characters.\n",
    "   \n",
    "3) Text with only upper and lower case roman characters; \n",
    "\n",
    "   String – strings should be used for all text in Python 3.\n",
    "   \n",
    "4) A series of integers no larger than 255. \n",
    "\n",
    "   Bytes – a byte is an integer no larger than 255, so the bytes type is perfect for storing integers like this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB 6: PREPROCESSING TEXT\n",
    "In processing raw text it’s quite often necessary to clean and normalize the text before doing anything else. If we want to find the frequency of words in a text for example, we can make the job easier if before we start counting we make sure that everything is lower case (or upper case, if you prefer) and that all punctuation has been removed. It can also make things easier if the text is broken into a series of words. \n",
    "\n",
    "In this lab the task is to read an excerpt of the first chapter of Moby Dick, make sure that everything is one case, remove all punctuation, and write the words one per line to a second file. Again, since we haven’t yet covered reading and writing files, the code for those operations is supplied below. \n",
    "```\n",
    "with open(\"moby_01.txt\") as infile, open(\"moby_01_clean.txt\", \"w\") as outfile:\n",
    "    for line in infile:\n",
    "        # make all one case\n",
    "        # remove punctuation\n",
    "        # split into words\n",
    "        # write all words for line\n",
    "        outfile.write(cleaned_words)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "punct = str.maketrans(\"\",  \"\", \"!.,:;-?\")\n",
    "\n",
    "with open(\"moby_01.txt\") as infile, open(\"moby_01_clean.txt\", \"w\") as outfile:\n",
    "    for line in infile:\n",
    "        # make all one case\n",
    "        cleaned_line = line.lower()\n",
    "        \n",
    "        # remove punctuation\n",
    "        cleaned_line = cleaned_line.translate(punct)\n",
    "        \n",
    "        # split into words\n",
    "        words = cleaned_line.split()\n",
    "        cleaned_words = \"\\n\".join(words) + \"\\n\"\n",
    "        # write all words for line\n",
    "        outfile.write(cleaned_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: CREATE A DICTIONARY\n",
    "Write the code to ask the user for 3 names and 3 ages. After they are entered ask the user for one on the names and print the correct age. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name? Tom\n",
      "Age? 33\n",
      "Name? Talita\n",
      "Age? 28\n",
      "Name? Rania\n",
      "Age? 35\n",
      "Name to find? Talita\n",
      "28\n"
     ]
    }
   ],
   "source": [
    ">>> name_age = {}\n",
    ">>> for i in range(3):\n",
    "...     name = input(\"Name? \")\n",
    "...     age = int(input(\"Age? \"))\n",
    "...     name_age[name] = age\n",
    "\n",
    ">>> name_choice = input(\"Name to find? \")\n",
    ">>> print(name_age[name_choice])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: DICTIONARY OPERATIONS\n",
    "Assume that we have a dictionary `x = {'a':1, 'b':2, 'c':3, 'd':4}` and a dictionary `y = {'a':6, 'e':5, 'f':6}`. What would be the contents of x after the following snippets of code have executed. \n",
    "```\n",
    "del x['d']\n",
    "z = x.setdefault('g', 7)\n",
    "x.update(y)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'b': 2, 'c': 3, 'a': 1}\n",
      "{'b': 2, 'c': 3, 'a': 1, 'g': 7}\n",
      "{'b': 2, 'c': 3, 'a': 6, 'e': 5, 'g': 7, 'f': 6}\n"
     ]
    }
   ],
   "source": [
    "x = {'a':1, 'b':2, 'c':3, 'd':4}\n",
    "y = {'a':6, 'e':5, 'f':6}\n",
    "del x['d']\n",
    "print(x)\n",
    "z = x.setdefault('g', 7)\n",
    "print(x)\n",
    "x.update(y)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: WHAT CAN BE A KEY?\n",
    "Decide which of the following expressions can be a dictionary key: \n",
    "```\n",
    "1 - Yes.\n",
    "'bob' - Yes.\n",
    "('tom', [1, 2, 3]) - No, contains a list, which is not hashable.\n",
    "[\"filename\"] - No, it's a list, which is not hashable.\n",
    "\"filename\" - Yes.\n",
    "(\"filename\",  \"extension\") - Yes, it's a tuple.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: USING DICTIONARIES\n",
    "Suppose you were writing a program that worked like a spreadsheet. How might you use a dictionary to store the contents of a sheet? Write some sample code to both store a value and retrieve a value in a particular cell. What might be some drawbacks to this approach?\n",
    "\n",
    "You could use tuples of row, column values as keys to store the values in a dictionary. One drawback would be that the keys would not be sorted, so you would have to manage that as you grabbed the keys/values to render as a spreadhsheet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100\n"
     ]
    }
   ],
   "source": [
    ">>> sheet = {}\n",
    ">>> sheet[('A', 1)] = 100\n",
    ">>> sheet[('B', 1)] = 1000\n",
    "\n",
    ">>> print(sheet[('A', 1)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB 7: WORD COUNTING\n",
    "In the last lab we took the text of the first chapter of Moby Dick, normalized the case, removed punctuation, and wrote the separated words to a file. In this lab, we’ll read that file and use a dictionary to count the number of times each word occurs and then report the most common and least common words.\n",
    "\n",
    "Use this code to read the words from the file into a list called `moby_words`:\n",
    "```moby_words = []\n",
    "with open('moby_01_clean.txt') as infile:\n",
    "    for word in infile:\n",
    "        moby_words.append(word.strip())\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Most common words:\n",
      "('the', 14)\n",
      "('and', 9)\n",
      "('i', 9)\n",
      "('of', 8)\n",
      "('is', 7)\n",
      "\n",
      "Least common words:\n",
      "('see', 1)\n",
      "('growing', 1)\n",
      "('soul', 1)\n",
      "('having', 1)\n",
      "('regulating', 1)\n"
     ]
    }
   ],
   "source": [
    "moby_words = []\n",
    "with open('moby_01_clean.txt') as infile:\n",
    "    for word in infile:\n",
    "        if word.strip():\n",
    "            moby_words.append(word.strip())\n",
    "        \n",
    "word_count = {}\n",
    "for word in moby_words:\n",
    "    count = word_count.setdefault(word, 0)\n",
    "    count += 1\n",
    "    word_count[word] += 1\n",
    "    \n",
    "word_list = list(word_count.items())\n",
    "word_list.sort(key=lambda x: x[1])\n",
    "print(\"Most common words:\")\n",
    "for word in reversed(word_list[-5:]):\n",
    "    print(word)\n",
    "print(\"\\nLeast common words:\")\n",
    "for word in word_list[:5]:\n",
    "    print(word)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'': 1,\n",
       " 'a': 6,\n",
       " 'about': 2,\n",
       " 'account': 1,\n",
       " 'agonever': 1,\n",
       " 'all': 1,\n",
       " 'almost': 1,\n",
       " 'an': 1,\n",
       " 'and': 9,\n",
       " 'as': 3,\n",
       " 'at': 1,\n",
       " 'ball': 1,\n",
       " 'battery': 1,\n",
       " 'before': 1,\n",
       " 'belted': 1,\n",
       " 'breezes': 1,\n",
       " 'bringing': 1,\n",
       " 'but': 1,\n",
       " 'by': 4,\n",
       " 'call': 1,\n",
       " 'can': 1,\n",
       " 'cato': 1,\n",
       " 'cherish': 1,\n",
       " 'circulation': 1,\n",
       " 'city': 1,\n",
       " 'coffin': 1,\n",
       " 'cooled': 1,\n",
       " 'coral': 1,\n",
       " 'crowds': 1,\n",
       " 'damp': 1,\n",
       " 'degree': 1,\n",
       " 'deliberately': 1,\n",
       " 'downtown': 1,\n",
       " 'driving': 1,\n",
       " 'drizzly': 1,\n",
       " 'especially': 1,\n",
       " 'every': 1,\n",
       " 'extreme': 1,\n",
       " 'feelings': 1,\n",
       " 'few': 1,\n",
       " 'find': 2,\n",
       " 'flourish': 1,\n",
       " 'for': 1,\n",
       " 'from': 1,\n",
       " 'funeral': 1,\n",
       " 'get': 2,\n",
       " 'grim': 1,\n",
       " 'growing': 1,\n",
       " 'hand': 1,\n",
       " 'hats': 1,\n",
       " 'have': 1,\n",
       " 'having': 1,\n",
       " 'her': 1,\n",
       " 'high': 1,\n",
       " 'himself': 1,\n",
       " 'his': 1,\n",
       " 'hours': 1,\n",
       " 'how': 1,\n",
       " 'hypos': 1,\n",
       " 'i': 9,\n",
       " 'if': 1,\n",
       " 'in': 4,\n",
       " 'indian': 1,\n",
       " 'insular': 1,\n",
       " 'interest': 1,\n",
       " 'into': 1,\n",
       " 'involuntarily': 1,\n",
       " 'is': 7,\n",
       " 'ishmael': 1,\n",
       " 'isles': 1,\n",
       " 'it': 6,\n",
       " 'its': 1,\n",
       " 'knew': 1,\n",
       " 'knocking': 1,\n",
       " 'land': 1,\n",
       " 'left': 1,\n",
       " 'little': 2,\n",
       " 'long': 1,\n",
       " 'look': 1,\n",
       " 'manhattoes': 1,\n",
       " 'me': 5,\n",
       " 'meet': 1,\n",
       " 'men': 1,\n",
       " 'methodically': 1,\n",
       " 'mind': 1,\n",
       " 'mole': 1,\n",
       " 'money': 1,\n",
       " 'moral': 1,\n",
       " 'mouth': 1,\n",
       " 'my': 4,\n",
       " 'myself': 2,\n",
       " 'nearly': 1,\n",
       " 'no': 1,\n",
       " 'noble': 1,\n",
       " 'nothing': 2,\n",
       " 'november': 1,\n",
       " 'now': 1,\n",
       " 'ocean': 1,\n",
       " 'of': 8,\n",
       " 'off': 1,\n",
       " 'offthen': 1,\n",
       " 'on': 1,\n",
       " 'or': 2,\n",
       " 'other': 1,\n",
       " 'out': 1,\n",
       " 'part': 1,\n",
       " 'particular': 1,\n",
       " 'pausing': 1,\n",
       " \"people's\": 1,\n",
       " 'philosophical': 1,\n",
       " 'pistol': 1,\n",
       " 'precisely': 1,\n",
       " 'prevent': 1,\n",
       " 'previous': 1,\n",
       " 'principle': 1,\n",
       " 'purse': 1,\n",
       " 'quietly': 1,\n",
       " 'rear': 1,\n",
       " 'reefscommerce': 1,\n",
       " 'regulating': 1,\n",
       " 'requires': 1,\n",
       " 'right': 1,\n",
       " 'round': 1,\n",
       " 'sail': 1,\n",
       " 'same': 1,\n",
       " 'sea': 1,\n",
       " 'see': 1,\n",
       " 'ship': 1,\n",
       " 'shore': 1,\n",
       " 'sight': 1,\n",
       " 'some': 2,\n",
       " 'soon': 1,\n",
       " 'soul': 1,\n",
       " 'spleen': 1,\n",
       " 'stepping': 1,\n",
       " 'street': 1,\n",
       " 'streets': 1,\n",
       " 'strong': 1,\n",
       " 'substitute': 1,\n",
       " 'such': 1,\n",
       " 'surf': 1,\n",
       " 'surprising': 1,\n",
       " 'surrounds': 1,\n",
       " 'sword': 1,\n",
       " 'take': 2,\n",
       " 'that': 2,\n",
       " 'the': 14,\n",
       " 'their': 1,\n",
       " 'there': 3,\n",
       " 'they': 1,\n",
       " 'this': 2,\n",
       " 'thought': 1,\n",
       " 'throws': 1,\n",
       " 'time': 2,\n",
       " 'to': 5,\n",
       " 'towards': 1,\n",
       " 'up': 1,\n",
       " 'upon': 1,\n",
       " 'upper': 1,\n",
       " 'very': 1,\n",
       " 'warehouses': 1,\n",
       " 'washed': 1,\n",
       " 'watergazers': 1,\n",
       " 'waterward': 1,\n",
       " 'watery': 1,\n",
       " 'waves': 1,\n",
       " 'way': 1,\n",
       " 'were': 1,\n",
       " 'wharves': 1,\n",
       " 'whenever': 4,\n",
       " 'where': 1,\n",
       " 'which': 1,\n",
       " 'with': 3,\n",
       " 'world': 1,\n",
       " 'would': 1,\n",
       " 'years': 1,\n",
       " 'you': 1,\n",
       " 'your': 1}"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 8"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: LOOPING AND IF STATEMENTS\n",
    "Suppose you have a list `x = [1, 3, 5, 0, -1, 3, -2]` and you need to remove all negative numbers from that list. Write the code to do this.\n",
    "\n",
    "How would you count the total number of negative numbers in a list `y = [[1, -1, 0], [2, 5, -9], [-2, -3, 0]]`? \n",
    "\n",
    "What code would use to print “very low” if the value of x is below -5, “low” if it’s from -4 up to 0, “neutral” if it’s equal to zero, “high” if it’s greater than 0 up to 5, and “very high” if it’s greather than 5? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 3, 5, 0, 3]\n"
     ]
    }
   ],
   "source": [
    "x = [1, 3, 5, 0, -1, 3, -2]\n",
    "for i in x:\n",
    "    if i < 0:\n",
    "        x.remove(i)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n"
     ]
    }
   ],
   "source": [
    "count = 0\n",
    "y = [[1, -1, 0], [2, 5, -9], [-2, -3, 0]]\n",
    "for row in y:\n",
    "    for col in row:\n",
    "        if col < 0:\n",
    "            count += 1\n",
    "print(count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "very high\n"
     ]
    }
   ],
   "source": [
    "if x < -5:\n",
    "    print(\"very low\")\n",
    "elif x <= 0:\n",
    "    print(\"low\")\n",
    "elif x <= 5:\n",
    "    print(\"high\")\n",
    "else:\n",
    "    print(\"very high\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: COMPREHENSIONS\n",
    "What list comprehension would you use to process the list x so that all negative values were removed?\n",
    "\n",
    "Create a generator that returns only odd numbers from 1 to 100. (Hint: a number is odd if there is a remainder if divided by 2, use % 2 to for this).\n",
    "\n",
    "Write the code to create a dictionary of the numbers and their cubes from 11 through 15."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 3, 5, 0, 3]\n"
     ]
    }
   ],
   "source": [
    "x = [1, 3, 5, 0, -1, 3, -2]\n",
    "new_x = [i for i in x if i >= 0]\n",
    "print(new_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "3\n",
      "5\n",
      "7\n",
      "9\n",
      "11\n",
      "13\n",
      "15\n",
      "17\n",
      "19\n",
      "21\n",
      "23\n",
      "25\n",
      "27\n",
      "29\n",
      "31\n",
      "33\n",
      "35\n",
      "37\n",
      "39\n",
      "41\n",
      "43\n",
      "45\n",
      "47\n",
      "49\n",
      "51\n",
      "53\n",
      "55\n",
      "57\n",
      "59\n",
      "61\n",
      "63\n",
      "65\n",
      "67\n",
      "69\n",
      "71\n",
      "73\n",
      "75\n",
      "77\n",
      "79\n",
      "81\n",
      "83\n",
      "85\n",
      "87\n",
      "89\n",
      "91\n",
      "93\n",
      "95\n",
      "97\n",
      "99\n"
     ]
    }
   ],
   "source": [
    "odd_100 = (x for x in range(100) if x % 2)            \n",
    "for i in odd_100:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{11: 1331, 12: 1728, 13: 2197, 14: 2744, 15: 3375}\n"
     ]
    }
   ],
   "source": [
    "cubes = {x: x**3 for x in range(11, 16)}\n",
    "print(cubes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: BOOLEANS AND TRUTHINESS\n",
    "Decide if the following statements are true or false: 1, 0, -1, [0], 1 and 0, 1 > 0 or []\n",
    "```\n",
    "1 -> True.\n",
    "0 -> False.\n",
    "-1 - True.\n",
    "[0] - True, it's a list containing one item.\n",
    "1 and 0 - False.\n",
    "1 > 0 or [] - True.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: REFACTOR WORD_COUNT\n",
    "Rewrite the word count program to make it shorter. You may want to look at the string and list operations already discussed, as well as thinking about different ways to organize the code. You may also want to make it smarter, so that only alphabetic strings (not symbols or punctuation) count as words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# File: word_count_refactored.py\n",
    "\"\"\" Reads a file and returns the number of lines, words,\n",
    "    and characters - similar to the UNIX wc utility\n",
    "\"\"\"\n",
    "\n",
    "# initialze counts\n",
    "line_count = 0\n",
    "word_count = 0\n",
    "char_count = 0\n",
    "\n",
    "# open the file\n",
    "with  open('word_count.tst') as infile:\n",
    "    for line in infile:\n",
    "        line_count += 1\n",
    "        char_count += len(line)\n",
    "        words = line.split()\n",
    "        word_count += len(words)\n",
    "\n",
    "# print the answers using the format() method\n",
    "print(\"File has {0} lines, {1} words, {2} characters\".format(line_count, \n",
    "                                               word_count, char_count))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: FUNCTIONS AND PARAMETERS\n",
    "How would you write a function that could take any number of unnamed arguments and print their values out in reverse order?\n",
    "\n",
    "What do you need to do to create a procedure or “void” function, that is a function with no return value?\n",
    "\n",
    "Either don't return a value (use a bare return) or don't use a return statement at all.\n",
    "\n",
    "What happens if you capture the return value of a function with a variable?\n",
    "\n",
    "The only result is that you can use that value, whatever it might be."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n",
      "3\n",
      "2\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "def my_funct(*params):\n",
    "    for i in reversed(params):\n",
    "        print(i)\n",
    "\n",
    "my_funct(1,2,3,4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: MUTABLE FUNCTION PARAMETERS\n",
    "\n",
    "What would be the result of changing a list or dictionary that was passed into a function as a parameter value? \n",
    "\n",
    "The changes would persist for future uses of the default parameter.\n",
    "\n",
    "Which operations would be likely to create changes that would be visible outside the function? What steps might you take to minimize that risk?\n",
    "\n",
    "Operations like adding and deleteing elements, as well as changing the value of an element. To minimize the risk, it's better not to use mutable types as default parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: GLOBAL VS LOCAL VARIABLES\n",
    "Assuming x = 5, what will be the value of x after calling funct_1() below executes? After calling funct_2()?\n",
    "```\n",
    "def funct_1():\n",
    "    x = 3\n",
    "def funct_2():\n",
    "    global x\n",
    "    x = 2 \n",
    "```    \n",
    "After calling `funct_1()` x will be unchanged; after `funct_2()` the value in the global x will be 2. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: GENERATOR FUNCTIONS\n",
    "What would you need to modify in the code for the function four() above to make it work for any number? What would you need to add to allow the starting point to also be set?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "in generator, x = 0\n",
      "0\n",
      "in generator, x = 1\n",
      "1\n",
      "in generator, x = 2\n",
      "2\n",
      "in generator, x = 3\n",
      "3\n"
     ]
    }
   ],
   "source": [
    ">>> def four(limit):\n",
    "...     x = 0 \n",
    "...     while x < limit:\n",
    "...         print(\"in generator, x =\", x)\n",
    "...         yield x\n",
    "...         x += 1\n",
    "... \n",
    ">>> for i in four(4):\n",
    "...     print(i)\n",
    "...\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "in generator, x = 1\n",
      "1\n",
      "in generator, x = 2\n",
      "2\n",
      "in generator, x = 3\n",
      "3\n"
     ]
    }
   ],
   "source": [
    ">>> def four(start, limit):\n",
    "...     x = start \n",
    "...     while x < limit:\n",
    "...         print(\"in generator, x =\", x)\n",
    "...         yield x\n",
    "...         x += 1\n",
    "... \n",
    ">>> for i in four(1, 4):\n",
    "...     print(i)\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: DECORATORS\n",
    "How would you modify the code for the decorator function above to remove unneeded messages and enclose the return value of wrapped function in \"<html>\" and \"</html>\", so that myfunction(\"hello\") would return \"<html>hello<html>\"?\n",
    "    \n",
    "This is a hard one, since in order to define a function that changes the return value we need to add an inner wrapper function to call the orginal function and add to the return value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<html>Test<html>\n"
     ]
    }
   ],
   "source": [
    "def decorate(func):\n",
    "    def wrapper_func(*args):\n",
    "        def inner_wrapper(*args):\n",
    "                return_value = func(*args)\n",
    "                return \"<html>{}<html>\".format(return_value)\n",
    "                \n",
    "        return inner_wrapper(*args)\n",
    "    return wrapper_func\n",
    "\n",
    "@decorate\n",
    "def myfunction(parameter):\n",
    "    return parameter \n",
    "\n",
    "print(myfunction(\"Test\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: USEFUL FUNCTIONS\n",
    "Looking back at the labs for chapters 6 and 7, refactor that code into functions for cleaning and processing the data. The goal should be that most of the logic is moved into functions. Use your own judgement as to the types of functions and parameters, but keep in mind that functions should do just one thing, and they should not have any side effects that carry over outside of the function.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [],
   "source": [
    "punct = str.maketrans(\"\",  \"\", \"!.,:;-?\")\n",
    "\n",
    "def clean_line(line):\n",
    "    \"\"\"changes case and removes punctuation\"\"\"\n",
    "    # make all one case\n",
    "    cleaned_line = line.lower()\n",
    "        \n",
    "    # remove punctuation\n",
    "    cleaned_line = cleaned_line.translate(punct)\n",
    "    return cleaned_line\n",
    "\n",
    "\n",
    "def get_words(line):\n",
    "    \"\"\"splits line into words, and rejoins with newlines\"\"\"\n",
    "    words = line.split()\n",
    "    return \"\\n\".join(words) + \"\\n\"\n",
    "     \n",
    "    \n",
    "with open(\"moby_01.txt\") as infile, open(\"moby_01_clean.txt\", \"w\") as outfile:\n",
    "    for line in infile:\n",
    "        cleaned_line = clean_line(line)\n",
    "                \n",
    "        cleaned_words = get_words(cleaned_line)\n",
    "        \n",
    "        # write all words for line\n",
    "        outfile.write(cleaned_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Most common words:\n",
      "('the', 14)\n",
      "('and', 9)\n",
      "('i', 9)\n",
      "('of', 8)\n",
      "('is', 7)\n",
      "\n",
      "Least common words:\n",
      "('see', 1)\n",
      "('growing', 1)\n",
      "('soul', 1)\n",
      "('having', 1)\n",
      "('regulating', 1)\n"
     ]
    }
   ],
   "source": [
    "def count_words(words):\n",
    "    \"\"\"takes list of cleaned words, returns count dictionary\"\"\"\n",
    "    word_count = {}\n",
    "    for word in moby_words:\n",
    "        count = word_count.setdefault(word, 0)\n",
    "        word_count[word] += 1\n",
    "    return word_count\n",
    "\n",
    "\n",
    "def word_stats(word_count):\n",
    "    \"\"\"Takes word count dictionary and returns top and bottom five entries\"\"\"\n",
    "    word_list = list(word_count.items())\n",
    "    word_list.sort(key=lambda x: x[1])\n",
    "    least_common = word_list[:5]\n",
    "    most_common = word_list[-1:-6:-1]\n",
    "    return most_common, least_common\n",
    "\n",
    "moby_words = []\n",
    "with open('moby_01_clean.txt') as infile:\n",
    "    for word in infile:\n",
    "        if word.strip():\n",
    "            moby_words.append(word.strip())\n",
    "        \n",
    "word_count = count_words(moby_words)\n",
    "\n",
    "most, least = word_stats(word_count)\n",
    "print(\"Most common words:\")\n",
    "for word in most:\n",
    "    print(word)\n",
    "print(\"\\nLeast common words:\")\n",
    "for word in least:\n",
    "    print(word)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: MODULES\n",
    "Suppose you have a module called new_math that contains a function called new_divide. What are the ways that you might import and then use that function? What are the pros and cons of each?\n",
    "\n",
    "```\n",
    "import new_math\n",
    "new_math.new_divide(...)\n",
    "```\n",
    "This is often preferred, since there will not be a clash between any identifiers in new_module and the importing namespace. However it's less convenient to type.\n",
    "\n",
    "```\n",
    "from new_math import new_divide\n",
    "new_divide(...)\n",
    "```\n",
    "More convenient to use, but increases the chance of name clashes between identifiers in the module and the importing namespace.\n",
    "\n",
    "Suppose that the new_math module contains a function call \\_helper_math(). How will the underscore character affect the way that \\_helper_math() is imported? \n",
    "\n",
    "It won't be imported if you use `from new_math import *`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: NAMESPACES AND SCOPE\n",
    "Consider a variable `width` which is in the module make_window.py. In which of the following contexts is `width` in scope? \n",
    "\n",
    "1. within the module itself,  \n",
    "2. inside the resize() function in the module, \n",
    "3. within the script that imported the make_window.py module.\n",
    "\n",
    "1 and 2, but not 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: CREATE A MODULE\n",
    "Package the functions that you created at the end of chapter 9 as a standalone module. While you can include code to run the module as the main program, the goal should be for the functions to be completely usable from another script. \n",
    "\n",
    "(no answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 11"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: MAKING A SCRIPT EXECUTABLE\n",
    "Experiment with executing scripts on your platform. Also try to redirect input and output into and out of your scripts.\n",
    "\n",
    "(no answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: PROGRAMS AND MODULES\n",
    "What issue is the use of if `__name__ == \"__main__\":` meant to avoid, and how does it do that? Can you think of any other way to avoid this issue?\n",
    "\n",
    "When Python loads a module all of its code is executed. By using the pattern above you can have certain code run only if it’s being executed as the main script file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: CREATING A PROGRAM\n",
    "In chapter 8, we created a version of the UNIX wc utility to count the lines, words, and characters in a file.  Now that we have more tools at our disposal, lets refactor that program to make it work more like the original. In particular, it should have options to show only lines (-l), only words (-w), and only characters (-c). If none of those options are given, all three stats will be displayed. But if any one of them is present, then only the specified stats will be shown.\n",
    "\n",
    "For an extra challenge, have a look at the man page for wc on a Linux/UNIX system and add the -L to show the longest line length. Feel free to try to implement the complete behavior as listed in the man page and test it against your system’s wc utility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 9, 58)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# File: word_count_program.py\n",
    "\"\"\" Reads a file and returns the number of lines, words,\n",
    "    and characters - similar to the UNIX wc utility\n",
    "\"\"\"\n",
    "import sys\n",
    "\n",
    "\n",
    "def main():\n",
    "    # initialze counts\n",
    "    line_count = 0\n",
    "    word_count = 0\n",
    "    char_count = 0\n",
    "    \n",
    "    option = None\n",
    "    params = sys.argv[1:]\n",
    "    if len(params) > 1:\n",
    "        # if more than one param, pop the first one as the option\n",
    "        option = params.pop(0).lower().strip()\n",
    "    filename = params[0]    # open the file\n",
    "    with  open(filename) as infile:\n",
    "        for line in infile:\n",
    "            line_count += 1\n",
    "            char_count += len(line)\n",
    "            words = line.split()\n",
    "            word_count += len(words)\n",
    "    \n",
    "    if option == \"-c\":\n",
    "        print(\"File has {} characters\".format(char_count))\n",
    "    elif option == \"-w\":\n",
    "        print(\"File has {} words\".format(word_count))\n",
    "    elif option == \"-l\":\n",
    "        print(\"File has {} lines\".format(line_count))\n",
    "    else:\n",
    "        # print the answers using the format() method\n",
    "        print(\"File has {0} lines, {1} words, {2} characters\".format(line_count, \n",
    "                                                       word_count, char_count))\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    main()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 12"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: MANIPULATING PATHS\n",
    "How would you use the os module’s functions to take a path to a file called `test.log` and create a new file path in the same directory for a file called `test.log.old`? How would you do the same thing using the pathlib module?\n",
    "\n",
    "What path would you get if you created a pathlib Path object from os.pardir? Try it and find out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/naomi/Documents/QPB3E/qpbe3e/exercise_answers/test.log\n",
      "/home/naomi/Documents/QPB3E/qpbe3e/exercise_answers/test.log.old\n"
     ]
    }
   ],
   "source": [
    "import os.path\n",
    "old_path = os.path.abspath('test.log')\n",
    "print(old_path)\n",
    "new_path = '{}.{}'.format(old_path, \"old\")\n",
    "print(new_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/naomi/Documents/QPB3E/qpbe3e/exercise_answers/test.log\n",
      "/home/naomi/Documents/QPB3E/qpbe3e/exercise_answers/test.log.old\n"
     ]
    }
   ],
   "source": [
    "import pathlib\n",
    "path = pathlib.Path('test.log')\n",
    "abs_path = path.resolve()\n",
    "print(abs_path)\n",
    "new_path = str(abs_path) + \".old\"\n",
    "print(new_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "..\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "PosixPath('/home/naomi/Documents/QPB3E/qpbe3e')"
      ]
     },
     "execution_count": 194,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_path = pathlib.Path(os.pardir)\n",
    "print(test_path)\n",
    "test_path.resolve()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: MORE FILE OPERATIONS\n",
    "How might you calculate the total size of all files ending with .txt that are not symlinks in a directory? If your first answer was using os.path, also try it with pathlib, and vice versa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6887\n"
     ]
    }
   ],
   "source": [
    "import pathlib\n",
    "cur_path = pathlib.Path(\".\")\n",
    "\n",
    "size = 0\n",
    "for text_path in cur_path.glob(\"*.txt\"):\n",
    "    if not text_path.is_symlink():\n",
    "        size += text_path.stat().st_size\n",
    "        \n",
    "print(size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write some code that builds off your solution above to move the same .txt files in the question above to a new subdirectory called 'backup' in the same directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "ename": "FileNotFoundError",
     "evalue": "[Errno 2] No such file or directory: 'moby_01.txt' -> 'backup/moby_01.txt'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-19-5e5613b8adbe>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      7\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mtext_path\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_symlink\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m         \u001b[0msize\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0mtext_path\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mst_size\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m         \u001b[0mtext_path\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrename\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_path\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoinpath\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext_path\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     10\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/lib/python3.5/pathlib.py\u001b[0m in \u001b[0;36mrename\u001b[0;34m(self, target)\u001b[0m\n\u001b[1;32m   1277\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_closed\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1278\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_raise_closed\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1279\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_accessor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrename\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1280\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1281\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mreplace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/lib/python3.5/pathlib.py\u001b[0m in \u001b[0;36mwrapped\u001b[0;34m(pathobjA, pathobjB, *args)\u001b[0m\n\u001b[1;32m    375\u001b[0m         \u001b[0;34m@\u001b[0m\u001b[0mfunctools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwraps\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstrfunc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    376\u001b[0m         \u001b[0;32mdef\u001b[0m \u001b[0mwrapped\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpathobjA\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpathobjB\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 377\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mstrfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpathobjA\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpathobjB\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    378\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mstaticmethod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mwrapped\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    379\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: 'moby_01.txt' -> 'backup/moby_01.txt'"
     ]
    }
   ],
   "source": [
    "import pathlib\n",
    "cur_path = pathlib.Path(\".\")\n",
    "new_path = cur_path.joinpath(\"backup\")\n",
    "\n",
    "size = 0\n",
    "for text_path in cur_path.glob(\"*.txt\"):\n",
    "    if not text_path.is_symlink():\n",
    "        size += text_path.stat().st_size\n",
    "        text_path.rename(new_path.joinpath(text_path.name))\n",
    "        \n",
    "print(size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 13"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK:\n",
    "What is the significance of adding a \"b\" to the file open mode string?\n",
    "\n",
    "It makes the file open in binary mode, reading and writng bytes, not characters.\n",
    "\n",
    "Suppose you want to open a file named myfile.txt and write some additional data on the end of it. What command would you use to open myfile.txt? What command would you use to re-open the file to read from the beginning?\n",
    "\n",
    "```\n",
    "open(\"myfile.txt\", \"a\")\n",
    "open(\"myfile.txt\")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: REDIRECTING INPUT AND OUTPUT\n",
    "Write some code to use the mio.py module above to capture all of the print output of a script to a file named \"myfile.txt\" and then reset the standard output to the screen, and print that file to screen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# mio_test.py\n",
    "\n",
    "import mio\n",
    "\n",
    "def main():\n",
    "    mio.capture_output(\"myfile.txt\")\n",
    "    print(\"hello\")\n",
    "    print(1 + 3)\n",
    "    mio.restore_output()\n",
    "    \n",
    "    mio.print_file(\"myfile.txt\")\n",
    "    \n",
    "    \n",
    "if __name__ == '__main__':\n",
    "    main()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: STRUCT\n",
    "What use cases can you think of where the struct module would be useful for either reading or writing binary data?\n",
    "\n",
    "* You are trying to read/write from a binary format application file, or image file.\n",
    "* You are reading from some external interface, e.g., a thermometer or accelerometer, and want to save the raw dat exactly as it was transmitted.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: PICKLES\n",
    "Think about why a pickle would be a good solution for the following use cases, or why not: A) saving some state variables from one on run to the next; B) keeping a high score list for a game; C) storing user names and passwords; D) storing a large dictionary of English terms.\n",
    "\n",
    "A & B would be reasonable, although pickles are not secure. \n",
    "\n",
    "C & D would not be good, the lack of security would be a big problem for C, and for D there would be a need to load the entire pickle into memory. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: SHELVE\n",
    "Using a shelf object looks very much like using a dictionary. In what ways is using a shelf object different? What disadvantages would you expect there to be in using a shelf object?\n",
    "\n",
    "The key difference is that the objects are stored on disk, not in memory. With very large amounts of data, particularly with lots of inserts and/or deletes, you would expect disk access to make things slow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: FINAL FIXES TO WC\n",
    "If you look at the man page for the wc utility you will see that there are two command line options which do very similar things. -c makes the utility count the bytes in the file, while -m makes it count characters (which in the case of some Unicode characters can be two or more bytes long). In addition, if a file is given it should read from and process that file, but if no file is given, it should read from and process stdin.\n",
    "\n",
    "Rewrite your version of the `wc` utility to implement both the distinction between bytes and characters and the ability to read from files and standard input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-20-5afaab331889>, line 17)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-20-5afaab331889>\"\u001b[0;36m, line \u001b[0;32m17\u001b[0m\n\u001b[0;31m    if params and params[0].startswith(\"-\":\u001b[0m\n\u001b[0m                                          ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "# File: word_count_program_stdin.py\n",
    "\"\"\" Reads a file and returns the number of lines, words,\n",
    "    and characters - similar to the UNIX wc utility\n",
    "\"\"\"\n",
    "import sys\n",
    "\n",
    "\n",
    "def main():\n",
    "    # initialze counts\n",
    "    line_count = 0\n",
    "    word_count = 0\n",
    "    char_count = 0\n",
    "    filename = None\n",
    "    \n",
    "    option = None\n",
    "    if len(sys.argv) > 1:\n",
    "        params = sys.argv[1:]\n",
    "        if params[0].startswith(\"-\"):\n",
    "        # if more than one param, pop the first one as the option\n",
    "            option = params.pop(0).lower().strip()\n",
    "        if params:\n",
    "            filename = params[0]    # open the file\n",
    "    file_mode = \"r\"\n",
    "    if option == \"-c\":\n",
    "        file_mode = \"rb\"\n",
    "    if filename:\n",
    "        infile =  open(filename, file_mode)\n",
    "    else:\n",
    "        infile = sys.stdin\n",
    "    with infile:\n",
    "        for line in infile:\n",
    "            line_count += 1\n",
    "            char_count += len(line)\n",
    "            words = line.split()\n",
    "            word_count += len(words)\n",
    "    \n",
    "    if option in (\"-c\", \"-m\"):\n",
    "        print(\"File has {} characters\".format(char_count))\n",
    "    elif option == \"-w\":\n",
    "        print(\"File has {} words\".format(word_count))\n",
    "    elif option == \"-l\":\n",
    "        print(\"File has {} lines\".format(line_count))\n",
    "    else:\n",
    "        # print the answers using the format() method\n",
    "        print(\"File has {0} lines, {1} words, {2} characters\".format(line_count, \n",
    "                                                       word_count, char_count))\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    main()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 14"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: CATCHING EXCEPTIONS\n",
    "Write some code that gets two numbers from the user and divides the first by the second. Check for and catch the exception that occurs if the second number is zero (ZeroDivisionError)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Please enter an integer: 1\n",
      "Please enter another integer: 0\n",
      "Can't divide by zero.\n"
     ]
    }
   ],
   "source": [
    "# the code of your program should do the following\n",
    "x = int(input(\"Please enter an integer: \"))\n",
    "y = int(input(\"Please enter another integer: \"))\n",
    "\n",
    "try:\n",
    "    z = x / y\n",
    "except ZeroDivisionError as e:\n",
    "    print(\"Can't divide by zero.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: EXCEPTIONS AS CLASSES\n",
    "If `MyError` inherits from `Exception`, what will be the difference between `except Exception as e` and `except MyError as e`?\n",
    "\n",
    "The first will catch any exception that inherits from `Exception`, i.e., most of them, while the second will only catch `MyError` exceptions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: THE ASSERT STATEMENT\n",
    "Write a simple program which gets a number from the user and then uses the assert statement to raise an exception if the number is zero. Test to make sure the assert fires, then turn it off using one of the methods mentioned above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Please enter a non-zero integer: 0\n"
     ]
    },
    {
     "ename": "AssertionError",
     "evalue": "Integer can not be zero.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAssertionError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-222-9f7a09820a1c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Please enter a non-zero integer: \"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32massert\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"Integer can not be zero.\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mAssertionError\u001b[0m: Integer can not be zero."
     ]
    }
   ],
   "source": [
    "# the code of your program should do the following\n",
    "x = int(input(\"Please enter a non-zero integer: \"))\n",
    "\n",
    "assert x != 0, \"Integer can not be zero.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: EXCEPTIONS\n",
    "Do Python exceptions force a program to halt?\n",
    "\n",
    "No, if they are caught and handled correctly the program won't need to halt.\n",
    "\n",
    "Suppose you wanted accessing a dictionary `x` to always return `None` if a key didn’t exist in the dictionary (i.e., if `KeyError` exception was raised)? What code would you use to achieve that?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    x = my_dict[some_key]\n",
    "except KeyError as e:\n",
    "    x = None\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: EXCEPTIONS\n",
    "What code would you use to create a custom ValueTooLarge exception and raise that exception if the variable x is over 1000?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueTooLarge",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueTooLarge\u001b[0m                             Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-163-c422915c7c52>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1001\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m1000\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m     \u001b[0;32mraise\u001b[0m \u001b[0mValueTooLarge\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueTooLarge\u001b[0m: "
     ]
    }
   ],
   "source": [
    "class ValueTooLarge(Exception):\n",
    "    pass\n",
    "\n",
    "x = 1001\n",
    "if x > 1000:\n",
    "    raise ValueTooLarge()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: CONTEXT MANAGERS\n",
    "Assume you are using a context manager in a script that reads and/or writes several files. Which of the following approaches do you think would be the best? A) put the entire script in a block managed by a `with` statement; B) Use one `with` statement for all file reads and another for all file writes; C) a `with` statement each time you read a file or write a file, i.e., for each line; D) use a `with` statement for each file that you read or write.\n",
    "\n",
    "Probably D is the best - since part of what the context manager does for file access is to make sure that a file is closed, it would probably make sense to use a separate w`with` for each case where you open a file for reading or writing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: CUSTOM EXCEPTIONS\n",
    "Think about the module you wrote in chapter 9 to count word frequencies. What errors might reasonably occur in those functions? Rewrite the code to handle those exception conditions appropriately. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [],
   "source": [
    "punct = str.maketrans(\"\",  \"\", \"!.,:;-?\")\n",
    "class EmptyStringError(Exception):\n",
    "    pass\n",
    "\n",
    "def clean_line(line):\n",
    "    \"\"\"changes case and removes punctuation\"\"\"\n",
    "    \n",
    "    # raise exception if line is empty\n",
    "    if not line.strip():\n",
    "        raise EmptyStringError()\n",
    "    # make all one case\n",
    "    cleaned_line = line.lower()\n",
    "        \n",
    "    # remove punctuation\n",
    "    cleaned_line = cleaned_line.translate(punct)\n",
    "    return cleaned_line\n",
    "\n",
    "\n",
    "def get_words(line):\n",
    "    \"\"\"splits line into words, and rejoins with newlines\"\"\"\n",
    "    words = line.split()\n",
    "    return \"\\n\".join(words) + \"\\n\"\n",
    "     \n",
    "    \n",
    "with open(\"moby_01.txt\") as infile, open(\"moby_01_clean.txt\", \"w\") as outfile:\n",
    "    for line in infile:\n",
    "        cleaned_line = clean_line(line)\n",
    "                \n",
    "        cleaned_words = get_words(cleaned_line)\n",
    "        \n",
    "        # write all words for line\n",
    "        outfile.write(cleaned_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Most common words:\n",
      "('the', 14)\n",
      "('i', 9)\n",
      "('and', 9)\n",
      "('of', 8)\n",
      "('is', 7)\n",
      "\n",
      "Least common words:\n",
      "('can', 1)\n",
      "('money', 1)\n",
      "('philosophical', 1)\n",
      "('at', 1)\n",
      "('into', 1)\n"
     ]
    }
   ],
   "source": [
    "def count_words(words):\n",
    "    \"\"\"takes list of cleaned words, returns count dictionary\"\"\"\n",
    "    word_count = {}\n",
    "    for word in words:\n",
    "        try:\n",
    "            count = word_count.setdefault(word, 0)\n",
    "        except TypeError:\n",
    "            #if 'word' is not hashable, skip to next word.\n",
    "            pass\n",
    "        word_count[word] += 1\n",
    "    return word_count\n",
    "\n",
    "\n",
    "def word_stats(word_count):\n",
    "    \"\"\"Takes word count dictionary and returns top and bottom five entries\"\"\"\n",
    "    word_list = list(word_count.items())\n",
    "    word_list.sort(key=lambda x: x[1])\n",
    "    try:\n",
    "        least_common = word_list[:5]\n",
    "        most_common = word_list[-1:-6:-1]\n",
    "    except IndexError as e:\n",
    "        # if list is empty or too short, just return list\n",
    "        least_common = word_list\n",
    "        most_common = list(reversed(word_list))\n",
    "        \n",
    "    return most_common, least_common\n",
    "\n",
    "moby_words = []\n",
    "with open('moby_01_clean.txt') as infile:\n",
    "    for word in infile:\n",
    "        if word.strip():\n",
    "            moby_words.append(word.strip())\n",
    "        \n",
    "word_count = count_words(moby_words)\n",
    "\n",
    "most, least = word_stats(word_count)\n",
    "print(\"Most common words:\")\n",
    "for word in most:\n",
    "    print(word)\n",
    "print(\"\\nLeast common words:\")\n",
    "for word in least:\n",
    "    print(word)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 15"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: CLASSES AS RECORDS\n",
    "What code would you use to create a `Rectangle` class?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Rectangle:\n",
    "    def __init__(self): \n",
    "        self.height = 1\n",
    "        self.width = 2\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: INSTANCE VARIABLES AND METHODS\n",
    "Update the code for a `Rectangle` class so that you can set the dimensions when an instance is created, just as for the `Circle` class above. Also add an `area()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Rectangle:\n",
    "    def __init__(self, width, height): \n",
    "        self.height = height\n",
    "        self.width = width\n",
    "        \n",
    "    def area(self):\n",
    "        return self.height * self.width"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: CLASS METHODS\n",
    "Write a class method similar to `total_area()`, but that would return the total circumference of all circles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Circle:\n",
    "    pi = 3.14159\n",
    "    all_circles = []\n",
    "    def __init__(self, radius):\n",
    "        self.radius = radius\n",
    "        self.__class__.all_circles.append(self)    \n",
    "        \n",
    "    def area(self):\n",
    "        return self.radius * self.radius * Circle.pi\n",
    "    \n",
    "    def circumference(self):\n",
    "        return 2 * self.radius * Circle.pi\n",
    "    \n",
    "    @classmethod\n",
    "    def total_circumference(cls):\n",
    "        \"\"\"class method to total the circumference of all Circles \"\"\"\n",
    "        total = 0\n",
    "        for c in cls.all_circles:\n",
    "            total = total + c.circumference()\n",
    "        return total\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: INHERITANCE\n",
    "Rewrite the code for a Rectangle class to inherit from Shape. Since squares and rectangles are related, would it make sense to inherit one from the other? If so, which would be the base class and which would inherit?\n",
    "\n",
    "How would you write the code to add an area() method for the Square class? Should the area method be moved into the base Shape class and inherited by circle, square and rectangle? What issues would that cause?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Shape:\n",
    "    def __init__(self, x, y):\n",
    "        self.x = x\n",
    "        self.y = y\n",
    "\n",
    "class Rectangle(Shape):\n",
    "    def __init__(self, x, y):\n",
    "        super().__init__(x, y)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It probably would make sense. Since squares are a special kind of rectangle, Square should inherit from the Rectangle class.\n",
    "\n",
    "If square was specializes so that it had only one dimension x, then one would write:\n",
    "```\n",
    "def area(self):\n",
    "    return self.x * self.x\n",
    "```\n",
    "It make sense to put the area method in a Rectangle class that Square inherits from, but putting it in Shape wouldn't be very helpful, since different types of shapes have their own rules for calculating area, so every shape would be overriding the base area method anyway."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: PRIVATE INSTANCE VARIABLES\n",
    "Modify the Rectangle class’s code to make the dimension variables private. What restriction will this impose on using the class?\n",
    "\n",
    "The dimension variables will no longer be accessible outside the class via `.x` and `.y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Rectangle():\n",
    "    def __init__(self, x, y):\n",
    "        self.__x = x\n",
    "        self.__y = y\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: PROPERTIES\n",
    "Update the dimensions of the Rectangle class to be properties with getter and setters that will not allow negative sizes.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 2\n",
      "4 5\n"
     ]
    }
   ],
   "source": [
    "class Rectangle():\n",
    "    def __init__(self, x, y):\n",
    "        self.__x = x\n",
    "        self.__y = y\n",
    "    \n",
    "    @property\n",
    "    def x(self):\n",
    "        return self.__x\n",
    "            \n",
    "    @x.setter\n",
    "    def x(self, new_x):\n",
    "        if new_x >= 0:\n",
    "            self.__x = new_x\n",
    "            \n",
    "    @property\n",
    "    def y(self):\n",
    "        return self.__y\n",
    "            \n",
    "    @y.setter\n",
    "    def y(self, new_y):\n",
    "        if new_y >= 0:\n",
    "            self.__y = new_y\n",
    "            \n",
    "my_rect = Rectangle(1,2)\n",
    "print(my_rect.x, my_rect.y)\n",
    "my_rect.x = 4\n",
    "my_rect.y = 5\n",
    "print(my_rect.x, my_rect.y)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: HTML CLASSES\n",
    "In this lab we’ll create classes to represent an HTML document. To keep things simple lets just assume that each element can contain only text and one sub-element. So the `<html>` element will only contain a `<body>` element, and the `<body>` element will contain (optional)text and a `<p>` element, which will contain only text. \n",
    "  \n",
    "The key feature to implement is the `__str__()` method, which will in turn call its sub-element's `__str__()` method, so that the entire document is returned when the str() function is called on an `<html>` element. We can assume that any text comes before the subelement.\n",
    "    \n",
    "Example output from using the classes:\n",
    "```\n",
    "para = p(text=\"this is some body text\")\n",
    "doc_body = body(text=\"This is the body\", subelement=para)\n",
    "doc = html(subelement=doc_body)\n",
    "print(doc)\n",
    "\n",
    "<html>\n",
    "<body>\n",
    "This is the body\n",
    "<p>\n",
    "this is some body text\n",
    "</p>\n",
    "</body>\n",
    "</html>\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<html>\n",
      "<body>\n",
      "This is the body\n",
      "<p>\n",
      "this is some body text\n",
      "</p>\n",
      "</body>\n",
      "</html>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "class element:\n",
    "    def __init__(self, text=None, subelement=None):\n",
    "        self.subelement = subelement\n",
    "        self.text = text\n",
    "        \n",
    "    def __str__(self):\n",
    "        value = \"<{}>\\n\".format(self.__class__.__name__)\n",
    "        if self.text:\n",
    "            value += \"{}\\n\".format(self.text)\n",
    "        if self.subelement:\n",
    "            value += str(self.subelement)\n",
    "        value += \"</{}>\\n\".format(self.__class__.__name__)\n",
    "        return value\n",
    "\n",
    "class html(element):\n",
    "    def __init__ (self, text=None, subelement=None):\n",
    "        super().__init__(text, subelement)\n",
    "    def __str__(self):\n",
    "        return super().__str__()\n",
    "        \n",
    "class body(element):\n",
    "    def __init__ (self, text=None, subelement=None):\n",
    "        return super().__init__(text, subelement)\n",
    "    def __str__(self):\n",
    "        return super().__str__()\n",
    "        \n",
    "class p(element):\n",
    "    def __init__(self, text=None, subelement=None):\n",
    "        super().__init__(text, subelement)\n",
    "    def __str__(self):\n",
    "        return super().__str__()\n",
    "        \n",
    "        \n",
    "para = p(text=\"this is some body text\")\n",
    "doc_body = body(text=\"This is the body\", subelement=para)\n",
    "doc = html(subelement=doc_body)\n",
    "print(doc)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 16"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: SPECIAL CHARACTERS IN REGULAR EXPRESSIONS\n",
    "What regular expression would you use to match strings which represent the numbers -5 through 5?\n",
    "\n",
    "`r\"-{0,1}[0-5]\"` will match strings which represent the numbers -5 through 5.\n",
    "\n",
    "What regular expression would you use to match a hexadecimal digit? Assume that allowed hexadecimal digits are 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, A, a, B, b, C, c, D, d, E, e, F, and f.\n",
    "\n",
    "`r\"[0-9A-Fa-f]\"`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: EXTRACTING MATCHED TEXT\n",
    "Making international calls usually requires a '+' and the country code. Assuming that the country code will be two digits, how would you modify the code above to extract the plus and the country code as part of the number. (Again, not all numbers will have a country code.) How would you make it handle country codes of one to three digits?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 296,
   "metadata": {},
   "outputs": [],
   "source": [
    "re.match(r\": (?P<phone>(\\+\\d{2}-)?(\\d\\d\\d-)?\\d\\d\\d-\\d\\d\\d\\d)\", \": +01-111-222-3333\")\n",
    "#or\n",
    "re.match(r\": (?P<phone>(\\+\\d{2}-)?(\\d{3}-)?\\d{3}-\\d{4})\", \": +01-111-222-3333\")\n",
    "#for 1 to 3 digit country codes:\n",
    "re.match(r\": (?P<phone>(\\+\\d{1,3}-)?(\\d{3}-)?\\d{3}-\\d{4})\", \": +011-111-222-3333\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: REPLACING TEXT\n",
    "In the checkpoint above you extended a phone number regular expression to also recognize a country code. How would you now use a function to make any numbers that didn’t have a country code now have '+1' (the country code for the US and Canada)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 286,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'+1 111-222-3333'"
      ]
     },
     "execution_count": 286,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def add_code(match_obj):\n",
    "    return(\"+1 \"+match_obj.group('phone'))\n",
    "\n",
    "re.sub(r\"(?P<phone>(\\d{3}-)?\\d{3}-\\d{4})\", add_code, \"111-222-3333\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB 16: PHONE NUMBER NORMALIZER\n",
    "In the USA and Canada phone numbers consist of 10 digits, usually separated into a 3 digit area code, a 3 digit exchange code, and 4 digit station code. As mentioned above they may or may not be preceded by +1, the country code. However, in practice there are many ways of formatting a phone number: (NNN) NNN-NNNN, NNN-NNN-NNNN, NNN NNN-NNNN, NNN.NNN.NNNN, and NNN NNN NNNN, to name a few. And the country code may or may not be present, and may or may not have a plus, and is usually (not always) separated from the number by a space or dash. Whew!\n",
    "In this lab the task is to create a phone number normalizer that will take any of the formats mentioned above and return a normalized phone number 1-NNN-NNN-NNNN. \n",
    "The following are all possible phone numbers:\n",
    "    \n",
    "+1 223-456-7890\t\n",
    "1-223-456-7890\t\n",
    "+1 223 456-7890\n",
    "(223) 456-7890\t\n",
    "1 223 456 7890\t\n",
    "223.456.7890\n",
    "\n",
    "Bonus: The first digit of the area code and the exchange code can only be 2-9, and the second digit of an area code can’t be 9. Use this information to validate the input and return a `ValueError` Exception of “invalid phone number” if the number is invalid."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1-223-456-7890\n",
      "1-223-456-7890\n",
      "1-223-456-7890\n",
      "1-223-456-7890\n",
      "1-223-456-7890\n",
      "1-223-456-7890\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "invalid phone number exchange 111",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-4-718a27352435>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     27\u001b[0m \u001b[0mregexp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mre\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mr\"\\+?(?P<country>\\d{1,3})?[- .]?\\(?(?P<area>\\d{3})\\)?[- .]?(?P<exch>(\\d{3}))[- .](?P<number>\\d{4})\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     28\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mnumber\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtest_numbers\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mregexp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msub\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreturn_number\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnumber\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-4-718a27352435>\u001b[0m in \u001b[0;36mreturn_number\u001b[0;34m(match_obj)\u001b[0m\n\u001b[1;32m     15\u001b[0m         \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"invalid phone number area code {}\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmatch_obj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgroup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"area\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mre\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mr\"[2-9]\\d\\d\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmatch_obj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgroup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"exch\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 17\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"invalid phone number exchange {}\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmatch_obj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgroup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"exch\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     18\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     19\u001b[0m     \u001b[0mcountry\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmatch_obj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgroup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"country\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: invalid phone number exchange 111"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "test_numbers = [\"+1 223-456-7890\",\n",
    "                \"1-223-456-7890\",\n",
    "                \"+1 223 456-7890\",\n",
    "                \"(223) 456-7890\",\n",
    "                \"1 223 456 7890\",\n",
    "                \"223.456.7890\",\n",
    "               \"1-989-111-2222\"]\n",
    "\n",
    "def return_number(match_obj):\n",
    "    \n",
    "    # validate number raise ValueError if not valid\n",
    "    if not re.match(r\"[2-9][0-8]\\d\", match_obj.group(\"area\") ): \n",
    "        raise ValueError(\"invalid phone number area code {}\".format(match_obj.group(\"area\")))\n",
    "    if not re.match(r\"[2-9]\\d\\d\", match_obj.group(\"exch\") ):\n",
    "        raise ValueError(\"invalid phone number exchange {}\".format(match_obj.group(\"exch\")))\n",
    "        \n",
    "    country = match_obj.group(\"country\")\n",
    "    if not country:\n",
    "        country = \"1\"\n",
    "        \n",
    "    return(\"{}-{}-{}-{}\".format(country, match_obj.group('area'), \n",
    "                                match_obj.group('exch'), match_obj.group('number')))\n",
    "\n",
    "\n",
    "regexp = re.compile(r\"\\+?(?P<country>\\d{1,3})?[- .]?\\(?(?P<area>\\d{3})\\)?[- .]?(?P<exch>(\\d{3}))[- .](?P<number>\\d{4})\")\n",
    "for number in test_numbers:\n",
    "    print(regexp.sub(return_number, number))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 17"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: TYPES\n",
    "Suppose you wanted to make sure that object x was a list before you tried appending to it? What code would you use? What would be the difference between using type() and isinstance()? Would this be the LBYL or EAFP of programming? What other options might you have besides checking the type explicitly? \n",
    "\n",
    "```\n",
    "x = []\n",
    "if isinstance(x, list):\n",
    "    print(\"is list\")\n",
    "```\n",
    "Using `type` would only get lists, not anything that subclassed lists. Either way it's LBYL programming. \n",
    "\n",
    "You might also wrap the append in a try... except block and catch TypeError exceptions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "is list\n"
     ]
    }
   ],
   "source": [
    "x = []\n",
    "if isinstance(x, list):\n",
    "    print(\"is list\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: __GETITEM__\n",
    "The example use of \\__getitem__ above is very limited and will not work correctly in many situations. What are some cases where the implementation above will fail or work incorrectly?\n",
    "\n",
    "The implementation of will not work if you try to access an item directly by index, nor can you move backwards. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: IMPLEMENTING LIST SPECIAL METHODS\n",
    "Try implementing the `__len__` and `__delitem__` special methods listed above, and an `append` method. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n",
      "[1, 2, 1]\n"
     ]
    }
   ],
   "source": [
    "class TypedList:\n",
    "    def __init__(self, example_element, initial_list=[]):\n",
    "        self.type = type(example_element)\n",
    "        if not isinstance(initial_list, list):\n",
    "            raise TypeError(\"Second argument of TypedList must \" \n",
    "                            \"be a list.\")\n",
    "        for element in initial_list: \n",
    "            self.__check(element)\n",
    "        self.elements = initial_list[:]\n",
    "    def __check(self, element):\n",
    "        if type(element) != self.type:\n",
    "            raise TypeError(\"Attempted to add an element of \" \n",
    "                            \"incorrect type to a typed list.\")\n",
    "    def __setitem__(self, i, element):\n",
    "        self.__check(element)\n",
    "        self.elements[i] = element\n",
    "    def __getitem__(self, i):\n",
    "        return self.elements[i]\n",
    "\n",
    "    # added methods\n",
    "    def __delitem__(self, i):\n",
    "        del self.elements[i]\n",
    "    def __len__(self):\n",
    "        return len(self.elements)\n",
    "    def append(self, element):\n",
    "        self.__check(element)\n",
    "        self.elements.append(element)\n",
    "\n",
    "x = TypedList(1, [1,2,3])\n",
    "print(len(x))\n",
    "x.append(1)\n",
    "del x[2]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: SPECIAL METHOD ATTRIBUTES AND SUBCLASSING EXISTING TYPES\n",
    "Suppose you wanted a dictionary-like type that only allowed strings as keys (maybe to make it work like a shelf object as described in Chapter 13). What options would you have for creating such a class? What would be the advantages and disadvantages of each?\n",
    "\n",
    "You could use the same approach as we did for typed list and inherit from the UserDict class. You could also inherit directly from dict, or you could implement all of the dict functionality yourself. \n",
    "\n",
    "Implementing everything yourself gives the most control, but is the most work, and will be most prone to bugs. If the changes you need to make are small (in this case, just checking the type before adding a key), it might make the most sense to inherit directly from dict. On the other hand, inheriting from UserDict is probably the safest, since the internal dict object will continue to be a regular dict, which is a highly optimized and mature implementation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 18"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: PACKAGES\n",
    "Suppose you are writing a package that will take a URL, retrieve all images on the page pointed to by that URL, resize them to a standard size, and store them. Leaving aside the exact details of how each of these functions will be coded, how would you organize those features into a package?\n",
    "\n",
    "There are three separate types of action the package will be doing: first, fetching a page and parsing the HTML for image URL's; second, fetching the images; and third, resizing them. Because of this, you might consider having 3 modules to keep things separate:\n",
    "\n",
    "```\n",
    "picture_fetch/\n",
    "    __init__.py\n",
    "    find.py\n",
    "    fetch.py\n",
    "    resize.py\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: CREATE A PACKAGE\n",
    "In chapter 15 we added error handling to the text cleaning and word frequency counting module we created in chapter 11. Refactor that code into a package containing one module for the cleaning functions, one for the processing functions, and one for the custom exceptions. Then write a simple main function that uses all three.\n",
    "\n",
    "```\n",
    "word_count\n",
    "    __init__.py\n",
    "    exceptions.py\n",
    "    cleaning.py\n",
    "    counter.py\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#__init__.py\n",
    "from exceptions import EmptyStringError\n",
    "from cleaning import clean_line, get_words\n",
    "from counter import count_words, word_stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# exceptions.py\n",
    "class EmptyStringError(Exception):\n",
    "    pass\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [],
   "source": [
    "# cleaning.py\n",
    "from word_count import EmptyStringError\n",
    "\n",
    "punct = str.maketrans(\"\",  \"\", \"!.,:;-?\")\n",
    "\n",
    "def clean_line(line):\n",
    "    \"\"\"changes case and removes punctuation\"\"\"\n",
    "    \n",
    "    # raise exception if line is empty\n",
    "    if not line.strip():\n",
    "        raise EmptyStringError()\n",
    "    # make all one case\n",
    "    cleaned_line = line.lower()\n",
    "        \n",
    "    # remove punctuation\n",
    "    cleaned_line = cleaned_line.translate(punct)\n",
    "    return cleaned_line\n",
    "\n",
    "\n",
    "def get_words(line):\n",
    "    \"\"\"splits line into words, and rejoins with newlines\"\"\"\n",
    "    words = line.split()\n",
    "    return \"\\n\".join(words) + \"\\n\"\n",
    "     \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Most common words:\n",
      "('the', 14)\n",
      "('and', 9)\n",
      "('i', 9)\n",
      "('of', 8)\n",
      "('is', 7)\n",
      "\n",
      "Least common words:\n",
      "('see', 1)\n",
      "('growing', 1)\n",
      "('soul', 1)\n",
      "('having', 1)\n",
      "('regulating', 1)\n"
     ]
    }
   ],
   "source": [
    "# counter.py\n",
    "\n",
    "def count_words(words):\n",
    "    \"\"\"takes list of cleaned words, returns count dictionary\"\"\"\n",
    "    word_count = {}\n",
    "    for word in moby_words:\n",
    "        try:\n",
    "            count = word_count.setdefault(word, 0)\n",
    "        except TypeError:\n",
    "            #if 'word' is not hashable, skip to next word.\n",
    "            pass\n",
    "        word_count[word] += 1\n",
    "    return word_count\n",
    "\n",
    "\n",
    "def word_stats(word_count):\n",
    "    \"\"\"Takes word count dictionary and returns top and bottom five entries\"\"\"\n",
    "    word_list = list(word_count.items())\n",
    "    word_list.sort(key=lambda x: x[1])\n",
    "    try:\n",
    "        least_common = word_list[:5]\n",
    "        most_common = word_list[-1:-6:-1]\n",
    "    except IndexError as e:\n",
    "        # if list is empty or too short, just return list\n",
    "        least_common = word_list\n",
    "        most_common = list(reversed(word_list))\n",
    "        \n",
    "    return most_common, least_common\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Most common words:\n",
      "('the', 14)\n",
      "('i', 9)\n",
      "('and', 9)\n",
      "('of', 8)\n",
      "('is', 7)\n",
      "\n",
      "Least common words:\n",
      "('can', 1)\n",
      "('money', 1)\n",
      "('philosophical', 1)\n",
      "('at', 1)\n",
      "('into', 1)\n"
     ]
    }
   ],
   "source": [
    "from word_count import clean_line, get_words, count_words, word_stats, EmptyStringError\n",
    "\n",
    "with open(\"moby_01.txt\") as infile, open(\"moby_01_clean.txt\", \"w\") as outfile:\n",
    "    for line in infile:\n",
    "        try:\n",
    "            cleaned_line = clean_line(line)\n",
    "        except EmptyStringError:\n",
    "            continue\n",
    "                \n",
    "        cleaned_words = get_words(cleaned_line)\n",
    "        \n",
    "        # write all words for line\n",
    "        outfile.write(cleaned_words)\n",
    "        \n",
    "moby_words = []\n",
    "with open('moby_01_clean.txt') as infile:\n",
    "    for word in infile:\n",
    "        if word.strip():\n",
    "            moby_words.append(word.strip())\n",
    "        \n",
    "word_count = count_words(moby_words)\n",
    "\n",
    "most, least = word_stats(word_count)\n",
    "print(\"Most common words:\")\n",
    "for word in most:\n",
    "    print(word)\n",
    "print(\"\\nLeast common words:\")\n",
    "for word in least:\n",
    "    print(word)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['EmptyStringError',\n",
       " '__builtins__',\n",
       " '__cached__',\n",
       " '__doc__',\n",
       " '__file__',\n",
       " '__loader__',\n",
       " '__name__',\n",
       " '__package__',\n",
       " '__path__',\n",
       " '__spec__',\n",
       " 'clean_line',\n",
       " 'cleaning',\n",
       " 'count_words',\n",
       " 'counter',\n",
       " 'get_words',\n",
       " 'word_stats']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import word_count\n",
    "dir(word_count)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK : CONSIDER THE CHOICES\n",
    "Take a moment to consider what our options are for handling the tasks we’ve identified above. What modules in the standard library can you think of that will do the job? If you want, you can even stop right now and work out the code to do it and compare your solution with the one we’ll develop below.\n",
    "\n",
    "From the standard library, `datetime` for managing the dates/times of the files, and either `os.path` and `os` or `pathlib` for renaming and archiving the files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: POTENTIAL PROBLEMS\n",
    "Since our solution above is very simple there are likely to be many situations that it won’t handle well. What are some potential issues or problems that might arise with the script above? How might you remedy these problems?\n",
    "\n",
    "Multiple files during the same day would be a problem, for one thing. If there are lots of files, it will become increasingly difficult to navigate the archive directory.\n",
    "\n",
    "\n",
    "Consider the naming convention used for the files, which is based on the year, month and name, in that order. What advantages do you see in that? What might be the disadvantages? Can you make any arguments for putting the date string somewhere else in the file name, like the beginning or the very end?\n",
    "\n",
    "Using year-month-day date formats will make a text based sort of the files sort by date as well. Putting the date at the end of the file name, but before the extension will make it more difficult to parse the date element visually.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: IMPLEMENTATION OF MULTIPLE DIRECTORIES\n",
    "Using the code we developed above as a starting point, how would you modify it to implement archiving each set of files in subdirectories named according to the date received? Feel free to take the time to implement the code and test it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "import pathlib\n",
    "\n",
    "FILE_PATTERN = \"*.txt\"\n",
    "ARCHIVE = \"archive\"\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    \n",
    "    date_string = datetime.date.today().strftime(\"%Y-%m-%d\")\n",
    "\n",
    "    cur_path = pathlib.Path(\".\")\n",
    "\n",
    "    new_path = cur_path.joinpath(ARCHIVE, date_string)\n",
    "    new_path.mkdir()                                    #A\n",
    "\n",
    "    paths = cur_path.glob(FILE_PATTERN)\n",
    "\n",
    "    for path in paths:\n",
    "        path.rename(new_path.joinpath(path.name))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: ALTERNATE SOLUTIONS\n",
    "How might you create a script that does the same thing, without using pathlib? What libraries and functions would you use?\n",
    "\n",
    "You would use the `os.path` and `os` libraries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: ARCHIVING TO ZIP FILES PSEUDOCODE\n",
    "Take a moment to write the pseudocode for a solution that would store data files in zip files as shown above. What modules and functions or methods do you intend to use? Try coding your solution to make sure it works.\n",
    "\n",
    "Pseudocode:\n",
    "\n",
    "```\n",
    "create path for zip file\n",
    "create empty zipfile\n",
    "for each file\n",
    "    write into zipfile\n",
    "    remove original file\n",
    "```\n",
    "\n",
    "(see following text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "import pathlib\n",
    "import zipfile\n",
    "\n",
    "FILE_PATTERN = \"*.txt\"\n",
    "ARCHIVE = \"archive\"\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    \n",
    "    date_string = datetime.date.today().strftime(\"%Y-%m-%d\")\n",
    "\n",
    "    cur_path = pathlib.Path(\".\")\n",
    "    paths = cur_path.glob(FILE_PATTERN)\n",
    "\n",
    "    zip_file_path = cur_path.joinpath(ARCHIVE, date_string + \".zip\")\n",
    "    zip_file = zipfile.ZipFile(str(zip_file_path), \"w\")\n",
    "\n",
    "    for path in paths:\n",
    "        zip_file.write(str(path))\n",
    "        path.unlink()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: CONSIDER DIFFERENT PARAMETERS\n",
    "Take some time to consider different grooming options. How would you modify this to keep only one file a month? How would you change it so that files from the previous month and older were groomed to save one a week? (Note: this is not the same as older than 30 days!)\n",
    "\n",
    "You could use something similar to the code above but also check the month of the file against the current month."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 21"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: NORMALIZATION\n",
    "Look closely at the list of words that was generated above. Do you see any issues with the normalization so far? What other issues do you think you might encounter with a longer section of text? How do you think you might deal those issues?\n",
    "\n",
    "Double hyphens for em-dashes, hyphenation for line breaks and otherwise, other punctuation would all be potential problems. \n",
    "\n",
    "Enhancing the word cleaning module created in chapter 18 would be a good way to cover most of the issues."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: READ A FILE\n",
    "Write the code to read a text file (assume it’s the file temp_data_00a.txt as show in the the example above) and split each line of the file into a list of values, and add that list to a single list of records. \n",
    "\n",
    "(no answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What issues or problems did you encounter in implementing this? How might you go about converting the last three fields into the correct date, real, and int types?\n",
    "\n",
    "You could use a list comprehension and explicitly convert those fields."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: HANDLING QUOTING\n",
    "Consider how you would approach the problems of handling quoted fields and embedded delimiter characters if you didn't have the `csv` library. Which is easier to handle, the quoting or the embedded delimiters?\n",
    "\n",
    "Without using the `csv` module you would have to check to see if a field began and ended with the quote characters, and then strip() them off. \n",
    "\n",
    "To handle embedded delimiters without using the `csv` library, you would have to determine quoted fields and treat them differently, then split the rest of the fields o the delimiter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: CLEANING DATA\n",
    "How would you handle the fields with “Missing” as possible values for math calculations? Can you write a snippet of code that would average one of those columns?\n",
    "\n",
    "```\n",
    "clean_field = [float(x[13]) for x in data_rows if x[13] != 'Missing']\n",
    "average = sum(clean_field)/len(clean_field)\n",
    "```\n",
    "\n",
    "What would you do with the average column at the end, so that you could also report the average coverage? In your opinion would the solution to this problem be at all linked to the way that the “Missing” entries were handled?\n",
    "\n",
    "```\n",
    "coverage_values = [float(x[-1].strip(\"%\"))/100]\n",
    "```\n",
    "\n",
    "It might or might not be done at the same time as the \"Missing\" values are handled. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB\n",
    "The file of weather observations provided here is by month and then by county for the state of Illinois from 1979 to 2011. Write the code to process this file extract the data for Chicago (Cook County) into a single CSV  or spreadsheet file. This will include replacing the “Missing” strings with empty strings, and translating the percentage to a decimal. You may also consider what fields are repetitive and can be either omitted or stored elsewhere. The proof that you’ve got it right will be in loading the file into a spreadsheet. You can download a solution with the book’s source code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 22"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: RETRIEVING A FILE\n",
    "If you were working with the data file above and wanted to break each line into separate fields, how might you do that? What other processing would you expect to do? Try writing some code to retrieve this file and calculating the average annual rainfall, or for more of challenge, the average maximum and minimum temperature for each year."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average rainfall = 50.43794749403351 mm\n"
     ]
    }
   ],
   "source": [
    "import requests \n",
    "response = requests.get(\"http://www.metoffice.gov.uk/pub/data/weather/uk/climate/stationdata/heathrowdata.txt\")\n",
    "\n",
    "data = response.text\n",
    "data_rows = []\n",
    "rainfall = []\n",
    "for row in data.split(\"\\r\\n\")[7:]:\n",
    "    fields = [x for x in row.split(\" \") if x]\n",
    "    data_rows.append(fields)\n",
    "    rainfall.append(float(fields[5]))\n",
    "\n",
    "print(\"Average rainfall = {} mm\".format(sum(rainfall)/len(rainfall)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: ACCESSING AN API\n",
    "Write some code to fetch some data from the city of Chicago site used above. Look at the fields mentioned in the results and see if you can select on records based on another field in combination with the date range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{\"arrest\":true,\"beat\":\"0431\",\"block\":\"095XX S BENNETT AVE\",\"case_number\":\"HY110050\",\"community_area\":\"51\",\"date\":\"2015-01-10T12:00:00.000\",\"description\":\"TELEPHONE THREAT\",\"district\":\"004\",\"domestic\":true,\"fbi_code\":\"26\",\"id\":\"9921038\",\"iucr\":\"2820\",\"latitude\":\"41.721683546\",\"location\":{\"type\":\"Point\",\"coordinates\":[-87.57776968,41.721683546]},\"location_description\":\"RESIDENCE\",\"longitude\":\"-87.57776968\",\"primary_type\":\"OTHER OFFENSE\",\"updated_on\":\"2015-08-17T15:03:40.000\",\"ward\":\"7\",\"x_coordinate\":\"1190515\",\"y_coordinate\":\"1842078\",\"year\":\"2015\"}\n",
      ",{\"arrest\":true,\"beat\":\"0833\",\"block\":\"076XX S CICERO AVE\",\"case_number\":\"HY110086\",\"community_area\":\"65\",\"date\":\"2015-01-10T12:12:00.000\",\"description\":\"RETAIL THEFT\",\"district\":\"008\",\"domestic\":false,\"fbi_code\":\"06\",\"id\":\"9921110\",\"iucr\":\"0860\",\"latitude\":\"41.754592961\",\"location\":{\"type\":\"Point\",\"coordinates\":[-87.741528537,41.754592961]},\"location_description\":\"DEPARTMENT STORE\",\"longitude\":\"-87.741528537\",\"primary_type\":\"THEFT\",\"updated_on\":\"2015-08-17T15:03:40.000\",\"ward\":\"13\",\"x_coordinate\":\"1145727\",\"y_coordinate\":\"1853720\",\"year\":\"2015\"}\n",
      ",{\"arrest\":true,\"beat\":\"0832\",\"block\":\"024XX W 66TH ST\",\"case_number\":\"HY110078\",\"community_area\":\"66\",\"date\":\"2015-01-10T12:15:00.000\",\"description\":\"RECKLESS CONDUCT\",\"district\":\"008\",\"domestic\":false,\"fbi_code\":\"24\",\"id\":\"9921105\",\"iucr\":\"0470\",\"latitude\":\"41.773786968\",\"location\":{\"type\":\"Point\",\"coordinates\":[-87.684173165,41.773786968]},\"location_description\":\"SIDEWALK\",\"longitude\":\"-87.684173165\",\"primary_type\":\"PUBLIC PEACE VIOLATION\",\"updated_on\":\"2015-08-17T15:03:40.000\",\"ward\":\"15\",\"x_coordinate\":\"1161324\",\"y_coordinate\":\"1860827\",\"year\":\"2015\"}\n",
      ",{\"arrest\":true,\"beat\":\"1533\",\"block\":\"003XX S CICERO AVE\",\"case_number\":\"HY110081\",\"community_area\":\"25\",\"date\":\"2015-01-10T12:25:00.000\",\"description\":\"ATTEMPT: AGGRAVATED\",\"district\":\"015\",\"domestic\":false,\"fbi_code\":\"03\",\"id\":\"9921156\",\"iucr\":\"0331\",\"latitude\":\"41.876581037\",\"location\":{\"type\":\"Point\",\"coordinates\":[-87.745137513,41.876581037]},\"location_description\":\"GAS STATION\",\"longitude\":\"-87.745137513\",\"primary_type\":\"ROBBERY\",\"updated_on\":\"2015-08-17T15:03:40.000\",\"ward\":\"24\",\"x_coordinate\":\"1144438\",\"y_coordinate\":\"1898165\",\"year\":\"2015\"}\n",
      ",{\"arrest\":true,\"beat\":\"1511\",\"block\":\"009XX N CENTRAL AVE\",\"case_number\":\"HY110108\",\"community_area\":\"25\",\"date\":\"2015-01-10T12:30:00.000\",\"description\":\"FORCIBLE ENTRY\",\"district\":\"015\",\"domestic\":false,\"fbi_code\":\"05\",\"id\":\"9921155\",\"iucr\":\"0610\",\"latitude\":\"41.897833444\",\"location\":{\"type\":\"Point\",\"coordinates\":[-87.765500945,41.897833444]},\"location_description\":\"SCHOOL, PUBLIC, BUILDING\",\"longitude\":\"-87.765500945\",\"primary_type\":\"BURGLARY\",\"updated_on\":\"2015-08-17T15:03:40.000\",\"ward\":\"29\",\"x_coordinate\":\"1138841\",\"y_coordinate\":\"1905872\",\"year\":\"2015\"}\n",
      ",{\"arrest\":true,\"beat\":\"2422\",\"block\":\"017XX W HOWARD ST\",\"case_number\":\"HY110092\",\"community_area\":\"1\",\"date\":\"2015-01-10T12:35:00.000\",\"description\":\"RETAIL THEFT\",\"district\":\"024\",\"domestic\":false,\"fbi_code\":\"06\",\"id\":\"9921128\",\"iucr\":\"0860\",\"latitude\":\"42.019399237\",\"location\":{\"type\":\"Point\",\"coordinates\":[-87.675049485,42.019399237]},\"location_description\":\"GROCERY FOOD STORE\",\"longitude\":\"-87.675049485\",\"primary_type\":\"THEFT\",\"updated_on\":\"2015-08-17T15:03:40.000\",\"ward\":\"49\",\"x_coordinate\":\"1163126\",\"y_coordinate\":\"1950347\",\"year\":\"2015\"}\n",
      ",{\"arrest\":true,\"beat\":\"1012\",\"block\":\"015XX S KOSTNER AVE\",\"case_number\":\"HY110094\",\"community_area\":\"29\",\"date\":\"2015-01-10T12:43:00.000\",\"description\":\"POSS: HEROIN(WHITE)\",\"district\":\"010\",\"domestic\":false,\"fbi_code\":\"18\",\"id\":\"9921091\",\"iucr\":\"2024\",\"latitude\":\"41.859894538\",\"location\":{\"type\":\"Point\",\"coordinates\":[-87.734769543,41.859894538]},\"location_description\":\"RESIDENTIAL YARD (FRONT/BACK)\",\"longitude\":\"-87.734769543\",\"primary_type\":\"NARCOTICS\",\"updated_on\":\"2015-08-17T15:03:40.000\",\"ward\":\"24\",\"x_coordinate\":\"1147304\",\"y_coordinate\":\"1892104\",\"year\":\"2015\"}\n",
      ",{\"arrest\":true,\"beat\":\"1113\",\"block\":\"0000X N KOSTNER AVE\",\"case_number\":\"HY110112\",\"community_area\":\"26\",\"date\":\"2015-01-10T12:47:00.000\",\"description\":\"POSS: CANNABIS 30GMS OR LESS\",\"district\":\"011\",\"domestic\":false,\"fbi_code\":\"18\",\"id\":\"9921127\",\"iucr\":\"1811\",\"latitude\":\"41.881009839\",\"location\":{\"type\":\"Point\",\"coordinates\":[-87.735490558,41.881009839]},\"location_description\":\"SIDEWALK\",\"longitude\":\"-87.735490558\",\"primary_type\":\"NARCOTICS\",\"updated_on\":\"2015-08-17T15:03:40.000\",\"ward\":\"28\",\"x_coordinate\":\"1147054\",\"y_coordinate\":\"1899797\",\"year\":\"2015\"}]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import requests \n",
    "response = requests.get(\"https://data.cityofchicago.org/resource/6zsd-86xi.json?$where=date between '2015-01-10T12:00:00' and '2015-01-10T13:00:00'&arrest=true\")\n",
    "\n",
    "print(response.text)\n",
    "                        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: SAVING SOME JSON CRIME DATA\n",
    "Modify the code you wrote to fetch Chicago crime data in section 22.2 above to convert the fetched data from a JSON formatted string to Python object. Then see if you can save the crime events both as a series of separate JSON objects in one fine and as one JSON object in another file. Then see what code is needed to load each file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import requests \n",
    "\n",
    "response = requests.get(\"https://data.cityofchicago.org/resource/6zsd-86xi.json?$where=date between '2015-01-10T12:00:00' and '2015-01-10T13:00:00'&arrest=true\")\n",
    "\n",
    "crime_data = json.loads(response.text)\n",
    "\n",
    "with open(\"crime_all.json\", \"w\") as outfile:\n",
    "    json.dump(crime_data, outfile)\n",
    "\n",
    "with open(\"crime_series.json\", \"w\") as outfile:\n",
    "    for record in crime_data:\n",
    "        json.dump(record, outfile)\n",
    "        outfile.write(\"\\n\")\n",
    "\n",
    "with open(\"crime_all.json\") as infile:\n",
    "    crime_data_2 = json.load(infile)\n",
    "\n",
    "crime_data_3 = []\n",
    "with open(\"crime_series.json\") as infile:\n",
    "    for line in infile:\n",
    "        crime_data_3 = json.loads(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: FETCHING AND PARSING XML\n",
    "Write the code to pull the Chicago XML weather forecast from https://graphical.weather.gov/xml/SOAP_server/ndfdXMLclient.php?whichClient=NDFDgen&lat=41.87&lon=+-87.65&product=glance. The use xmltodict to parse the xml into a Python dictionary and extract tomorrow’s fordeecast maximum temperature. Hint: to match up time layouts and values, compare the layout-key value of the first time-layout section and the time-layout attribute of the temperature element of the parameters element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import xmltodict\n",
    "\n",
    "response = requests.get(\"https://graphical.weather.gov/xml/SOAP_server/ndfdXMLclient.php?whichClient=NDFDgen&lat=41.87&lon=+-87.65&product=glance\")\n",
    "\n",
    "parsed_dict = xmltodict.parse(response.text)\n",
    "layout_key = parsed_dict['dwml']['data']['time-layout'][0]['layout-key']\n",
    "forecast_temp = parsed_dict['dwml']['data']['parameters']['temperature'][0]['value'][0]\n",
    "print(layout_key)\n",
    "print(forecast_temp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: PARSING HTML\n",
    "Given the file forecast.html (which can also be found with the code on this book’s web site), write a script using Beautiful Soup that will extract the data and save it as a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('Tonight', 'A slight chance of showers and thunderstorms before 10pm. Mostly cloudy, with a low around 66. West southwest wind around 9 mph. Chance of precipitation is 20%. New rainfall amounts between a tenth and quarter of an inch possible.'), ('Friday', 'Partly sunny. High near 77, with temperatures falling to around 75 in the afternoon. Northwest wind 7 to 12 mph, with gusts as high as 18 mph.'), ('Friday Night', 'Mostly cloudy, with a low around 63. North wind 7 to 10 mph.'), ('Saturday', 'Mostly sunny, with a high near 73. North wind around 10 mph.'), ('Saturday Night', 'Partly cloudy, with a low around 63. North wind 5 to 10 mph.'), ('Sunday', 'Mostly sunny, with a high near 73.'), ('Sunday Night', 'Mostly cloudy, with a low around 64.'), ('Monday', 'Mostly sunny, with a high near 74.'), ('Monday Night', 'Mostly clear, with a low around 65.'), ('Tuesday', 'Sunny, with a high near 75.'), ('Tuesday Night', 'Mostly clear, with a low around 65.'), ('Wednesday', 'Sunny, with a high near 77.'), ('Wednesday Night', 'Mostly clear, with a low around 67.'), ('Thursday', 'A chance of rain showers after 1pm. Mostly sunny, with a high near 81. Chance of precipitation is 30%.')]\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "import bs4\n",
    "\n",
    "def read_html(filename):\n",
    "    with open(filename) as html_file:\n",
    "        html = html_file.read()\n",
    "        return html\n",
    "\n",
    "\n",
    "def parse_html(html):\n",
    "    bs = bs4.BeautifulSoup(html, \"html.parser\")\n",
    "    labels = [x.text for x in bs.select(\".forecast-label\")]\n",
    "    forecasts = [x.text for x in bs.select(\".forecast-text\")]\n",
    "    \n",
    "    return list(zip(labels, forecasts))\n",
    "    \n",
    "def write_to_csv(data, outfilename):\n",
    "    csv.writer(open(outfilename, \"w\")).writerows(data)\n",
    "        \n",
    "if __name__ == '__main__':\n",
    "    html = read_html(\"forecast.html\")\n",
    "    values = parse_html(html)\n",
    "    write_to_csv(values, \"forecast.csv\")\n",
    "    print(values)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### LAB: TRACK CURIOSITY’S WEATHER\n",
    "Use the API described above in section 22.2 to gather a weather history of Curiosity’s stay on Mars for a month. Transform the data so that you can load it into a spreadsheet and graph it.  For a version of this project see the book’s source code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import csv\n",
    "import requests\n",
    "\n",
    "for sol in range(1830, 1863):\n",
    "    response = requests.get(\"http://marsweather.ingenology.com/v1/archive/?sol={}&format=json\".format(sol))\n",
    "    result = json.loads(response.text)\n",
    "    if not result['count']:\n",
    "        continue\n",
    "    weather = result['results'][0]\n",
    "    print(weather)\n",
    "    csv.DictWriter(open(\"mars_weather.csv\", \"a\"), list(weather.keys())).writerow(weather)\n",
    "    \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 23"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: CREATING AND MODIFYING TABLES\n",
    "Using sqlite3 write the code that creates a database table for the Illinois weather data we loaded from a flat file in section 21.2. Suppose that we had similar data for more states, and wanted to store store more information about the states themselves – how could you modify your database to use a related table to store the state information?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sqlite3\n",
    "conn = sqlite3.connect(\"datafile.db\")\n",
    "\n",
    "cursor = conn.cursor()\n",
    "\n",
    "cursor.execute(\"\"\"create table weather (id integer primary key, state text, state_code text,\n",
    "              year_text text, year_code text, avg_max_temp real,  max_temp_count integer, \n",
    "              max_temp_low real, max_temp_high real,\n",
    "              avg_min_temp real, min_temp_count integer,\n",
    "              min_temp_low real, min_temp_high real,\n",
    "              heat_index real, heat_index_count integer, \n",
    "              heat_index_low real, heat_index_high real,\n",
    "              heat_index_coverage text)\n",
    "              \"\"\")\n",
    "conn.commit()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You could add a state table and only store each state's ID field in the weather database."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: USING AN ORM\n",
    "Using the database from section 22.3 above, write a SQLAlchemy class to map to the data table and use it to read the records from the table. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sqlalchemy import create_engine, select, MetaData, Table, Column, Integer, String, Float\n",
    "from sqlalchemy.orm import sessionmaker\n",
    "\n",
    "dbPath = 'datafile.db'\n",
    "engine = create_engine('sqlite:///%s' % dbPath)\n",
    "metadata = MetaData(engine)\n",
    "weather  = Table('weather', metadata, \n",
    "                Column('id', Integer, primary_key=True),\n",
    "                Column(\"state\", String),\n",
    "                Column(\"state_code\", String),\n",
    "                Column(\"year_text\", String ),\n",
    "                Column(\"year_code\", String), \n",
    "                Column(\"avg_max_temp\", Float),\n",
    "                Column(\"max_temp_count\", Integer),\n",
    "                Column(\"max_temp_low\", Float),\n",
    "                Column(\"max_temp_high\", Float),\n",
    "                Column(\"avg_min_temp\", Float), \n",
    "                Column(\"min_temp_count\", Integer),\n",
    "                Column(\"min_temp_low\", Float), \n",
    "                Column(\"min_temp_high\", Float),\n",
    "                Column(\"heat_index\", Float), \n",
    "                Column(\"heat_index_count\", Integer),\n",
    "                Column(\"heat_index_low\", Float), \n",
    "                Column(\"heat_index_high\", Float),\n",
    "                Column(\"heat_index_coverage\", String)\n",
    "                )\n",
    "Session = sessionmaker(bind=engine)\n",
    "session = Session()\n",
    "result = session.execute(select([weather]))\n",
    "for row in result:\n",
    "    print(row)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: MODIFYING A DATABASE WITH ALEMBIC\n",
    "Experiment with creating an alembic upgrade that adds a state table to your database, with columns for id, state name, and abbreviation. Upgrade and downgrade. What other changes would be needed if you were going to use the state table along with the existing data table?\n",
    "\n",
    "You would need to create a table or perhaps a field relating states to the other tables, and populate it with state data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK : USES OF KEY:VALUE STORES\n",
    "What sorts of data and applications would benefit most from a key:value store like Redis? \n",
    "\n",
    "* Quick look up of data\n",
    "* Caching"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: USES OF MONGODB\n",
    "Thinking back over the various data samples we’ve seen so far and other types of data in your experience, can you come up with any that you think would be well suited to being stored in a database like MongoDB? Are there others that would clearly not be suited, and if so, why not? \n",
    "\n",
    "Data that comes in large and/or more loosely organized \"chunks\" is suited to MondoDB. For example the contents of a web page, or document, etc.\n",
    "\n",
    "Data with a specific structure is better suited to relational data. The weather data we've seen is a good example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LAB: CREATE A DATABASE\n",
    "Choose one of the datasets we’ve discussed in the past few chapters, and decide which type of database would be best to store that data. Create that database and write the code to load the data into it. Then choose the 2 most common and/or likely types of search criteria and write the code to retrieve both single and multiple matching records.\n",
    "\n",
    "(no answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter 24"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: USING JUPYTER NOTEBOOK\n",
    "Enter some code in the notebook and experiment with running it. Check out the Edit, Cell, and Kernel menus to see what options are there. Once you have a little code running, use the Kernel menu to restart the kernel, then repeat your steps,  then use the cell menu to rerun the code in all of the cells.\n",
    "\n",
    "(no answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: CLEANING DATA WITH AND WITHOUT PANDAS\n",
    "Experiment with the operations mentioned above. Once the final column has been converted to a fraction, can you think of a way to convert it back to string with the trailing percent sign?\n",
    "\n",
    "```\n",
    "temp[17] = temp[17].mul(100)\n",
    "temp[17] = temp[17].astype(str) + '%'\n",
    "```\n",
    "\n",
    "For contrast, load the same data into a plain Python list of using the csv module, and apply the same changes using just plain Python.  \n",
    "\n",
    "Use list comprehensions. So to convert the last column from a string to a fraction, use something like:\n",
    "```\n",
    "data = [x[:-1]+[float(x[-1].strip(\"%\"))*100] for x in data]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: MERGING DATA SETS\n",
    "How would you go about actually merging to data sets like the above in Python?\n",
    "\n",
    "If you were sure you had exactly the same number of items in each set and that they were in the right order, you could use the `zip()` function. Otherwise, you could create a dictionary with the keys being something common between the two data sets and then append the date by key from both sets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUICK CHECK: SELECTING IN PYTHON\n",
    "What Python code structure would you use to select only rows meeting certain conditions? \n",
    "\n",
    "Probably a list comprehension:\n",
    "```\n",
    "selected = [x for x in old_list if <x meets selection criteria>]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: GROUPING AND AGGREGATING\n",
    "Experiment with pandas and the data above. Can you get the calls and amounts by both team member and month?\n",
    "\n",
    "```\n",
    "calls_revenue[['Team member','Month', 'Calls', 'Amount']].groupby(['Team member','Month']).sum())\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TRY THIS: PLOTTING\n",
    "Plot a line graph of the monthly average amount per call. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f5bc9b1d588>"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEKCAYAAAD6q1UVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd8FHX+x/HXJ51AIAFCL8HQIdRQ\nFKToeSCgiOUEpSmKIipYT/R3Z7nz9M47RfRQaUoTURQFQREFRPQoCb0TIDSRBEhoISHl+/tjJxgi\nmN1kk9nyeT4eeWR2dnb2Mwz5ZDLznfeKMQallFL+JcDuApRSSpU9bf5KKeWHtPkrpZQf0uavlFJ+\nSJu/Ukr5IW3+Sinlh7T5K6WUH3Kq+YtIsohsEZGNIpJQYP4jIrJTRLaJyL8KzB8nIkkisktEepVG\n4UoppYovyIVlexpjjuc/EJGeQH+gtTEmS0SqWfObAwOBFkAt4FsRaWyMyXVj3UoppUrAleZf2Cjg\nVWNMFoAxJsWa3x/4yJq/X0SSgI7A/660oqpVq5qYmJgSlKKUUv4nMTHxuDEmujivdbb5G+AbETHA\ne8aYSUBj4FoReRnIBJ40xqwDagOrC7z2sDXvimJiYkhISPi9RZRSShUiIgeK+1pnm39XY8wR69TO\nUhHZab22MtAZ6AB8LCJXOfvGIjISGAlQr14916pWSilVIk5d8DXGHLG+pwDzcZzGOQx8ZhzWAnlA\nVeAIULfAy+tY8wqvc5IxJt4YEx8dXay/WpRSShVTkc1fRMqLSET+NPBHYCvwOdDTmt8YCAGOAwuA\ngSISKiINgEbA2tIpXymlVHE4c9qnOjBfRPKX/9AY87WIhADTRGQrcAEYZhz50NtE5GNgO5ADjNaR\nPkp5p+zsbA4fPkxmZqbdpfi1sLAw6tSpQ3BwsNvWKZ6Q5x8fH2/0gq9Snmf//v1ERERQpUoVrANA\nVcaMMZw4cYIzZ87QoEGDS54TkURjTHxx1qt3+CqlrigzM1Mbv81EhCpVqrj9ry9t/kqp36WN336l\nsQ+0+asSWbjpZ7YeOWV3GUopF5XkDl/l5z5cc5Bn528hMEB4uGdDHr6uIcGBejyhlDfQn1RVLD/t\nPc5fv9hKt8bR3Ny6Fm9+t4dbJ/5EUsoZu0tTPuiXX35h4MCBxMbG0r59e/r06cPu3buvuHyFChUA\nSE5OpmXLlkWuf+zYsdSuXZu8vDy31eyq9PR0Jk6cWGbvp81fuWz/8XOMmrWemKrlefuutrxxZxsm\n3t2Ow2kZ9Jmwiqmr9pOXZ/8oMuUbjDEMGDCAHj16sHfvXhITE3nllVc4duyYW9afl5fH/PnzqVu3\nLt9//71b1lkcZd389bSPcsmp89mMmL6OAIGpw+KpGOYYd9wnribxMVGM+3QLf/tyO0u3/8K/72hN\nnahwmytW7vLiwm1s//m0W9fZvFZFnr+pxe8us3z5coKDg3nwwQcvzmvdujVnz57l+uuvJy0tjezs\nbP7+97/Tv39/l2tYsWIFLVq04M4772TOnDn07NkTgBdeeIH9+/ezb98+Dh48yBtvvMHq1av56quv\nqF27NgsXLiQ4OJjvvvuOJ598kpycHDp06MA777xDaGjoxcyyqlWrkpCQwJNPPsmKFSt44YUXOHjw\n4MX1jh07lkcffZRnnnmGvXv30qZNG2644QZee+01l7fFFXrkr5yWk5vHwx+u59DJDN4d3J76Vcpf\n8ny1iDCmDIvnn7fFseXwKXqP/4FPEg7hCfeSKO+1detW2rdv/5v5YWFhzJ8/n/Xr17N8+XKeeOKJ\nYv1fmzNnDoMGDWLAgAEsWrSI7Ozsi8/t3buXZcuWsWDBAgYPHkzPnj3ZsmUL5cqVY9GiRWRmZjJ8\n+HDmzp3Lli1byMnJ4Z133inyPXfu3MmSJUtYu3YtL774ItnZ2bz66qvExsaycePGUm/8oEf+ygUv\nfbmdH/Yc51+3taLTVVUuu4yIcGeHelwTW5UnPt7EU/M28832Y7xyaxxVK4SWccXKnYo6Qi9rxhie\nffZZVq5cSUBAAEeOHOHYsWPUqFHD6XVcuHCBxYsX8/rrrxMREUGnTp1YsmQJ/fr1A+DGG28kODiY\nuLg4cnNz6d27NwBxcXEkJyeza9cuGjRoQOPGjQEYNmwY//3vfxk7duzvvm/fvn0JDQ0lNDSUatWq\nue0Uliv0yF85Zcb/kpnxvwOM7HYVf+pQt8jl61YOZ87IzjzXpxnf70ql1xsrWbLtl9IvVPmcFi1a\nkJiY+Jv5s2fPJjU1lcTERDZu3Ej16tVdvhFqyZIlpKenExcXR0xMDKtWrWLOnDkXnw8NdRywBAQE\nEBwcfHG8fUBAADk5Ob+77qCgoIsXkAvXlb9egMDAwCLXVRq0+asi/bAnlRcXbuf6ptX4c++mTr8u\nMEC4v9tVLHykK9UrhvHAzESe+HgTpzOzi36xUpbrrruOrKwsJk2adHHe5s2bOXDgANWqVSM4OJjl\ny5dz4IDr0fZz5sxhypQpJCcnk5yczP79+1m6dCkZGRlOvb5JkyYkJyeTlJQEwMyZM+nevTvg+JyS\n/F9an376aZHrioiI4MyZshstp81f/a6klLM8NHs9japV4M1BbQkMcP1OwyY1Ivh8dBceua4h8zcc\n5sbxP/DT3uNFv1ApHKcS58+fz7fffktsbCwtWrRg3Lhx9OnTh4SEBOLi4pgxYwZNmzp/YAKQkZHB\n119/Td++fS/OK1++PF27dmXhwoVOrSMsLIz333+fO+64g7i4OAICAi5emH7++ecZM2YM8fHxBAYG\nFrmuKlWq0KVLF1q2bMlTTz3l0rYUhwa7qStKO3eBWyb+yLmsHD4f3cUtI3fWH0zjiY83sf/4Oe7p\nEsOfezclLLjoHwxljx07dtCsWTO7y1Bcfl9osJtyuws5eYyancjR9EzeG9LebUM229WLYtGjXRl6\ndX3e/zGZvhN+YPPhdLesWynlPG3+6jeMMTy/YCur953kn7fH0b5+ZbeuPzwkiJf6t2TGvR05l5XL\ngIk/Mf7b3WTn2nd3pfJtS5YsoU2bNpd8DRgwwO6ybKVDPdVvTPsxmTlrDzG6ZywD2tYptffp1jia\nJWO78fyCrYz/dg/Ldqbw+p9a07BaRKm9p3KdMcbrkz179epFr1697C6j2Erj9Lwe+atLLN+ZwsuL\nttOrRXWeuKFJqb9fpfBgxg9sy8S723HoZAZ9J6ximsZDeIywsDBOnDihN+rZKP/DXMLCwty6Xr3g\nqy7afewMt078iXqVw5k36mrCQ8r2D8OU05k889kWlu1M4eqrqvDvP7WmdmS5Mq1BXUo/xtEzXOlj\nHEtywVebvwLgxNksbpn4I5nZeXwxugu1bGq6xhjmrjvE377cToAIz9/cgtva1fb60w5KlYZSH+0j\nIskiskVENopIQqHnnhARIyJVrcciIhNEJElENotIu+IUpspOVk4uD85KJOV0FpOHxtvW+MExpntg\nx3p8NaYbzWpW5MlPNvHAzESOn82yrSalfJEr5/x7GmPaFPwtIyJ1gT8CBwssdyPQyPoaCRSdcqRs\nY4zhuflbWZecxr/vaE2bupF2lwRAvSqOeIhn+zRlhRUP8Y3GQyjlNiW94PsG8DRQ8NxRf2CGcVgN\nRIpIzRK+jyolk1buY17iYcZc34ibWteyu5xLBAYII7vFXoyHGDkzkSc/0XgIpdzB2eZvgG9EJFFE\nRgKISH/giDFmU6FlawOHCjw+bM1THmbp9mO8+vVO+raqyZjrG9ldzhXlx0M83LMhn63XeAil3MHZ\n5t/VGNMOxymd0SLSDXgW+Gtx31hERopIgogkpKamFnc1qph2HD3NmI82EFe7Ev++vTUBxcjsKUsh\nQQE82asJ80ZdQ0hQAHdNXsNLC7eTmZ1rd2lKeSWnmr8x5oj1PQWYD3QHGgCbRCQZqAOsF5EawBGg\nYOZvHWte4XVOMsbEG2Pio6OjS7QRyjWpZ7K4b3oCFcOCmTw0nnIh3pOtUzAeYtqP+zUeQqliKrL5\ni0h5EYnIn8ZxgXedMaaaMSbGGBOD49ROO2PML8ACYKg16qczcMoYc7T0NkG5IjM7l5EzEzhxLosp\nw+KpXtG9N46UBY2HUKrknDnyrw6sEpFNwFpgkTHm699ZfjGwD0gCJgMPlbhK5RbGGJ75dDMbDqbz\nxp/a0LJ2JbtLKpH8eIibWtVk/Ld7uO2dn0hKOWt3WUp5Bb3Jy4/8d3kSry3ZxZN/bMzD13nuBd7i\nWLT5KM99voXzF3L5c++mDL8mxuOvYyhVUhrprIr09dajvLZkF7e0qcXong3tLsft+raqyTdju3FN\nbBVe+nI7g6eu4Uj6ebvLUspjafP3A1uPnOKxuZtoWy+SV29r5bNRCdUqhjFteAdeuTWOTYfS6f3G\nSj5NPKyhZEpdhjZ/H3fsdCb3TU8gKjyYSUPiff5Ts0SEQVY8RNOaETzxySYenJXICY2HUOoS2vx9\n2PkLudw/I4HTmdlMGdaB6IhQu0sqM/WqhPPRyKsZd2NTlu9Mpdd4jYdQqiBt/j7KGMOT8zax5cgp\n3hzYlua1KtpdUpkLDBAe6B7Lgke6EB3hiId46pNNnNF4CKW0+fuq8d/uYdHmozzTuyk3NK9udzm2\nalqjIl+M7sLonrF8uv4wvcf/wP/2nrC7LKVspc3fBy3Y9DNvfreH29vXYWS3q+wuxyOEBAXwVK+m\nfPLgNQQHCoMmr+ZvX2o8hPJf2vx9zMZD6Tz1ySY6xETx8oCWPjuyp7ja149i8ZhrGdK5PlNX7aff\nW6vYcviU3WUpVea0+fuQo6fOc/+MBKpVDOXdwe0JDfLtkT3FFR4SxN9uacn0eztyJjObARN/5M1v\n92g8hPIr2vx9RMaFHO6bnsD5C7lMHdaBKhX8Z2RPcXVvHM03Y7vTt1VN3vh2N7drPITyI9r8fUBe\nnuGxuRvZcfQ0b93VlsbVI+wuyWtUCg/mzYFtefuuthw4mUHfCT/w/o/7ycvTG8OUb9Pm7wP+s3QX\nS7Yd47m+zenZpJrd5Xilfq1qXYyHeHGhxkMo36fN38vN33CY/y7fy6COdbm3S4zd5Xi1gvEQG614\niM/WazyE8k3a/L1Y4oE0/jxvC52vqsxL/XVkjzvkx0N8bcVDPP6xxkMo36TN30sdTsvggZkJ1IoM\n45272xMcqLvSnS4XD7F0+zG7y1LKbbRjeKGzWY6RPVk5eUwZ1oGo8iF2l+STCsdD3D8jgafnaTyE\n8g3a/L1Mbp5hzJwN7Ek5y8S729GwWgW7S/J5BeMh5iVqPITyDdr8vcy/vt7JdztTeOGm5lzbSD/4\nvqxoPITyNdr8vcjHCYd4b+U+hl5dnyFXx9hdjl/Kj4cY3LkeU1ft5yaNh1BeyqnmLyLJIrJFRDaK\nSII17zUR2Skim0VkvohEFlh+nIgkicguEelVWsX7kzX7TvDc/C10bViVv/Zrbnc5fi08JIi/3xLH\n9Hs7ctqKh5jw3R5yNB5CeRFXjvx7GmPaFPiw4KVAS2NMK2A3MA5ARJoDA4EWQG9goohoyEwJHDyR\nwYOzEqlbOZz/3tWOIB3Z4xEKxkO8vnQ3t737P/amajyE8g7F7iLGmG+MMTnWw9VAHWu6P/CRMSbL\nGLMfSAI6lqxM/3U6M5sR09eRZ2DqsA5UCg+2uyRVwCXxECfO0efNH/hA4yGUF3C2+RvgGxFJFJGR\nl3n+XuAra7o2cKjAc4eteZcQkZEikiAiCampqa7U7DdycvN45MMN7D9+jncGt6NB1fJ2l6SuoF+r\nWiwZ242rY6vwwsLtDJm2hp81HkJ5MGebf1djTDvgRmC0iHTLf0JEngNygNmuvLExZpIxJt4YEx8d\nraNWLuflxTv4fncqf7ulJdfEVrW7HFWE6hXDeH94B/4xII4NB9PpNV7jIZTncqr5G2OOWN9TgPlY\np3FEZDjQD7jb/Po//AhQt8DL61jzlAtmrznA+z8mc2+XBgzqWM/ucpSTRIS7OtXjqzHX0qS6Ix5i\n1Kz1Gg+hPE6RzV9EyotIRP408Edgq4j0Bp4GbjbGZBR4yQJgoIiEikgDoBGw1v2l+66fko7z/Bfb\n6NEkmuf6NrO7HFUM9auUZ+4DV/PMjU1ZtjNF4yGUx3HmyL86sEpENuFo4ouMMV8DbwMRwFJrCOi7\nAMaYbcDHwHbga2C0MUbvhHHS/uPnGDV7PQ2qluetQW0JDNCwNm8VGCA82D2WLx7uQtUKoRoPoTyK\neML5yPj4eJOQkGB3GbY7leEYM55+PpvPH+pCvSrhdpek3CQrJ5c3v93Du9/vpWalcvznT63pfFUV\nu8tSXk5EEgsMv3eJDhj3ENm5eTz0YSKH0jJ4d3B7bfw+JjQokKd7N+WTB68myIqH+LvGQygbafP3\nEC8t3M6PSSf4x4A4OjaobHc5qpS0r1+ZxY9ey92d6jHFiofYekTjIVTZ0+bvAWb8L5mZqw/wQPer\nuCO+bpHLK+9WPtQRD/HBPR04nZnNLf/9kbc0HkKVMW3+Nlu5O5UXF27nD82q83SvpnaXo8pQjybV\nWDK2G33iavIfjYdQZUybv42SUs4w+sP1NKpWgfED2+jIHj8UGR7ChEFteWtQW5KPn6PvBI2HUGVD\nm79N0s5dYMT0BEKDApgyLJ4KoUF2l6RsdFPrWnzzWDc6X+WIhxg6ba3GQ6hSpc3fBhdy8nhwViJH\nT2Xy3pB46kTpyB51aTzE+oNp9Bq/kvkbNB5ClQ5t/mXMGMPzC7ayZv9J/nVbK9rXj7K7JOVBCsdD\nPDZ3Ew/N1ngI5X7a/MvY1FX7mbP2EA/3bMgtbX8TdqoU8Gs8xJ97N+XbHcfoNf4HvtV4COVG2vzL\n0LKdx/jH4h30blGDx29obHc5ysMFBgijesSy4OGuVK0Qwn0zEvjzvM0aD6HcQpt/Gdn1yxkenbOR\n5rUq8vqdrQnQkT3KSc1qVuSLh7vwUI9YPkk8xI1v/sDqfSfsLkt5OW3+ZeDE2SxGTF9HeEggk4fG\nEx6iI3uUa/LjIT5+4GoCAxzxEC8v0ngIVXza/EtZVk4uD8xMJPVMFpOHxlOzUjm7S1JeLD7GEQ9x\nV8d6TP5B4yFU8WnzL0XGGMZ9toWEA2n850+taV030u6SlA8oHxrEywPieP+eDpw6r/EQqni0+Zei\n91bu47P1Rxj7h0b0a1XL7nKUj+nZpBrfPNaNG614iNs1HkK5QJt/Kflm2y/88+ud3NS6FmOub2R3\nOcpHRYaH8NagtkwY1Jb9VjzE9J+SNR5CFUmbfynY9vMpxs7dSKvalXjt9laI6MgeVbputuIhOjWo\nwvMLtmk8hCqSNn83SzmTyf3TE6gYFszkofGEBQfaXZLyE9UrhvHBPR14eUBLEg844iE+33BE4yHU\nZTnV/EUkWUS2WJ/Vm2DNqywiS0Vkj/U9ypovIjJBRJJEZLOItCvNDfAkmdm5jJyRSFpGNlOGxVOt\nYpjdJSk/IyLc3ak+X425lsbVIxg7dyMPzV7PyXMX7C5NeRhXjvx7GmPaFPi8yGeA74wxjYDvrMcA\nNwKNrK+RwDvuKtaTGWP486eb2XgonTfubEPL2pXsLkn5sZiq5fm4QDzEH99YyXc7NB5C/aokp336\nA9Ot6enALQXmzzAOq4FIEalZgvfxCv9dnsQXG3/mqV5N6N2yht3lKHUxHuKL0Y54iBHTE3jm082c\nzcqxuzTlAZxt/gb4RkQSRWSkNa+6MeaoNf0LUN2arg0cKvDaw9Y8n7V4y1H+/c1uBrStzUM9Yu0u\nR6lLNK/liIcY1SOWjxMO0Xv8StZoPITfc7b5dzXGtMNxSme0iHQr+KRxXFFy6aqSiIwUkQQRSUhN\nTXXlpR5ly+FTPP7xRtrVi+SVW+N0ZI/ySKFBgfzZiocIEGGgxkP4PaeavzHmiPU9BZgPdASO5Z/O\nsb6nWIsfAQp+Cnkda17hdU4yxsQbY+Kjo6OLvwU2OnY6k/tmrKNK+VDeG6Ije5Tni4+pzFdjfo2H\n6P/2jxw7nWl3WcoGRTZ/ESkvIhH508Afga3AAmCYtdgw4AtregEw1Br10xk4VeD0kM84fyGX+2ck\ncDYzhynD4omOCLW7JKWcUjAe4nBaBkOnriU9Q0cD+RtnjvyrA6tEZBOwFlhkjPkaeBW4QUT2AH+w\nHgMsBvYBScBk4CG3V22zvDzDk59sYsuRU7w5sC3Nala0uySlXNazSTUmD41n//Fz3PvBOjIu6IVg\nfyKecANIfHy8SUhIsLsMp72+dDcTvtvDs32aMrKbXuBV3u3rrUd5aPZ6ujaKZsrQeEKC9N5PbyEi\niQWG37tE97KLvth4hAnf7eGO9nW4/9qr7C5HqRLr3bImr9wax8rdqTz+8UZyNRfIL+inirhgw8E0\nnpq3mY4NKvPyAB3Zo3zHnR3qkZ6RzStf7SQqPISX+rfQ/98+Tpu/k35OP8/9MxKpXjGUdwe31z+N\nlc95oHssJzMu8N73+4gKD+bxPzaxuyRVirT5O+FcVg4jpieQlZ3LnPs7Ubl8iN0lKVUqnundlPRz\n2UxYlkRkeAj3dm1gd0mqlGjzL0JenuGxuRvZ9ctppg3vQKPqEXaXpFSpERFeHtCS9PMXeOnL7USV\nD2ZA2zp2l6VKgZ67KMK/v9nFN9uP8Zd+zenRpJrd5ShV6oICA3hzYFuuia3Ck59sZtlODYTzRdr8\nf8dn6w8zccVe7upUj+HXxNhdjlJlJiw4kElD42lRqyKjZq1n7f6Tdpek3Eyb/xUkJJ/kmU+3cE1s\nFV68WUc+KP9TITSI94d3oHZUOUZ8sI5tP5+yuyTlRtr8L+PQyQwemJlI7ahyTLy7HcGB+s+k/FOV\nCqHMHNGJCmFBDJu2juTj5+wuSbmJdrVCzmblcN/0BLJz85gyLJ7IcB3Zo/xb7chyzBzRidy8PAZP\nXaNBcD5Cm38BuXmGMXM2kJR6lol3tyc2uoLdJSnlERpWq8AH93Qk7dwFDYLzEdr8C3j1qx18tzOF\nF25uQddGVe0uRymP0rpuJJM0CM5naPO3zF13kMk/7GfY1fUZ0rm+3eUo5ZG6NKzKhEFt2HgonVGz\n1nMhJ8/uklQxafMHVu87wf99vpVrG1XlL/2a212OUh4tPwju+92pPPHJJvI0CM4r+f0dvgdOnGPU\nrETqVQ7n7bvaEaQje5Qq0p0d6pGWkc2rX+0kslywBsF5Ib9u/qczsxkxPQEDTB3WgUrlgu0uSSmv\n8WD3WNLOXeC9lfuIKh/C4zc0trsk5QK/bf45uXk8/OEGko+fY+aITsRULW93SUp5nWdubEpaxgUm\nfLeHqPBg7umiQXDewm+b/98X7WDl7lRevTWOq2Or2F2OUl5JRPjHgDhOnc/mxYXbiQzXIDhv4Zcn\nuGetPsAHPyVzX9cGDOxYz+5ylPJq+UFwV1+lQXDexOnmLyKBIrJBRL60Hl8vIutFZKOIrBKRhtb8\nUBGZKyJJIrJGRGJKp/Ti+SnpOM8v2EbPJtGM69PM7nKU8gmOILj2NK+pQXDewpUj/zHAjgKP3wHu\nNsa0AT4E/s+aPwJIM8Y0BN4A/umOQt1hX+pZHpyVSGx0eSYMaktggI5OUMpdIsKC+eAeKwhu+jq2\n/3za7pLU73Cq+YtIHaAvMKXAbANUtKYrAT9b0/2B6db0POB68YAxYKcyHCN7ggIDmDqsAxFhOrJH\nKXe7GAQXGsTQaWs1CM6DOXvkPx54Gih4O999wGIROQwMAV615tcGDgEYY3KAU8BvrqiKyEgRSRCR\nhNTU1GKW75zs3Dwe+jCRI2nneW9Ie+pWDi/V91PKnzmC4DpqEJyHK7L5i0g/IMUYk1joqceAPsaY\nOsD7wOuuvLExZpIxJt4YEx8dHe3KS11ijOGFBdv4MekE/7g1jg4xlUvtvZRSDg2rRfDBPR05aQXB\nncrItrskVYgzR/5dgJtFJBn4CLhORBYBrY0xa6xl5gLXWNNHgLoAIhKE45TQCXcW7YrpPyUze81B\nHuwey+3tdQiaUmWldd1IJucHwU3XIDhPU2TzN8aMM8bUMcbEAAOBZTjO61cSkfxb+m7g14vBC4Bh\n1vTtwDJjjC3hHyt2pfDSl9u5oXl1nu7VxI4SlPJr+UFwGw6maRCchynWOH/rXP79wKcisgnHOf+n\nrKenAlVEJAl4HHjGHYW6KinlDI98uIEmNSoy/s42BOjIHqVs0btlTf4xQIPgPI1Ld/gaY1YAK6zp\n+cD8yyyTCdzhhtqKLe3cBe79IIHQ4ECmDIunfKjf3sislEcY2NERBPfPr3cSFR6sn4vtAXyuK17I\nyePBWYn8cjqTj0Z2pnZkObtLUkoBD3a/irSMC0xauY+o8BAe0yA4W/lU8zfG8JfPt7Jm/0neHNiG\ndvWi7C5JKWUREcbd2JT0jAu8+d0eIjUIzlY+1fynrtrP3IRDPHJdQ/q3qW13OUqpQvKD4NIzHEFw\nUeEh3NJWf1bt4DPBbt/tOMbLi3fQJ64Gj/1B/5xUylMFBQYwYVBbOl9VmSc/2aRBcDbxiea/65cz\nPDpnAy1rVeI/d+jIHqU8XVhwIJOHxtPMCoJbl6xBcGXN65v/8bNZ3PvBOsqHBjF5aDzlQgLtLkkp\n5YSLQXCR5bj3Aw2CK2te3fyzcnJ5YGYiJ85lMWVYPDUqhdldklLKBVUqhDLzvl+D4A6c0CC4suK1\nzd8Yw7jPtpB4II3/3NGGVnUi7S5JKVUMhYPgUjQIrkx4bfN/9/t9fLb+CI/f0Ji+rWraXY5SqgQa\nVovg/Xs6cuLsBYZO0yC4suCVzX/Jtl/415Kd3NS6Fo9c19DucpRSbtCmbiSThsSzL1WD4MqC1zX/\nrUdOMfajjbSuE8lrt7fSW8SV8iFdG1XlzYEaBFcWvKr5p5zJ5P4ZCUSFBzNpaHvCgnVkj1K+5sa4\nmrxsBcE9qUFwpcZr7vDNzM5l5IxE0jOymTfqaqpF6MgepXzVoI71SMu4wL++3kVUeDAvaBCc23lF\n8zfG8PS8zWw8lM57Q9rTolYlu0tSSpWyUd1jSc/IZtLKfURqEJzbeUXzf2tZEgs2/czTvZvQq0UN\nu8tRSpWB/CC4tHOOILio8GCGaxCc23h881+0+SivL93Nre1qM6p7rN3lKKXKkIjwyq1xpJ/P5oWF\n24nUIDi38egLvpsPp/PEJxtP4X0zAAAPv0lEQVSJrx/FK7fG6Tk/pfxQUGAAbxUIglu+M8XuknyC\nxzb/X045RvZUKR/Ku0PaExqkI3uU8lf5QXBNa0YwanaiBsG5gdPNX0QCRWSDiHxpPRYReVlEdovI\nDhF5tMD8CSKSJCKbRaSdq0Wdv5DL/TMSOJuZw9Th8VStEOrqKpRSPsYRBNeRWpUcQXA7jmoQXEm4\ncuQ/BthR4PFwoC7Q1BjTDPjImn8j0Mj6Ggm840pBeXmGJz7ZyNafTzFhUFua1qjoysuVUj6saoVQ\nZozoSPkQDYIrKaeav4jUAfoCUwrMHgW8ZIzJAzDG5J+I6w/MMA6rgUgRcTp8Z/y3u1m85Ree69OM\n65tVd/ZlSik/UScqnJkjOpKdm8eQqWs1CK6YnD3yHw88DRS81zoWuFNEEkTkKxFpZM2vDRwqsNxh\na16Rvth4hAnLkrgzvi4juuqQLqXU5TWqHsEH93Tk+NksDYIrpiKbv4j0A1KMMYmFngoFMo0x8cBk\nYJorbywiI61fHAmpqalsOJjGU/M207FBZf52S0sd2aOU+l35QXB7U89y7/R1nL+Qa3dJXsWZI/8u\nwM0ikozjvP51IjILxxH9Z9Yy84FW1vQRHNcC8tWx5l3CGDPJGBNvjImPrFyF+2ckUqNiGO8Obk9I\nkMcOQlJKeRBHEFxb1h9MY9TsRLJzNQjOWUV2WWPMOGNMHWNMDDAQWGaMGQx8DvS0FusO7LamFwBD\nrVE/nYFTxpijv/ceyScyyMrOZdrweCqXDynutiil/FCfuJr8Y0AcK3ZpEJwrSnKH76vAbBF5DDgL\n3GfNXwz0AZKADOCeolaUmZ3L23e3o2G1iBKUo5TyVwWD4CLLaRCcM1xq/saYFcAKazodxwigwssY\nYLQr661XOZzujaNdeYlSSl1iVPdY0s5dYPIP+4kqH8LYP2gQ3O/xiGyfSuWC7S5BKeXlRIRn+zQj\nLSOb8d/uISo8hGHXxNhdlsfyiOavlFLuICK8emscp85n8/yCbUSGB9O/jQbBXY4Oq1FK+ZT8ILhO\nDSrzxMcaBHcl2vyVUj4nLDiQKcN+DYJL0CC439Dmr5TySflBcDU1CO6ytPkrpXxW1QqhzBzRkXAr\nCO7giQy7S/IY2vyVUj6tYBDc4KlrNAjOos1fKeXzGlWP4P3hHTQIrgBt/kopv9C2XhTvDWnP3tSz\njNAgOG3+Sin/cW2jaMbf2ZbEg2k85OdBcNr8lVJ+pW+rmrx8SxzL/TwITu/wVUr5nbs6OYLgXlvi\nv0Fw2vyVUn7poR6OILgpq/ZTuXwoY/7QqOgX+RBt/kopv1QwCO6Nb3cTVT6YoVfH2F1WmdHmr5Ty\nWwEBwj9v+zUIrlI5/wmC0wu+Sim/FhQYwNt3taVjjBUEt8s/guC0+Sul/F5YcCCTh8XTpEYEo2Yl\nknjA94PgtPkrpRRQMSyY6fc6guDued/3g+C0+SullMWfguCcbv4iEigiG0Tky0LzJ4jI2QKPQ0Vk\nrogkicgaEYlxX7lKKVW6fhMEd8Y3g+BcOfIfA+woOENE4oGoQsuNANKMMQ2BN4B/lqhCpZQqY5cE\nwU1dy6nzvhcE51TzF5E6QF9gSoF5gcBrwNOFFu8PTLem5wHXi7/dOqeU8nqXBMF94HtBcM4e+Y/H\n0eQLpiA9DCwwxhwttGxt4BCAMSYHOAVUKbxCERkpIgkikpCamupy4UopVdp8OQiuyOYvIv2AFGNM\nYoF5tYA7gLeK+8bGmEnGmHhjTHx0dHRxV6OUUqWqb6ua/P2WlizflcpTPhQE58wdvl2Am0WkDxAG\nVAS2AVlAknVGJ1xEkqzz/EeAusBhEQkCKgEnSqN4pZQqC3d3qk96RrYjCC48hOdvau71QXBFNn9j\nzDhgHICI9ACeNMb0K7iMiJy1Gj/AAmAY8D/gdmCZMcY3flUqpfzWQz1iOXnuAlNX7ScqPMTrg+BK\nI9tnKjBTRJKAk8DAUngPpZQqUyLCc32ake4jQXAuNX9jzApgxWXmVygwnYnjeoBSSvkUXwqC0zt8\nlVLKBflBcB2sILgVXhoEp81fKaVcFBYcyBQrCO5BLw2C0+avlFLFUDgIbucv3hUEp81fKaWKqWqF\nUGbc25FyIYEMnepdQXDa/JVSqgTqVg5n5ohOZOXkMWSa9wTBafNXSqkSalw9gvfv6UDqmSyGTVvn\nFUFw2vyVUsoN2tWL4t3B7UlKOcN90z0/CE6bv1JKuUm3xtG8cWcbEg6kMfrD9R4dBKfNXyml3Khf\nq1r8rX9Llu1M4el5mz02CK404h2UUsqvDe5cn/SMC/z7m91UKhfskUFw2vyVUqoUjO7ZkLSMbKau\n2k/l8iE8er1nBcFp81dKqVKQHwSXlnGB15fuJio8mCEeFASnzV8ppUqJIwiuFafPZ/PXBduoFB7C\nza1r2V0WoBd8lVKqVAUHBvD2Xe3oEFOZx+du9JggOG3+SilVyvKD4BpX95wgOG3+SilVBvKD4GpU\nDPOIIDht/kopVUaiI0KZOaLTxSC4QyftC4LT5q+UUmWobuVwZtzrCIIbPNW+IDinm7+IBIrIBhH5\n0no8W0R2ichWEZkmIsHWfBGRCSKSJCKbRaRdaRWvlFLeqEkNRxBcymn7guBcOfIfA+wo8Hg20BSI\nA8oB91nzbwQaWV8jgXdKXqZSSvmWdvWieHeIfUFwTjV/EakD9AWm5M8zxiw2FmAtUMd6qj8ww3pq\nNRApIjXdXLdSSnm97o2jef1PjiC4h8s4CM7ZI//xwNPAbyqzTvcMAb62ZtUGDhVY5LA1TymlVCE3\ntXYEwX1XxkFwRd7hKyL9gBRjTKKI9LjMIhOBlcaYH1x5YxEZieO0EPXq1XPlpUop5VPsCIJzJt6h\nC3CziPQBwoCKIjLLGDNYRJ4HooEHCix/BKhb4HEda94ljDGTgEkA8fHxnpl5qpRSZWR0z4acPJfN\ntB/3U6V8CI+UchBckad9jDHjjDF1jDExwEBgmdX47wN6AYOMMQVPBy0AhlqjfjoDp4wxR0ujeKWU\n8hUiwv/1bcatbWvzn6W7mbn6QKm+X0mC3d4FDgD/s/48+cwY8xKwGOgDJAEZwD0lLVIppfxBQIDw\nz9tbcTozm79+sZVK5YJLLQjOpeZvjFkBrLCmL/taa/TP6JIWppRS/ig/CG7o1LU8PncjFcOC6NGk\nmtvfR+/wVUopDxMWHMiU4fE0qh7BqFnrSTyQ5vb30OavlFIeqGJYMDPu7Uj1iqHc+8E6dv1yxq3r\n1+avlFIeKj8ILiw4gCFT17g1CE6bv1JKebDCQXCpZ7Lcsl5t/kop5eGa1Ihg2nBHENzQaWvdEgSn\nzV8ppbxA+/q/BsHdPz2BzOySBcFp81dKKS+RHwS37sBJRs9eX6J1leQmL6WUUmXspta1SD+fzV8+\n31qi9eiRv1JKeZkhnevzxA2NS7QObf5KKeWFShr8ps1fKaX8kDZ/pZTyQ9r8lVLKD2nzV0opP6TN\nXyml/JA2f6WU8kPa/JVSyg9p81dKKT8kjk9dtLkIkTPALrvrKEVVgeN2F1GKdPu8ly9vG/j+9jUx\nxkQU54Weku2zyxgTb3cRpUVEEnT7vJcvb58vbxv4x/YV97V62kcppfyQNn+llPJDntL8J9ldQCnT\n7fNuvrx9vrxtoNt3RR5xwVcppVTZ8pQjf6WUUmWozJq/iEwTkRQRuezHz4jDBBFJEpHNItKurGpz\nBye2r4eInBKRjdbXX8u6xpIQkboislxEtovINhEZc5llvHIfOrltXrv/RCRMRNaKyCZr+168zDKh\nIjLX2ndrRCSm7CstHie3b7iIpBbYf/fZUWtJiEigiGwQkS8v85zr+88YUyZfQDegHbD1Cs/3Ab4C\nBOgMrCmr2spo+3oAX9pdZwm2rybQzpqOAHYDzX1hHzq5bV67/6z9UcGaDgbWAJ0LLfMQ8K41PRCY\na3fdbt6+4cDbdtdawu18HPjwcv8Pi7P/yuzI3xizEjj5O4v0B2YYh9VApIjULJvqSs6J7fNqxpij\nxpj11vQZYAdQu9BiXrkPndw2r2Xtj7PWw2Drq/DFvv7AdGt6HnC9iEgZlVgiTm6fVxOROkBfYMoV\nFnF5/3nSOf/awKECjw/jQz+AlqutP02/EpEWdhdTXNaflG1xHGEV5PX78He2Dbx4/1mnDDYCKcBS\nY8wV950xJgc4BVQp2yqLz4ntA7jNOh05T0TqlnGJJTUeeBrIu8LzLu8/T2r+vm49UN8Y0xp4C/jc\n5nqKRUQqAJ8CY40xp+2ux52K2Dav3n/GmFxjTBugDtBRRFraXZM7ObF9C4EYY0wrYCm/HiV7PBHp\nB6QYYxLduV5Pav5HgIK/jetY83yCMeZ0/p+mxpjFQLCIVLW5LJeISDCO5jjbGPPZZRbx2n1Y1Lb5\nwv4DMMakA8uB3oWeurjvRCQIqAScKNvqSu5K22eMOWGMybIeTgHal3VtJdAFuFlEkoGPgOtEZFah\nZVzef57U/BcAQ60RI52BU8aYo3YX5S4iUiP/HJyIdMTxb+81P1xW7VOBHcaY16+wmFfuQ2e2zZv3\nn4hEi0ikNV0OuAHYWWixBcAwa/p2YJmxrh56Ome2r9C1p5txXNfxCsaYccaYOsaYGBwXc5cZYwYX\nWszl/VdmwW4iMgfHiImqInIYeB7HhRmMMe8Ci3GMFkkCMoB7yqo2d3Bi+24HRolIDnAeGOgtP1yW\nLsAQYIt1bhXgWaAeeP0+dGbbvHn/1QSmi0ggjl9aHxtjvhSRl4AEY8wCHL/8ZopIEo6BCwPtK9dl\nzmzfoyJyM5CDY/uG21atm5R0/+kdvkop5Yc86bSPUkqpMqLNXyml/JA2f6WU8kPa/JVSyg9p81dK\nKT+kzV/5BRExBW+MEZEgK+XxNwmJTq4vUkQeKvC4R3HXpZQdtPkrf3EOaGndBASOG4FKcvdxJI4k\nRaW8kjZ/5U8W40hGBBgEzMl/QkQqi8jnVvDXahFpZc1/QRyf1bBCRPaJyKPWS14FYq1s+NeseRWs\n0LCdIjLbW1IxlX/S5q/8yUfAQBEJA1pxaXLni8AGK/jrWWBGgeeaAr2AjsDzVg7QM8BeY0wbY8xT\n1nJtgbFAc+AqHHcOK+WRtPkrv2GM2QzE4DjqX1zo6a7ATGu5ZUAVEaloPbfIGJNljDmOIzK4+hXe\nYq0x5rAxJg/YaL2XUh6pzLJ9lPIQC4B/48hhcjavPqvAdC5X/rlxdjmlbKdH/srfTANeNMZsKTT/\nB+BucIzcAY4X8XkFZ3B85KNSXkmPTJRfMcYcBiZc5qkXgGkishlHIumwyyxTcD0nRORHEdmK43OL\nF7m7VqVKk6Z6KqWUH9LTPkop5Ye0+SullB/S5q+UUn5Im79SSvkhbf5KKeWHtPkrpZQf0uavlFJ+\nSJu/Ukr5of8H9t2CvH4354wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5bc9ae9dd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# see text for these\n",
    "calls = pd.read_csv(\"sales_calls.csv\")\n",
    "revenue = pd.read_csv(\"sales_revenue.csv\")\n",
    "calls_revenue = pd.merge(calls, revenue, on=['Territory', 'Month'])\n",
    "calls_revenue['Call_Amount'] = calls_revenue.Amount/calls_revenue.Calls\n",
    "\n",
    "# plot\n",
    "calls_revenue[['Month', 'Call_Amount']].groupby(['Month']).mean().plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0b2"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
