reading-journal-08.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Day 8 Reading Journal\n",
    "\n",
    "Read _Think Python_, [Chapter 9](http://greenteapress.com/thinkpython2/html/thinkpython2010.html), [13](http://www.greenteapress.com/thinkpython2/html/thinkpython2014.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [Chapter 9](http://greenteapress.com/thinkpython2/html/thinkpython2010.html) Case study: word play\n",
    "\n",
    "Download the word list for this chapter from [http://thinkpython2.com/code/words.txt](http://thinkpython2.com/code/words.txt) and save it in your ReadingJournal folder. You can then run the test code below to verify you've got the word list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Quick test: open 'words.txt' in the current folder and print the first couple entries (\"aa\", \"aah\")\n",
    "fp = open(\"words.txt\")\n",
    "for i in range(2):\n",
    "    word = fp.readline().strip()\n",
    "    print(\"Word #{num}:\".format(num=i), word)\n",
    "fp.close()  # Close the file when we're done with it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's encapsulate this behavior into a helper function to use with the rest of the exercises.\n",
    "\n",
    "In this function we use the Python `with` statement to open the file for us. We'll learn more about exactly what this does when we talk about Exceptions, but for now it's enough to say that it opens the file for us and closes it automatically at the end of the `with` block.\n",
    "\n",
    "If you're interested, you can read more about techniques for [reading and writing files](https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_lines(filename):\n",
    "    \"\"\"\n",
    "    Read all lines from `filename` and return a list of strings, \n",
    "    one per line, with whitespace stripped from the ends.\n",
    "    \"\"\"\n",
    "    lines = []\n",
    "    with open(filename) as fp:\n",
    "        for line in fp:\n",
    "            # Remove whitespace (or do whatever other processing you like)\n",
    "            processed_line = line.strip()\n",
    "            lines.append(processed_line)\n",
    "            \n",
    "    return lines\n",
    "\n",
    "# Test get_lines helper function by reading word list\n",
    "word_list = get_lines(\"words.txt\")\n",
    "print(word_list[78835])\n",
    "            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chapter 9.1\n",
    "\n",
    "Complete the `longer_than` function below and test it by finding long words in the word list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def longer_than(word, threshold):\n",
    "    \"\"\"\n",
    "    Return True if 'word' is longer than 'threshold'\n",
    "    \n",
    "    >>> longer_than(\"python\", 15)\n",
    "    False\n",
    "    >>> longer_than(\"documentation\", 6)\n",
    "    True\n",
    "    \"\"\"\n",
    "    return False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Look for long words by using your helper function in a list comprehension\n",
    "[ word for word in get_lines(\"words.txt\") if longer_than(word, 20) ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chapter 9.4\n",
    "\n",
    "Write a function named `uses_only` that takes a word and a string of letters, and that returns `True` if the word contains only letters in the allowed string. Can you make a sentence using only the letters \"acefhlo\"? Other than “Hoe alfalfa?”\n",
    "\n",
    "_Note_: If you're stuck you may want to try Exercise 2 first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def uses_only(word, allowed):\n",
    "    \"\"\"\n",
    "    Return True if 'word' contains only letters in 'allowed' string, False otherwise.\n",
    "    \n",
    "    >>> uses_only(\"software\", \"code\")\n",
    "    False\n",
    "    >>> uses_only(\"banana\", \"ban\")\n",
    "    True\n",
    "    \"\"\"\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chapter 9.7\n",
    "\n",
    "This question is based on a Puzzler that was broadcast on the radio program [Car Talk](http://www.cartalk.com/content/puzzlers):\n",
    "\n",
    "> Give me a word with three consecutive double letters. I’ll give you a couple of words that almost qualify, but don’t. For example, the word committee, c-o-m-m-i-t-t-e-e. It would be great except for the ‘i’ that sneaks in there. Or Mississippi: M-i-s-s-i-s-s-i-p-p-i. If you could take out those i’s it would work. But there is a word that has three consecutive pairs of letters and to the best of my knowledge this may be the only word. Of course there are probably 500 more but I can only think of one. What is the word? \n",
    "\n",
    "Write a program to find it. Try on your own, but if you get stuck you can review Allen's solution: http://thinkpython2.com/code/cartalk1.py."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [Chapter 13](http://www.greenteapress.com/thinkpython2/html/thinkpython2014.html) Case study: data structure selection\n",
    "\n",
    "The content in this chapter will be helpful for the next mini project on text mining. The exercises build on each other and get more interesting as you go. You should complete as many as your time and interest allows, and bookmark the rest to revisit as you work on MP3.\n",
    "\n",
    " - Section 13.3-4 gives a good example of some techniques for working with files, processing text, and doing some simple analysis. \n",
    " - Section 13.8 and the Markov generation in Exercise 8 can be a lot of fun. \n",
    " - Now that you know a wide range of different data structures, Section 13.9 starts to give some guidance for choosing between them\n",
    " - Section 13.10 explains Allen's \"4 r's\" of debugging strategy\n",
    " \n",
    "This chapter is also excellent preparation for the [Word Frequency Analysis Project Toolbox](https://sd19spring.github.io/toolboxes/word-frequency-analysis) if you're looking for a straightforward toolbox to start on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chapter 13.1\n",
    "\n",
    "Write a function that reads a file and returns its contents split into a list of single word strings. \n",
    "It should break each line into words, strip whitespace and punctuation from the words, and convert them to lowercase.\n",
    "\n",
    "Implementation tips:\n",
    "\n",
    " - Re-use or modify your `get_lines` function\n",
    " - Keep things simple by implementing the transformations step-by-step, possibly using helper functions\n",
    " - Consider using the [string methods](https://docs.python.org/3/library/stdtypes.html#string-methods) `strip`, `replace` and `translate`\n",
    "  ```python\n",
    "  >>> \"banana\".replace('a', 'o')\n",
    "  'bonono'\n",
    "  ```\n",
    " - The string module provides a string named `whitespace`, which contains space, tab, newline, etc., and `punctuation` which contains the punctuation characters.\n",
    " \n",
    "  ```python\n",
    ">>> import string\n",
    ">>> string.punctuation\n",
    "'!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~'\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chapter 13.2  \n",
    "\n",
    "Go to [Project Gutenberg](http://gutenberg.org) and download your favorite out-of-copyright book in plain text format.\n",
    "\n",
    "Use your function from the previous exercise to read the book you downloaded and break it into individual words. \n",
    "_Note_: for best results, skip over the header information at the beginning of the file.\n",
    "\n",
    "Write a program that can count the total number of words in the book, and the number of times each word is used.\n",
    "_Hint_: You can use a modified version of your `histogram` function from Reading Journal 7.\n",
    "\n",
    "For fun, try comparing different books by different authors, written in different eras. Which author uses the most extensive vocabulary? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}