Premium Primitive Examples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Premium Primitive Examples\n",
    "\n",
    "Examples by type\n",
    "\n",
    "* [Geospatial Primitives](#Geospatial-Primitives)\n",
    "* [NLP Primitives](#Natural-Language-Processing-Primitives)\n",
    "* [Date of Birth Primitives](#Date-of-Birth-Primitives)\n",
    "* [Time Primitives](#Time-Primitives)\n",
    "* [Phone Number Primitives](#Phone-Number-Primitives)\n",
    "* [ZIP Code Primitives](#ZIP-Code-Primitives)\n",
    "* [Numeric Primitives](#Numeric-Primitives)\n",
    "* [Miscellaneous Data Types](#Miscellaneous-Data-Types)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Geospatial Primitives\n",
    "\n",
    "`LatLong` variables store a tuple containing the Latitude and Longitude of a point on the globe.  These primitives transform that location (e.g. what country is it in) and can do comparions between multiple `LatLong` variables (e.g. distance between them).\n",
    "\n",
    "#### CityBlockDistance\n",
    "\n",
    "Calculates the distance between points in a city road grid."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    301.518836\n",
       "1    672.088624\n",
       "dtype: float64"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import CityblockDistance\n",
    "\n",
    "cityblock_distance = CityblockDistance()\n",
    "DC = (38, -77)\n",
    "Boston = (43, -71)\n",
    "NYC = (40, -74)\n",
    "cityblock_distance([DC, DC], [NYC, Boston])  # DC -> NYC, DC -> Boston"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### PathLength\n",
    "Determines the length of a path defined by a series of coordinates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "805.5203180792812"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import PathLength\n",
    "\n",
    "path_length_km = PathLength(unit='kilometers')\n",
    "path_length_km([(41.881832, -87.623177), (38.6270, -90.1994), (39.0997, -94.5786)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### LatLongToCity\n",
    "\n",
    "Determines city/town corresponding to given Latitude and Longitude coordinates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        Bayswater\n",
       "1           Cochin\n",
       "2    Mountain View\n",
       "3             None\n",
       "Name: results, dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import LatLongToCity\n",
    "\n",
    "latlong_to_city = LatLongToCity()\n",
    "latlong_to_city([(51.52, -0.17), (9.93, 76.25), (37.38, -122.08), (np.nan, np.nan)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Natural Language Processing Primitives\n",
    "\n",
    "NLP primitives can apply various natural language processing techniques to text data.\n",
    "\n",
    "#### PolarityScore\n",
    "\n",
    "Calculates the polarity of a text on a scale from -1 (negative) to 1 (positive)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.677\n",
       "1   -0.649\n",
       "2    0.000\n",
       "3    0.000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import PolarityScore\n",
    "\n",
    "x = ['He loves dogs', 'She hates cats', 'There is a dog', '']\n",
    "polarity_score = PolarityScore()\n",
    "polarity_score(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### PartOfSpeechCount\n",
    "\n",
    "Calculates the occurences of each different part of speech."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     [0.0, 0.0]\n",
       "1     [0.0, 0.0]\n",
       "2     [0.0, 0.0]\n",
       "3     [0.0, 0.0]\n",
       "4     [0.0, 0.0]\n",
       "5     [1.0, 0.0]\n",
       "6     [0.0, 0.0]\n",
       "7     [0.0, 0.0]\n",
       "8     [0.0, 0.0]\n",
       "9     [1.0, 0.0]\n",
       "10    [0.0, 0.0]\n",
       "11    [0.0, 0.0]\n",
       "12    [0.0, 0.0]\n",
       "13    [1.0, 0.0]\n",
       "14    [0.0, 0.0]\n",
       "dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import PartOfSpeechCount\n",
    "\n",
    "x = ['He was eating cheese', '']\n",
    "part_of_speech_count = PartOfSpeechCount()\n",
    "part_of_speech_count(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Date of Birth Primitives\n",
    "\n",
    "These primitives transform `DateOfBirth` type variables. They use the time of the feature calculation to extrapolate the current age of a person. This is set by using a [cutoff time](https://featuretools.featurelabs.com/automated_feature_engineering/handling_time.html#what-is-the-cutoff-time).\n",
    "\n",
    "#### Age\n",
    "\n",
    "Calculates the age in years as a floating point number given a date of birth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    19.013699\n",
       "1    35.616438\n",
       "2    21.221918\n",
       "dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import Age\n",
    "\n",
    "reference_date = pd.to_datetime(\"01-01-2019\")\n",
    "age = Age()\n",
    "input_ages = [pd.to_datetime(\"01-01-2000\"),\n",
    "              pd.to_datetime(\"05-30-1983\"),\n",
    "              pd.to_datetime(\"10-17-1997\")]\n",
    "age(input_ages, time=reference_date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are also primitives to see if a birth date falls within a give age range\n",
    "\n",
    "#### AgeOver18\n",
    "\n",
    "Determines whether a person is over 18 years old given their date of birth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    True\n",
       "1    True\n",
       "2    True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import AgeOver18\n",
    "\n",
    "over18 = AgeOver18()\n",
    "over18(input_ages, time=reference_date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### AgeUnder65\n",
    "\n",
    "Determines whether a person is under 65 years old given their date of birth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    True\n",
       "1    True\n",
       "2    True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import AgeUnder65\n",
    "\n",
    "under65 = AgeUnder65()\n",
    "under65(input_ages, time=reference_date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Time Primitives\n",
    "\n",
    "`Datetime` variables store dates or timestamps.  These primitives can extract special properties from a `Datetime` field like if it is a holiday.\n",
    "\n",
    "#### DateToHoliday\n",
    "\n",
    "If there is no holiday, it returns `NaN`. Currently only works for the United States and Canada with dates between 1950 and 2100."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([\"New Year's Day\", nan, 'Memorial Day', 'Independence Day'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import DateToHoliday\n",
    "\n",
    "date_to_holiday = DateToHoliday()\n",
    "dates = pd.Series([datetime(2016, 1, 1),\n",
    "         datetime(2016, 2, 27),\n",
    "         datetime(2017, 5, 29, 10, 30, 5),\n",
    "         datetime(2018, 7, 4)])\n",
    "date_to_holiday(dates)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### NUniqueDaysOfCalendarYear\n",
    "\n",
    "Determines the number of unique calendar days."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import NUniqueDaysOfCalendarYear\n",
    "\n",
    "n_unique_days_of_calendar_year = NUniqueDaysOfCalendarYear()\n",
    "times = [datetime(2019, 2, 1),\n",
    "         datetime(2019, 2, 1),\n",
    "         datetime(2018, 2, 1),\n",
    "         datetime(2019, 1, 1)]\n",
    "n_unique_days_of_calendar_year(times)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Phone Number Primitives\n",
    "\n",
    "The `PhoneNumber` variable stores phone numbers.  These primitives can transformations on the phone numbers to extract metadata like country or area code that are general enough for a model to use\n",
    "\n",
    "#### PhoneNumberToCountry\n",
    "\n",
    "Determines the country of a phone number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    BR\n",
       "1    JP\n",
       "2    US\n",
       "dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import PhoneNumberToCountry\n",
    "\n",
    "phone_number_to_country = PhoneNumberToCountry()\n",
    "phone_number_to_country(['+55 85 5555555', '+81 55-555-5555', '+1-541-754-3010',])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ZIP Code Primitives\n",
    "\n",
    "These primitives can bring in various metadata about a zip code (like geographic or economic information).\n",
    "\n",
    "#### ZIPCodeToState\n",
    "\n",
    "Extracts the state from a ZIPCode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    IL\n",
       "1    CA\n",
       "2    MA\n",
       "dtype: object"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import ZIPCodeToState\n",
    "\n",
    "zipcode_to_state = ZIPCodeToState()\n",
    "zipcode_to_state(['60622', '94120', '02111-1253'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### ZIPCodeToHouseholdIncome\n",
    "\n",
    "Determines the median household income for a ZIP Code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 59000., 103422., 103422.])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import ZIPCodeToHouseholdIncome\n",
    "\n",
    "zipcode_to_household_income = ZIPCodeToHouseholdIncome()\n",
    "zipcode_to_household_income([\"82838\", \"02116\", \"02116-3899\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numeric Primitives\n",
    "\n",
    "The premium primitives have additional nmumeric primitives that add new mathematical transformations and aggregations that aren't present in the open-source library. They are frequently useful in time-series analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### NumPeaks\n",
    "\n",
    "Determines the number of peaks in a list of numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import NumPeaks\n",
    "\n",
    "num_peaks = NumPeaks()\n",
    "num_peaks([-5, 0, 10, 0, 10, -5, -4, -5, 10, 0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### NumZeroCrossings\n",
    "Determines the number of times a list crosses 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import NumZeroCrossings\n",
    "\n",
    "num_zero_crossings = NumZeroCrossings()\n",
    "num_zero_crossings([1, -1, 2, -2, 3, -3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Correlation\n",
    "\n",
    "Computes the correlation between two columns of values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9221388919541468"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import Correlation\n",
    "\n",
    "correlation = Correlation()\n",
    "array_1 = [1, 4, 6, 7]\n",
    "array_2 = [1, 5, 9, 7]\n",
    "correlation(array_1, array_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### CountOutsideRange\n",
    "\n",
    "Determines the number of values that fall outside a certain range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import CountOutsideRange\n",
    "\n",
    "count_outside_range = CountOutsideRange(lower=1.5, upper=3.6)\n",
    "count_outside_range([1, 2, 3, 4, 5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Miscellaneous Data Types\n",
    "\n",
    "Other variable types include FullName, EmailAddress, URL, CountryCode, SubRegionCode, FilePath\n",
    "\n",
    "### FullNameToLastName\n",
    "\n",
    "Determines the first name from a person's name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          Spector\n",
       "1    Oliva y Ocana\n",
       "2             Ware\n",
       "3            Peter\n",
       "4            Brown\n",
       "Name: last_name, dtype: object"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import FullNameToLastName\n",
    "\n",
    "full_name_to_last_name = FullNameToLastName()\n",
    "names = ['Woolf Spector', 'Oliva y Ocana, Dona. Fermina',\n",
    "         'Ware, Mr. Frederick', 'Peter, Michael J', 'Mr. Brown']\n",
    "full_name_to_last_name(names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IsFreeEmailDomain\n",
    "\n",
    "Determines if an email address is from a free email domain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True, False])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import IsFreeEmailDomain\n",
    "\n",
    "is_free_email_domain = IsFreeEmailDomain()\n",
    "is_free_email_domain(['name@gmail.com', 'name@featuretools.com'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### URLToDomain\n",
    "\n",
    "Determines the domain of a url."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    play.google.com\n",
       "1       google.co.in\n",
       "2       facebook.com\n",
       "dtype: object"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import URLToDomain\n",
    "\n",
    "url_to_domain = URLToDomain()\n",
    "urls =  ['https://play.google.com', 'http://www.google.co.in', 'www.facebook.com']\n",
    "url_to_domain(urls)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CountryCodeToIncome\n",
    "\n",
    "Transforms a 2-digit or 3-digit ISO-3166-1 country code into Gross National Income (GNI) per capita."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([58270.,  3990.,  5920.])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import CountryCodeToIncome\n",
    "\n",
    "country_code_to_income = CountryCodeToIncome()\n",
    "country_code_to_income(['USA', 'AM', 'EC'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SubRegionCodeToMedianHouseholdIncome\n",
    "\n",
    "Determines the median household income of a US sub-region."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([51113, 63481, 63805, 83382, 57700, 62447])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import SubRegionCodeToMedianHouseholdIncome\n",
    "\n",
    "sub_region_code_to_median_household_income = SubRegionCodeToMedianHouseholdIncome()\n",
    "subregions = [\"US-AL\", \"US-IA\", \"US-VT\", \"US-DC\", \"US-MI\", \"US-NY\"]\n",
    "sub_region_code_to_median_household_income(subregions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FileExtension\n",
    "\n",
    "Determines the extension of a filepath."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     .txt\n",
       "1    .json\n",
       "2      NaN\n",
       "dtype: object"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from featuretools.primitives import FileExtension\n",
    "\n",
    "file_extension = FileExtension()\n",
    "file_extension(['doc.txt', '~/documents/data.json', 'file'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}