03 dataframes & plots.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Binder](https://mybinder.org/badge_logo.svg)](https://notebooks.gesis.org/binder/v2/gh/jolin-io/fall-in-love-with-julia/main?filepath=03%20dataframes%20%26%20plots.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fall in love with Julia: Dataframes & Plots\n",
    "![logo](logo-fall-in-love-with-julia.png)\n",
    "\n",
    "Welcome to the fall-in-love-with-julia 101 series. My name is Stephan Sahm. You are always welcome to reach me at stephan.sahm@gmx.de.\n",
    "\n",
    "This time we focus on plots and dataframes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Julia basics - Multiple Dispatch\n",
    "\n",
    "Let's start with recapping some julia basics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can easily define your custom type hierarchies. Mind that only leaf-types like `struct`s can take values. Abstract types are merely tags which are used for overloading functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "abstract type Pet end\n",
    "struct Cat <: Pet\n",
    "    name\n",
    "end\n",
    "struct Dog <: Pet\n",
    "    name\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kitty = Cat(\"Kitty\")\n",
    "leo = Cat(\"Leo\")\n",
    "rex = Dog(\"Rex\")\n",
    "charlie = Dog(\"Charlie\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can define functions super easily"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "name(pet) = pet.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can specify types by annotating arguments with e.g. `::Cat` to ensure it is of type `Cat`.\n",
    "This way you can overload/specialize the function to various specific methods.\n",
    "\n",
    "This is the core of julia and is called Multiple Dispatch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "meet(pet1::Cat, pet2::Cat) = \"$(name(pet1)) looks curiously at $(name(pet2))\"\n",
    "meet(pet1::Cat, pet2::Dog) = \"$(name(pet1)) hisses at $(name(pet2))\"\n",
    "meet(pet1::Pet, pet2::Pet) = \"$(name(pet1)) just looks at $(name(pet2)).\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function pet_story(pet1, pet2)\n",
    "    println(\"On a lovely autumn afternoon, $(name(pet1)) meets $(name(pet2))...\")\n",
    "    println(meet(pet1, pet2))\n",
    "    println(\"and then goes away\")\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pet_story(kitty, charlie)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Julia comes with full-fledged support for Arrays and Matrices. If you already liked numpy, this takes it to the next level."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pets = [kitty, leo, rex, charlie]\n",
    "name.(pets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[(pet1, pet2, meet(pet1, pet2)) for pet1 in pets for pet2 in pets]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[meet(pet1, pet2) for pet1 in pets, pet2 in pets]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## it is your time:\n",
    "Define a new Pet type Rabbit and let a duck meet a cat. 🐕 🐈 🐇"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# go on here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plots\n",
    "\n",
    "Let's jump into plotting things"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Plots  # top level go-to package for plotting in Julia, integrating with many many packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = 20\n",
    "x = 1:n\n",
    "y = rand(n)\n",
    "x, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = range(0, 1, length = n), rand(n, 3)*10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(x, y, labels = [\"one\" \"two\" \"three\"], xlabel = \"xlabel\", ylabel = \"αβγ\",  title = \"\\$ \\\\nabla_p^q \\\\sum_{i = 0}^{n} \\$\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can easily access documentation for the attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "?plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "plotattr(:Series)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plotattr(\"seriescolor\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also plot functions and let julia infer which concrete points need to be drawn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(sin, cos, 0, pi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also add to an existing plot by using `plot!` with exclamation mark"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot!(sin, cos, pi, 2pi)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(1:0.1:10, [sin, cos, log], labels=[\"sin\" \"cos\" \"log\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some easy layouts are quickly done"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1:10\n",
    "y = rand(10, 4)\n",
    "plot(x, y, layout = (2, 2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p1 = plot(x, y) # Make a line plot\n",
    "p2 = scatter(x, y) # Make a scatter plot\n",
    "p3 = plot(x, y, xlabel = \"This one is labelled\", lw = 3, title = \"Subtitle\")\n",
    "p4 = plot(x, y, seriestype = :histogram) # Four histograms each with 10 points? Why not!\n",
    "plot(p1, p2, p3, p4, layout = (2, 2), legend = false)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And a last fancy 3d animation using plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the Lorenz attractor\n",
    "Base.@kwdef mutable struct Lorenz\n",
    "    dt::Float64 = 0.02\n",
    "    σ::Float64 = 10\n",
    "    ρ::Float64 = 28\n",
    "    β::Float64 = 8/3\n",
    "    x::Float64 = 1\n",
    "    y::Float64 = 1\n",
    "    z::Float64 = 1\n",
    "end\n",
    "\n",
    "function step!(l::Lorenz)\n",
    "    dx = l.σ * (l.y - l.x)\n",
    "    l.x += l.dt * dx\n",
    "    \n",
    "    dy = l.x * (l.ρ - l.z) - l.y\n",
    "    l.y += l.dt * dy\n",
    "    \n",
    "    dz = l.x * l.y - l.β * l.z\n",
    "    l.z += l.dt * dz\n",
    "end\n",
    "\n",
    "attractor = Lorenz()\n",
    "\n",
    "\n",
    "# initialize a 3D plot with 1 empty series\n",
    "plt = plot3d(\n",
    "    1,\n",
    "    xlim = (-30, 30),\n",
    "    ylim = (-30, 30),\n",
    "    zlim = (0, 60),\n",
    "    title = \"Lorenz Attractor\",\n",
    "    marker = 2,\n",
    ")\n",
    "\n",
    "# build an animated gif by pushing new points to the plot, saving every 10th frame\n",
    "@gif for i=1:1500\n",
    "    step!(attractor)\n",
    "    push!(plt, attractor.x, attractor.y, attractor.z)\n",
    "end every 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## it is your time: \n",
    "\n",
    "Try to plot a heart ❤️"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# go on here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interact\n",
    "\n",
    "You can also add interactivity very easily"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Interact\n",
    "using Colors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@manipulate for r = 0:0.05:1, g = 0:0.05:1, b = 0:0.05:1\n",
    "    HTML(string(\"<div style='color:#\", hex(RGB(r,g,b)), \"'>Color me</div>\"))\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "@manipulate for n=1:25, g=[:scatter, :path], col=colorant\"red\"\n",
    "    plot(rand(n), rand(n), linetype=g, color=col)\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DataFrames\n",
    "\n",
    "Let's look at [DataFrames.jl](https://dataframes.juliadata.org/stable/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is very intuitive, much more intuitive than pandas.\n",
    "\n",
    "You can create dataframes the way you would first guess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = DataFrame(A = 1:4, B = [\"M\", \"F\", \"F\", \"M\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "you can index into dataframes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[:, :A]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[2, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And also update them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[:, :A] = [4,3,4,2]\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[1, :] = [100, \"Z\"]\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ".= gives you elementwise/broadcast behaviour"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[:, :A] .= 1\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RDatasets\n",
    "With [RDatasets](https://github.com/JuliaStats/RDatasets.jl) you have access to all standard datasets you know from R"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import RDatasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = RDatasets.dataset(\"datasets\", \"iris\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## StatsPlots\n",
    "[StatsPlots](https://github.com/JuliaPlots/StatsPlots.jl) gives you many so called plot recipes with which you can easily visualize your dataframes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using StatsPlots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataviewer(iris)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "StatPlots also gives you a special macro `@df` with which you can use normal plots functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@df iris scatter(:PetalLength, :PetalWidth)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "StatsModels also adds common datascience plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@df iris marginalkde(:PetalLength, :PetalWidth)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## it is your time:\n",
    "\n",
    "using the dataviewer, investigate another standard RDataset 📊 📈 📉"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# go on here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Transforming DataFrames - Query.jl\n",
    "\n",
    "If you know dplyr, [Query.jl](https://www.queryverse.org/Query.jl/stable/) will be very intuitive for you. If you don't know it, it will be similarly intuitive.\n",
    "\n",
    "It combines a functional style with so called piping together with functional operators like filter, map, groupby, joins etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Query, Statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = DataFrame(name=[\"John\", \"Sally\", \"Kirk\"], age=[23., 42., 59.], children=[3,5,2])\n",
    "\n",
    "df |>\n",
    "    @filter(_.age>50) |>\n",
    "    @map({_.name, _.children}) |>\n",
    "    DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Full list of commands can be found here\n",
    "http://www.queryverse.org/Query.jl/stable/standalonequerycommands/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = DataFrame(name=[\"John\", \"Sally\", \"Kirk\"], age=[23., 42., 59.], children=[3,2,2])\n",
    "\n",
    "df |>\n",
    "    @groupby(_.children) |>\n",
    "    @map({Key=key(_), Count=length(_)}) |>\n",
    "    DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = DataFrame(a=[1,1,2,3], b=[4,5,6,8])\n",
    "df |>\n",
    "    @groupby(_.a) |>\n",
    "    @map({a=key(_), b=mean(_.b)}) |>\n",
    "    @filter(_.b > 5) |>\n",
    "    @orderby_descending(_.b) |>\n",
    "    DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = DataFrame(a=[1,2,3], b=[1.,2.,3.])\n",
    "df2 = DataFrame(c=[2,4,2], d=[\"John\", \"Jim\",\"Sally\"])\n",
    "\n",
    "df1 |> @groupjoin(df2, _.a, _.c, {t1=_.a, t2=length(__)}) |> DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## it is your time:\n",
    "\n",
    "Apply your first transformation on the iris dataset\n",
    "* filter Petallength > 2\n",
    "* group_by Species\n",
    "* get the mean of Sepalwidth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# go on here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For those interested, note that Query.jl also supports LINQ syntax. See https://www.queryverse.org/Query.jl/stable/linqquerycommands/ for more details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Other Tables\n",
    "\n",
    "DataFrames are really just one of the several possible tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "namedtuple_of_vector = (one = [1,2,3,4], two = [9,1,3,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@df namedtuple_of_vector plot(:one, :two)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vector_of_namedtuple = [\n",
    "    (one = 1, two = 3.0),\n",
    "    (one = 1, two = 4.0),\n",
    "    (one = 2, two = 5.0),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vector_of_namedtuple |>\n",
    "    @groupby(_.one) |>\n",
    "    @map({one=_.one, two=mean(_.two)}) |>\n",
    "    collect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataviewer(vector_of_namedtuple)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating your own Table Type\n",
    "\n",
    "Finally, I want to show you that you can also implement your own rigorous Table type.\n",
    "(We follow the [Tables.jl](https://tables.juliadata.org/stable/) interface.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "struct MatrixTable <: Tables.AbstractColumns\n",
    "    names\n",
    "    matrix\n",
    "end\n",
    "# getter methods to avoid getproperty clash\n",
    "names(m::MatrixTable) = getfield(m, :names)\n",
    "matrix(m::MatrixTable) = getfield(m, :matrix)\n",
    "\n",
    "# declare that MatrixTable is a table\n",
    "Tables.istable(::Type{<:MatrixTable}) = true\n",
    "# schema is column names and types\n",
    "Tables.schema(m::MatrixTable) = Tables.Schema(names(m), fill(eltype(matrix(m)), size(matrix(m), 2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# column interface\n",
    "Tables.columnaccess(::Type{<:MatrixTable}) = true\n",
    "Tables.columns(m::MatrixTable) = m\n",
    "\n",
    "# required Tables.AbstractColumns object methods\n",
    "Tables.getcolumn(m::MatrixTable, nm::Symbol) = matrix(m)[:, findfirst(x -> x == nm, names(m))]\n",
    "Tables.getcolumn(m::MatrixTable, i::Int) = matrix(m)[:, i]\n",
    "Tables.columnnames(m::MatrixTable) = names(m)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally you also want to implement this older Table Interface, which may still be used from some julia packages, but quite rarely these days."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import IteratorInterfaceExtensions\n",
    "IteratorInterfaceExtensions.getiterator(x::MatrixTable) = Tables.datavaluerows(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see that it works"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mat = [1 4.0 \"7\"; 2 5.0 \"8\"; 3 6.0 \"9\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mattbl = MatrixTable([:one, :two, :three], mat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you have pretty support for indexing and the like (indexing is all columnwise)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mattbl[:one]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mattbl[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mattbl.one"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also access the table row-wise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "collect(Tables.rows(mattbl))[1].three"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## it is your time:\n",
    "\n",
    "try out query.jl, `@df` and `dataviewer` on the newly defined table type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# go on here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The time-to-first-plot problem\n",
    "\n",
    "The time-to-first-plot is julia's major drawback currently. While being optimized for high-level and high-performance, it was more throughput focussed so far, meaning that doing a simple plot from scratch can take really long.\n",
    "\n",
    "There are ways to improve these startup times\n",
    "- wait for newer julia versions, as this is one of the top 3 focus points of the julia development team right now\n",
    "- reuse your julia session as much as possible. Checkout [`Revise.jl`](https://github.com/timholy/Revise.jl).\n",
    "- use [`PackageCompiler`](https://github.com/JuliaLang/PackageCompiler.jl) to build faster precompiled versions of packages you use often. On JuliaCon 2020 there was a lovely [introduction talk for it](https://www.youtube.com/watch?v=d7avhSuK2NA&list=PLP8iPy9hna6Tl2UHTrm4jnIYrLkIcAROR).\n",
    "- use [`SnoopCompile`](https://github.com/timholy/SnoopCompile.jl) to speed up your own developed packages, or as an alternative to PackageCompiler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Thank you all for joining today!\n",
    "\n",
    "feel always welcome to contact me at stephan.sahm@gmx.de\n",
    "\n",
    "![bye bye](https://images.unsplash.com/photo-1581368242547-06eef3599510?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=2550&q=80)"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": "6653087beec44eefb3accc3ee7108161",
   "lastKernelId": "42e3a8c9-cb57-4fbd-ae8c-a0c9c81d58f8"
  },
  "kernelspec": {
   "display_name": "Julia 1.6.1",
   "language": "julia",
   "name": "julia-1.6"
  },
  "language_info": {
   "name": "julia",
   "version": ""
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}