pysummit18.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Swiss Python Summit 2018 – 16.02.2018\n",
    "\n",
    "Video recordings coming soon.\n",
    "\n",
    "- 08:00 – 09:00 \tRegistration + Coffee (open until 09:15)\n",
    "- 09:00 – 09:15 \tWelcome\n",
    "- 09:15 – 10:00 \tCarina Haupt, Rocket Science and Software Engineering\n",
    "- 10:00 - 10:40 \tTim Head, From Exploring Data Interactively to Creating Reproducible Pipelines\n",
    "- 10:40 - 11:10 \tCoffee break\n",
    "- 11:10 – 11:40 \tRaphael Das Gupta, Comprehensions – Origin, History, Use\n",
    "- 11:40 – 12:10 \tGabriel Krummenacher, Leveraging Neural Networks and Python to Forecast Train Delays in the Swiss Railway Network\n",
    "- 12:10 – 12:40 \tIacopo Spalletti, Real Time Django\n",
    "- 12:40 – 14:00 \tLunch\n",
    "- 14:00 – 14:30 \tSarah Mühlemann, SpyPi – An Attempt to Get Students Into Data Security\n",
    "- 14:30 – 15:00 \tAmit Kumar, Lets Talk About GIL!\n",
    "- 15:00 – 15:45 \tCoffee break\n",
    "- 15:45 – 16:15 \tJosef Spillner, Serverless Computing: FaaSter, Better, Cheaper and More Pythonic\n",
    "- 16:15 – 16:45 \tPeter Hoffmann, 12 Factor Apps for Data-Science with Python\n",
    "- 16:45 – 17:00 \tClosing\n",
    "- 17:00 – 20:00 \tSocial Event / Apéro\n",
    "\n",
    "http://www.python-summit.ch/\n",
    "\n",
    "Recordings and slides are available here: http://www.python-summit.ch/pages/recordings.html\n",
    "\n",
    "---\n",
    "\n",
    "Also check out my notes from 2016: https://github.com/etoulas/swisspythonsummit16/blob/master/pysummit16.md"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. From Exploring Data Interactively to Creating Reproducible Pipelines \n",
    "## aka. \"Dashboard\" by Data Scientist\n",
    "\n",
    "Tim Head, Wild Tree Tech\n",
    "\n",
    "- Slides: https://hub.mybinder.org/user/betatim-talk-sw-hon-summit-2018-m5shv32c/notebooks/slides.ipynb#/slide-0-0\n",
    "- Binder and Papermill are open-source.\n",
    "\n",
    "### Making notebooks reusable\n",
    "\n",
    "- Create `requirements.txt`\n",
    "- Run `repo2docker` https://repo2docker.readthedocs.io/\n",
    "\n",
    "### Binder\n",
    "\n",
    "- Firefox Extension\n",
    "- BinderHub - `pip install github/jupyther/binderhub`\n",
    "\n",
    "### Building pipelines out of Notebooks\n",
    "\n",
    "- Papermill - `pip install papermill`\n",
    "- Tag cell at the top as parameters (before even defaultparameters)\n",
    "- Run notebooks from the command-line (e.g. via a cronjob)\n",
    "    - `$ papermill input-file.ipynb output-file.ipynb -p year 2016`\n",
    "- Read in other notebook data from `read_notebook()`\n",
    "\n",
    "nbdime - `nbdiff` notebook diffs https://nbdime.readthedocs.io/en/stable/\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Comprehensions\n",
    "\n",
    "https://mybinder.org/v2/gh/das-g/comprehensions-talk/master\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Neural Networks and the Swiss Railway System\n",
    "\n",
    "### Supervised Text Classification\n",
    "\n",
    "- Set of documents\n",
    "- with labels: Letter, Blog, News\n",
    "- RandomForestClassifier() from scikit-learn, keras\n",
    "- fit the model to the data\n",
    "- use classifier to predict document type\n",
    "---\n",
    "\n",
    "- create feature vectors out of text\n",
    "- pandas, spaCy\n",
    "- fit the model, predict\n",
    "---\n",
    "\n",
    "- Bag of Words\n",
    "- scikit-learn CountVectorizer()\n",
    "- TF-IDF\n",
    "- TfidfVectorizer()\n",
    "- Problems\n",
    "    - word similarity\n",
    "    - order of words (sentence structure)\n",
    "---\n",
    "\n",
    "- Dense word embeddings\n",
    "     1. onehot encoding\n",
    "     2. embedding => word2vec\n",
    "---\n",
    "- Recurrent Neural Network\n",
    "- For sequential data points: text, audio, signal, time series\n",
    "- Keras LSTM model (LTSM over messages)\n",
    "- RF vs LR vs custom model => how good is the accurancy after 10 min (time matters when predicting the duration of the disruption\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Real Time Django\n",
    "\n",
    "Iacopo Spalletti, Founder Nephilalt\n",
    "\n",
    "- Channels (dealing with asynchronous protocols)\n",
    "    - websockets\n",
    "    - any protocol (bots, IoT)\n",
    "- Channels 2 release 2 weeks ago (not yet all features from v1)\n",
    "- Examples in Channel 1\n",
    "\n",
    "---\n",
    "\n",
    "Outline\n",
    "- Channel concepts\n",
    "- Demo\n",
    "- Code drill down\n",
    "- Channels 2\n",
    "\n",
    "### Channel concepts\n",
    "\n",
    "- A FIFO at-most-once queue\n",
    "- Messegar producers and consumers\n",
    "- Asynchronous: event-driven (Channels 1 \"hides\" async parts)\n",
    "- ASGI: protocol specs (async version of WSGI)\n",
    "    - Version 1 for Channel 1\n",
    "    - Version 2 for Channel 2\n",
    "- Protocol Server\n",
    "    - Daphne for websockets\n",
    "    - Defines channel\n",
    "- Routing\n",
    "    - map channels to consumers\n",
    "    - similar to Django URLconf\n",
    "- Consumers\n",
    "    - read/write messages on the channel\n",
    "    - similar to Django views (result is a message put on a channel)\n",
    "- Frontend\n",
    "    - Websockets means JS\n",
    "\n",
    "### Demo\n",
    "\n",
    "https://github.com/yakky/channeled-dashboard\n",
    "\n",
    "Features (like Google Docs)\n",
    "- Count active users\n",
    "- Concurrency check (on documents)\n",
    "- Browser notification\n",
    "\n",
    "### Dive in\n",
    "- Channel layers\n",
    "    - Config (like Redis, routing)\n",
    "- Routing\n",
    "    - like URLconf - works on URLs\n",
    "- Consumers\n",
    "    - JsonWebsocketConsumer (fairly low-level, lot of control)\n",
    "    - Counting users\n",
    "    - Concurrency monitor\n",
    "    - Browser notifications\n",
    "        - streams are subchannels, used by the multiplexer\n",
    "\n",
    "### Channel 2\n",
    "\n",
    "- based on asyncio\n",
    "- Python 3.5+\n",
    "- mix of sync / async code\n",
    "- protocol server and workers are on the same process as Django\n",
    "- persistent consumers\n",
    "\n",
    "### FAQ\n",
    "\n",
    "- Performance: Workers can be scaled \n",
    "- Deployment: WSGI can be replaced by Daphne (or WSGI and map wecksockets via paths to Daphne)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. SpyPi\n",
    "\n",
    "- https://github.com/sarah314/SpyPi\n",
    "- http://spypi.ch/\n",
    "\n",
    "---\n",
    "\n",
    "- Simultanious connections\n",
    "- SMS Spoofing: `clockwork` https://www.clockworksms.com/\n",
    "- MITM: mitmproxy\n",
    "\n",
    "---\n",
    "\n",
    "For Sarah:\n",
    "\n",
    "- http://codebutler.com/firesheep\n",
    "- https://en.wikipedia.org/wiki/Firesheep\n",
    "- https://www.quora.com/Is-there-a-present-day-equivalent-to-Firesheep\n",
    "- Bottom line, doesn't work anymore in FF. Concept used is mainly fixed now (SSL everywhere).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Lets Talk About GIL!\n",
    "\n",
    "- https://github.com/aktech/gil_talk\n",
    "- http://live.sympy.org/\n",
    "\n",
    "---\n",
    "\n",
    "- Python threads work good for I/O bound processes\n",
    "- `multprocessing`\n",
    "- C-extensions\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Serverless Computing: FaaSter, Better, Cheaper and More Pythonic\n",
    "\n",
    "Prof. Josef Spillner, ZHAW\n",
    "\n",
    "---\n",
    "\n",
    "- PyWren - improved conveyance of \"serverless\" paradigm (AWS Lambda)\n",
    "- Lambada - Deployment with dependencies (AWS Lambda)\n",
    "\n",
    "https://gitlab.com/josefspillner/lambada\n",
    "\n",
    "- Snafu https://github.com/serviceprototypinglab/snafu/\n",
    "    - self-hosted FaaS\n",
    "    - avoids vendor lock-in\n",
    "    - `pip install snafu` for toying/testing/debugging\n",
    "        - `snafu-import`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. 12 Factor\n",
    "\n",
    "- Logging / Monitoring\n",
    "https://landing.google.com/sre/book/chapters/part3.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}