Skip to content

Latest commit

 

History

History
23 lines (13 loc) · 928 Bytes

long-li-neville-scio-scala-on-google-cloud.md

File metadata and controls

23 lines (13 loc) · 928 Bytes

#Scio - Scala on Google Cloud

Neville Li (sinisa_lyh)

http://www.lyh.me/

Spotify

Abstract

We'll talk about Scio, a Scala API for Google Cloud Dataflow and Apache Beam, and how Spotify uses Scala for data processing and machine learning on Google Cloud Platform. We'll cover topics including Algebird, macros, and shapeless.

Notes

Related links: https://github.com/spotify/scio https://github.com/spotify/ratatool https://github.com/nevillelyh/shapeless-datatype

Bio

Neville is a software engineer at Spotify who works mainly on data infrastructure and tools for machine learning and advanced analytics. In the past few years he has been driving the adoption of Scala and new data tools for music recommendation, including Scalding, Spark, Storm and Parquet. Before that he worked on search quality at Yahoo! and old school distributed systems like MPI.