A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests very welcomed. Please do not list any confidential projects!
For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os
See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products
Google Internal | Google External | Open Source / Real-World |
---|---|---|
MapReduce | Apache Hadoop, Spark | |
Protocol Buffer | Protobuf | Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR |
Stubby | gRPC | Thrift, Bolt |
Chubby | Apache Zookeeper, etcd, HashiCorp Consul | |
Goops / PubSub | Apache Kafka, Apache Pulsar, Facebook LogDevice | |
Dapper | OpenTracing, jaeger, LightStep, Honeycomb | |
//base |
abseil |
Google Internal | Google External | Open Source / Real-World |
---|---|---|
Borg | Kubernetes | Apache Mesos, Apache Aurora, HashiCorp Nomad |
GSLB, GFE, Maglev, uberproxy | Cloud Load Balancing | ELB, HAProxy, Istio, F5, envoy |
uberproxy (sso proxy) | buzzfeed-sso Pomerium | |
data center hardware | open compute | |
Jupiter, Starblaze | ||
B4, Stargate, TE | ||
USPS, Andromeda |
Google Internal | Google External | Open Source / Real-World |
---|---|---|
GFS/Colossus | HDFS, Ceph, GlusterFS, MooseFS | |
BigTable | Cloud BigTable | PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB |
Spanner | Cloud Spanner | Vitess, CockroachDB, TiDB |
ColumnIO / Capacitor | Apache Parquet | |
sstable | levelDB | RocksDB |
zippy | Snappy | lz4 |
RecordIO | TFRecords |
Google Internal | Google External | Open Source | SaaS |
---|---|---|---|
Dremel | BigQuery | Apache Drill, Presto, Spark(sort-of), | AWS Athena, Redshift Spectrum, Snowflake |
Dremel UI | Redash, Metabase, Apache Superset | ||
Search (Mustang, Alexandria) | Elasticsearch, Solr, Lucene | algolia | |
pubsub | pubsub | NATS.io, RabbitMQ, PubNub | AWS SQS/SNS, AWS AppSync |
Flume (Java) | Apache Beam | Apache Crunch | |
MillWheel | Cloud Dataflow | Apache Flink | |
Colab | Colaboratory | Jupyter | |
PLX | Google Data Studio | Mode |
Google Internal | Google External | Open Source |
---|---|---|
prodaccess/LOAS | Keymaster | |
prod secrets/identity management | chamber, knox, SPIFFE |
Google Internal | Real-World |
---|---|
software engineering at google | |
valentine | Vault, 1Password, Lastpass, pass |
OWNERS files in repo | github CODEOWNERS |
snippets | Khan/snippets |
memegen | memegen |
SnipIt | recordit, CloudApp, dropbox screenshots |
edge, people ops training | LifeLabs |
googlegeist | Culture Amp, humu, tinypulse, peakon |
HRIS/ERP | Namely, BambooHR, Workday |
stuff (SaaS IT management) | productiv, intello, zylo |
stuff (Device Management) | Fleetsmith, jamf, rippling IT |
device security monitoring | Red Canary |
beyondcorp | beyondcorp |
go/ links | golinks, go, Goat, trotto |
google3 philosophy | innersource, monorepo |
doing code review | code review |
safely sharing 1-time secrets | croc, onetimesecret, privatebin |
messaging | mattermost, Slack, gchat |
peer bonus | bonus.ly, zestful |
kudos | heytaco, slack ++ bot |
also check out xoogler.co, which organizes events, slack channels etc
disclaimer: I'm not affiliated with any of the technologies mentioned above.
disclaimer: I left Google a number of years ago so some of the naming might be dated