GithubHelp home page GithubHelp logo

xg2xg's Introduction

A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests very welcomed. Please do not list any confidential projects!

For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products, Database of Databases

Technology

Core Technology

Google Internal Google External Open Source / Real-World
MapReduce Apache Hadoop, Spark
Flume DataFlow Apache Beam
Protocol Buffer Protobuf Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR
Stubby gRPC Thrift, Bolt
Chubby Apache Zookeeper, etcd, HashiCorp Consul
Goops / PubSub Apache Kafka, Apache Pulsar, Facebook LogDevice
//base abseil

Infrastructure

Google Internal Google External Open Source / Real-World
Borg Kubernetes Apache Mesos, Apache Aurora, HashiCorp Nomad
GSLB Cloud Load Balancing - Internal AWS ELB, Istio, envoy, linkerd
OnePlatform API Gateway Swagger
GFE, Maglev, uberproxy Cloud Load Balancing - HTTPS / External AWS ALB, HAProxy, nginx, F5
uberproxy (sso proxy) Identity-Aware Proxy buzzfeed-sso, Pomerium
Zanzibar Zanzibar Research Paper Ory Keto
data center hardware open compute
Jupiter, Starblaze
B4, Stargate, TE
USPS, Andromeda
ESDN Faucet

Storage

Google Internal Google External Open Source / Real-World
GFS/Colossus HDFS, Ceph, GlusterFS, MooseFS
BigTable Cloud BigTable PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB
Spanner Cloud Spanner Vitess, CockroachDB, TiDB
ColumnIO / Capacitor Apache Parquet
sstable levelDB RocksDB, pebble
zippy Snappy lz4
RecordIO TFRecords

Services

Google Internal Google External Open Source SaaS
Dremel BigQuery Apache Drill, Presto, Spark(sort-of), AWS Athena, Redshift Spectrum, Snowflake
Dremel UI Redash, Metabase, Apache Superset
Search (Mustang, Alexandria) Elasticsearch, Solr, Lucene algolia
pubsub pubsub NATS.io, RabbitMQ, PubNub AWS SQS/SNS, AWS AppSync
MillWheel Cloud Dataflow Apache Flink, beam
Colab Colaboratory Jupyter
PLX Google Data Studio Mode

DevOps

Google Internal Google External Real-World
Assimilator Dominator
Blaze Bazel Buck, Pants, please.build, Blade, recc, BuildBuddy, flare.build
Oncall PagerDuty, OpsGenie, VictorOps
varz/borgmon/monarch Datadog, Prometheus, M3, librato, newrelic, skylight, scout, Scotty/tricorder, netdata, bosun, also this and this
Viceroy Grafana
Exception/Error Tracking (??) Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps, ELK stack
styleguides google styleguides PEP-8, HoundCI auto-style-reviewer
Blaze/Forge/TAP/BuildCop Cloud Build buildkite, CircleCI, travis, jenkins, gitlabCI, github, Drone
Forge/ObjFS EngFlow, flare.build
Sandman(test env)/Guitar
Sisyphus / Rapid Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI, samson
MPM Docker
borg / borgcfg / gcl Jsonnet, Cue AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, kubecfg, pulumi
logging, analog StackDriver logstash, fluentd, PaperTrail, cernan
CodeSearch, Grimoire Zoekt kythe Sourcegraph, OpenGrok, livegrep
Critique, Gerrit, Mondrian etc. Gerrit Reviewable , Phabricator
cider Eclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab)
buganizer Google Issue Tracker JIRA, bugzilla, github issues, Linear
ToTT Google Test Blog Increment
Copybara / MOE Copybara, MOE
workflow/dependency management Luigi, Airflow, digdag, Pachyderm, Dask
ErrorProne ErrorProne SpotBugs, FindBugs
Dapper stackdriver trace zipkin, opentracing, jaeger, LightStep, Honeycomb
C++ Tips of the Week Abseil C++ Tips of the Week
DiRT ChaosMonkey, aws fis
Rosie microplane

Security

Google Internal Google External Open Source
prodaccess/LOAS Keymaster
prod secrets/identity management chamber, knox, SPIFFE

IT / Operations / Misc

Google Internal Real-World
software engineering at google Software Engineering at Google: Lessons Learned from Programming Over Time
SRE @ google
valentine Vault, 1Password, Lastpass, pass
OWNERS files in repo github CODEOWNERS
snippets Khan/snippets
SnipIt recordit, CloudApp, dropbox screenshots, Snippyly
stuff (SaaS IT management) productiv, intello, zylo
stuff (Device Management) Fleetsmith, jamf
device security monitoring Red Canary
beyondcorp beyondcorp
go/ links golinks, go, Goat, trotto, go-shorten
google3 philosophy innersource, monorepo
doing code review code review
safely sharing 1-time secrets croc, onetimesecret, privatebin
messaging mattermost, Slack, gchat
tech talks TechTalks @ Google
g3doc writethedocs, docs-as-code
GUTS spoke

PeopleOps / Culture

Google Internal Real-World
OKR CultureAmp OKR, Lattice Goals, Ally, workboard
HRIS/ERP Namely, BambooHR, Workday, Paylocity
peer bonus bonus.ly
kudos heytaco, slack ++ bot
perks fond
talks Talks @ Google
edge, people ops training LifeLabs
googlegeist Culture Amp, humu, tinypulse, peakon
Meng Search Inside Yourself
Lazlo Work Rules
Claire Stapleton Tech Support - existential advice for the modern tech worker
books about google How Google Works, In The Plex

also check out xoogler.co, which organizes events, slack channels etc

disclaimer: I'm not affiliated with any of the technologies mentioned above.

disclaimer: I left Google a number of years ago so some of the naming might be dated

xg2xg's People

Contributors

jhuangtw-dev avatar jhuangtw avatar iawia002 avatar mexisme avatar adamyi avatar nh2 avatar yegle avatar tslmy avatar githubfz avatar zellyn avatar issmirnov avatar jacktuck avatar yuikns avatar rgooch avatar minhaz avatar morgante avatar iandioch avatar pbiggar avatar phasmal avatar sqs avatar appwiz avatar rohantalip avatar sbrother avatar commure-stabai avatar stabai avatar thomas4019 avatar gritzko avatar xiaoxq avatar zachborboa avatar zachgrayio avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.