alexxnica / emojitracker

:dizzy: track ALL the emoji

emojitracker 💫

Emojitracker.com tracks realtime emoji usage on Twitter.

Emojitracker is a complex project with a service-oriented architecture, and thus has now been split up into multiple repositories. This repository mainly just serves as table of contents of sorts. Major components of the project are listed below.

Architecture

Bad diagram of how the main parts fit together in production:

Core Applications

The core applications of Emojitracker. Note that while these are open source for educational purposes, they are currently all rights reserved. Please contact me directly if you want to use them for something.

• emojitrack-web web frontend and application server.
• emojitrack-feeder consumes the Twitter Streaming API and feeds our data pipeline.
• emojitrack-streamer handles streaming updates to clients via SSE.
  • Ruby version (deprecated)
  • NodeJS version (deprecated)
  • Go version (production)
  • API spec defines the streamer spec, tests servers in staging.

Libraries and Tools

Most of the generalizable and useful pieces of Emojitracker have been carved out into maintained open-source libraries. These libraries are all freely licensed (see individual repositories for details).

Emoji Encoding

• emoji_data.rb utility library for handling the Emoji vs Unicode nightmare (Ruby).
• emoji-data-js utility library for handling the Emoji vs Unicode nightmare (NodeJS port).
• exmoji utility library for handling the Emoji vs Unicode nightmare (Elixir/Erlang port).

Emoji Assets

• emojistatic Generates static Emoji assets for a public CDN.
• cssquirt Embeds images (or directories of images) directly into CSS via the Data URI scheme.

Streaming

• sseserver High-performance Server-Sent Events endpoint for Go.
• sse-bench Benchmarks and load tests Server-Sent Events endpoints.

Other Information

The narrative version of how version 1.0 of this project was built is in the Medium post "How I Built Emojitracker". Note however, it is quite out of date, and does not reflect a substantial amount of change over the years.