Playing w/ Spotify's Scio
- Ingest https://www.meetup.com/fr-FR/meetup_api/
- Streaming processing w/ Apache-Beam X Scio
- Query trending topics from BigQuery
# Playing w/ Spotify's Scio
# Using DirectRunner
# Build
$ sbt pack
# Run
$ target/pack/bin/main
# Just run
sbt run
# (BigQuery) Get top 5 trending topics within the last 5 minutes
SELECT SUM(score) as score, topic_name, timestamp
FROM `project_id.dataset_name.table_name`
WHERE timestamp > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 5 MINUTE)
GROUP BY timestamp, topic_name
ORDER BY score DESC
LIMIT 5