144lucky Goto Github PK

12306

12306智能刷票，订票

api

hbase2mongo

app-consumer-loan

blog

markdown content of garyelephant's blog.http://garyelephant.me

brickhouse

Hive UDF's for the data warehouse

dcos

DC/OS Build and Release tools

hadoop-cluster-docker

Quickly build arbitrary size Hadoop Cluster based on Docker

hbase-tools

hbase-tools try easy to use and test the hbase,

hbaseapi

hbase operations

hdata

一个支持多数据源的ETL数据导入/导出工具

hive-third-functions

Some useful custom hive udf functions, especial array and json functions.

hive-udaf-maxrow

A simple aggregate function (UDAF) for Hive -- like max() but it allows you to refer to additional columns in the maximal row.

ibis

Productivity-centric Python data analysis framework for SQL systems and the Hadoop platform. Co-founded by the creator of pandas

incubator-carbondata

Mirror of Apache CarbonData (Incubating)

incubator-predictionio

PredictionIO, a machine learning server for developers and ML engineers. Built on Apache Spark, HBase and Spray.

kafka-eagle

Used to monitor the consumer status of Kafka clusters, as well as offsets, metadata and other information.

kafka-elasticsearch-consumer

This application, Kafka ES Indexer, will read the messages from Kafka, processes (if needed) and batch index them into ElasticSearch.

kafka-python

Python client for Apache Kafka

kafka-spark-consumer

High Performance Kafka Consumer for Spark Streaming. Now Support Kafka 0.10

kafkaoffsetmanager

Pluggable Kafka offset manager for use in Spark streaming jobs

kylin-mondrian-interaction

Some information about Apache Kylin interaction with Pentaho Mondrian

kylo

Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on Apache Hadoop and Spark. Kylo is licensed under Apache 2.0 and contributed by Think Big, A Teradata Company

lambda-arch-spark

locktail

Lock tailing on your rotating files

logstash-patterns-core

mobike-crawler

摩拜单车爬虫

mt-flume

美团对 flume 的扩展和改进

shc

sparkstreaming_store_kafkatopicoffset_to_hbase

Kafka delivery semantics in the case of failure depend on how and when offsets are stored. Spark output operations are at-least-once. So if you want the equivalent of exactly-once semantics, you must either store offsets after an idempotent output, or store offsets in an atomic transaction alongside output.There is Spark Streaming how to store Kafka topic offset with HBase.

user-portrait

用户画像相关的参考代码