GithubHelp home page GithubHelp logo

wonghulin / flinkk8soperator Goto Github PK

View Code? Open in Web Editor NEW

This project forked from lyft/flinkk8soperator

0.0 2.0 0.0 9.88 MB

Kubernetes operator that provides control plane for managing Apache Flink applications

License: Apache License 2.0

Dockerfile 0.65% Makefile 0.34% Shell 3.99% Go 94.11% Java 0.92%

flinkk8soperator's Introduction

Current Release Build Status GoDoc License CodeCoverage Go Report Card Commit activity Commit since last release Slack

Flinkk8soperator

FlinkK8sOperator is a Kubernetes operator that manages Flink applications on Kubernetes. The operator acts as control plane to manage the complete deployment lifecycle of the application.

Project Status

Beta

The operator is in use for some less-critical jobs at Lyft. At this point the focus is on testing and stability. While in Beta, we will attempt to limit the number of backwards-incompatible changes, but they may still occur as necessary.

Prerequisites

  • Version >= 1.10 of Kubernetes (versions < 1.13 require --feature-gates=CustomResourceSubresources=true)
  • Version >= 1.7 of Apache Flink.

Overview

Flink operator overview

The goal of running Flink on Kubernetes is to enable more flexible, lighter-weight deployment of streaming applications, without needing to manage infrastructure. The Flink operator aims to abstract out the complexity of hosting, configuring, managing and operating Flink clusters from application developers. It achieves this by extending any kubernetes cluster using custom resources.

The Operator creates flink clusters dynamically using the specified custom resource. Flink clusters in kubernetes consist of the following:

Deploying and managing Flink applications in Kubernetes involves two steps:

  • Building Flink application packaged as a docker image: A docker image is built containing the application source code with the necessary dependencies built in. This is required to bootstrap the Jobmanager and Taskmanager pods. At Lyft we use Source-To-Image S2I as the image build tool that provides a common builder image with Apache Flink pre-installed. The docker image could be built using any pre-existing workflows at an organization.

  • Creating the Flink application custom resource: The custom resource for Flink application provides the spec for configuring and managing flink clusters in Kubernetes. The FlinkK8sOperator, deployed on Kubernetes, continuously monitors the resource and the corresponding flink cluster, and performs actions based on the diff.

Documentation

Community

We welcome you to contribute and make the operator better! For questions, and changes please create an issue or submit a pull request.

flinkk8soperator's People

Contributors

alberto-quesada-scopely avatar anandswaminathan avatar annymsmthd avatar glaksh100 avatar haltwise avatar itstemo avatar jmdacruz avatar jonnyincognito avatar kelly-sm avatar lrao100 avatar maghamravi avatar mwylde avatar nasdin avatar premsantosh avatar rjbordon avatar tweise avatar yuchaoran2011 avatar yuvalitzchakov avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.