This is a testing project for build a Big Data Ecosystem on top of OpenShift Origin.
I this project we explore the different alternatives to run Big Data workloads in OpenShift Origin, the foundations for BDaaS - Big Data as a Service, agile development cycle (devops), rapid development, and so forth.
Rapid artifact testing of this project with the new oc cluster up command from v1.3.0-alpha.3 OpenShift Origin version:
-
Install Docker with your platform's package manager:
(Fedora 24) $ sudo dnf install docker
-
Configure the Docker daemon with an insecure registry parameter of
172.30.0.0/16
In RHEL and Fedora, edit the/etc/sysconfig/docker
file and add or uncomment the following line:INSECURE_REGISTRY='--insecure-registry 172.30.0.0/16' $ sudo systemctl restart docker
-
Download the Linux
oc
binary from openshift-origin-client-tools-v1.3.0-alpha.3-7998ae4-linux-64bit.tar.gz -
Open a terminal with a user that has permission to run Docker commands and run:
$ oc cluster up [...] OpenShift server started. The server is accessible via web console at: https://192.168.43.137:8443 You are logged in as: User: developer Password: developer To login as administrator: oc login -u system:admin
-
To stop your cluster, run:
$ oc cluster down
Note: By default, etcd data will not be preserved between container restarts. If you wish to preserve your data, specify a value for --host-data-dir and the --use-existing-config flag.
Default routes are setup using xip.io and the host ip of your cluster. To use a different routing suffix, use the --routing-suffix flag.
The following is a table with the working status for each tool.
The table is based on the following workflow:
Tool | Overview | Docker | OpenShift manually | Template |
---|---|---|---|---|
Apache Flume | Done | Done | Done | Done |
Apache Zookeeper | Done | Done | Done | Done |
Apache Solr | Done | TODO | TODO | TODO |
Banana Fusion | Done | TODO | TODO | TODO |
Apache Kafka | Done | TODO | TODO | TODO |
MongoDB | Done | Done | TODO | TODO |
TensorFlow | Done | TODO | TODO | TODO |
LogStash | Done | TODO | TODO | TODO |
If you would like to contribute, please take a look to the workflow we are following: