kubernetes / minikube

Run Kubernetes locally

minikube

minikube implements a local Kubernetes cluster on macOS, Linux, and Windows. minikube's primary goals are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit.

Features

minikube runs the latest stable release of Kubernetes, with support for standard Kubernetes features like:

• LoadBalancer - using minikube tunnel
• Multi-cluster - using minikube start -p <name>
• NodePorts - using minikube service
• Persistent Volumes
• Ingress
• Dashboard - minikube dashboard
• Container runtimes - minikube start --container-runtime
• Configure apiserver and kubelet options via command-line flags
• Supports common CI environments

As well as developer-friendly features:

• Addons - a marketplace for developers to share configurations for running services on minikube
• NVIDIA GPU support - for machine learning
• Filesystem mounts

For more information, see the official minikube website

Installation

See the Getting Started Guide

📣 Please fill out our fast 5-question survey so that we can learn how & why you use minikube, and what improvements we should make. Thank you! 👯

Documentation

See https://minikube.sigs.k8s.io/docs/

More Examples

See minikube in action here

Governance

Kubernetes project is governed by a framework of principles, values, policies and processes to help our community and constituents towards our shared goals.

The Kubernetes Community is the launching point for learning about how we organize ourselves.

The Kubernetes Steering community repo is used by the Kubernetes Steering Committee, which oversees governance of the Kubernetes project.

Community

minikube is a Kubernetes #sig-cluster-lifecycle project.

• #minikube on Kubernetes Slack - Live chat with minikube developers!

• minikube-users mailing list

• minikube-dev mailing list

• Contributing

• Development Roadmap

Join our community meetings:

• Bi-weekly office hours, Mondays @ 11am PST
• Triage Party

minikube's Issues

Switch to glide from godeps

See #33.

Glide gives more control in pinning the versions of dependencies, like k8s.io/kubernetes. It might make it easier to update to newer versions of k8s than godep currently does.

cc @ethernetdan @vishh @luxas
Thoughts?

Support downward APIs and Config Maps

Add a stop command

This should stop the cluster VM, but not delete any data.

Add multi-node support

To provide a complete kubernetes experience, user might want to play & experience with features like scheduling and daemon set, etc. If minikube can emulate multiple kubernetes nodes, users can then use most of the kubernetes features.

This is probably not necessary in the first few versions of minikube. Once the single node setup is stable, we can look at emulating multiple nodes.

Up votes from users on this issue can be used as a signal to start working on this feature.

Support creating a VM inside virtualbox on osx

Use iptables based proxy

Currently localkube is using the legacy userspace proxy - we should change it to use iptables.

There is a TODO.

Minikube should (build and) use the latest localkube binary

Allow configuration of Kube components

Currently there isn't a way to change localkube's configuration without recompiling. It would be useful if localkube could be configured using a set of YAML files.

Kube

• APIServer - api.yml
• KubeControllerManagerConfiguration - controller-manager.yml
• KubeletConfiguration - kubelet.yml
• KubeProxyConfiguration - proxy.yml
• KubeSchedulerConfiguration - scheduler.yml

Non-kube

• Config - skydns.yml
• ServerConfig - etcd.yml + etcd-skydns.yml

A potential issue is that the structs for API Server, SkyDNS, and Etcd don't have JSON tags.

It would be nice if these could be editable on the host computer (perhaps in ~/.minikube) and could be synced over to the VM or mounted.

Build and release iso

This should have a fixed version of docker and other dependencies. We can release it separately (via GCS download) or maybe use go-bindata to pack it all up into the CLI binary.

Proxy for easier access to NodePort services

One of the major hurdles people have using k8s as a development platform is having easy access to DNS and uncomplicated access to "localhost" ports.

This might be something we can tackle in this bug, I discussed the idea here: coreos/coreos-kubernetes#444

Option 1 - Fancy Proxy

This is an idea to make working with the single-node cluster easier. The basic idea would be to have something like kubectl port-forward that forwards every nodePort to localhost based on the original targetPort. So, for example:

# User does something like this
$ kubectl run --image quay.io/philips/golang-outyet outyet
$ kubectl expose deployment outyet --target-port=8080 --port=8080 --type=NodePort


# This is the part that needs automating: 
$ socat tcp-listen:8080,reuseaddr,fork tcp:172.17.4.99:$(kubectl get service outyet -o template --template="{{range.spec.ports}}{{.nodePort}}{{end}}")

This would be a huge boon to people trying to use kubernetes as a development workflow for running caches, services, etc and developing against those APIs.

Psuedo code event loop:

for {
  for _, e := range kubernetesServiceEvent() {
    if e == newNodePort {
      go proxy(context, e.NodePort, e.NodeIP, "localhost", e.TargetIP)
    } 
    if e == dyingNodePort {
      contexts[e.NodePort].Done()
    }
  } 
}

Option 2 - VPN

Having a simple VPN setup would allow a user to get access to cluster DNS and cluster networking. The downside here is that stuff like OpenVPN, etc is a major hurdle. Anyone know of a simple VPN in Go that works cross platform?

Add command to setup docker env variables, like $(docker-machine env)

Need a reliable and low latency local cluster setup for Kubernetes

Derived from kubernetes/kubernetes#24106

1. A new Kubernetes user should be able to setup a cluster locally in less than a minute.
2. A Kubernetes user should be able to test their application using the local cluster
3. For the features supported in the local cluster, environment in the local cluster must match that of production.

Update Kubernetes to HEAD

@dlorenc you mentioned you were working on this.

Good to know, as I just had been thinking about doing it too :)
Assigning that to you then.

Write a travis CI test to enforce copyright stanzas

Secure access to minikube

We should have secure access to minikube.
Even with a self signed cert, it should be https and with token access.

Use GOVENDOREXPERIMENT=1 from the start

Use godep with GOVENDOREXPERMIENT set to 1.
Store files in vendor/ (makes godep used only when adding/removing deps)
Also, require go1.6+ for building

Add a delete command

This should stop the cluster VM and delete all data associated with it.

Run localkube outside of a container in the guest VM.

Support mounting host directories into pods

Having the ability to mount local directories to pods will improve the developer experience a lot.

• Builds can be done in a local kube cluster
• Testing changes to an app will be much faster, without requiring creation of a docker container and a registry.

Given that most hypervisors can already support mapping host directories, adding this feature should be straightforward.

Compatible with Docker for Mac

Several Localkube users have requested support for Docker for Mac, instead of relying directly on VirtualBox. Wanted to open an issue to discuss whether this is something we want to pursue.

Generate basic auth credentials

This would make it easier to authenticate when accessing through a browser, e.g. accessing Dashboard.

ssh onto node

An issue with documentation I guess, how can I ssh onto the machine that minikube starts?

This is an issue when using private docker repos for example as I need to login to docker hub on the node that minikube will pull images on.

Support kubernetes dashboard.

Investigate using go instead of easyrsa to create certificates.

I tried using this but wasn't able to get certificates to validate: https://github.com/SvenDowideit/generate_cert/blob/master/generate_cert.go

Figure out testing strategy for localkube/k8s

We'll need to test localkube vs. kubernetes at HEAD (or as close to it as possible). This is non-trivial because localkube vendors and directly links in kubernetes packages.

One idea:

Have a CI job that updates all vendored kubernetes components from master daily and runs tests, then discards the changes. This job should alert on failures, which would indicate manual work will need to be done.

Weekly/monthly updates can be made and committed by an actual person.

cc @vishh Does this make sense?

Do not depend on localkube when released

Package/install/link in kubectl code

If users don't have kubectl on their path we should offer to install it somehow. We could link it into the minikube binary like monokube does or use an official build.

Add a status command

This should determine whether or not the cluster is running correctly.

too much memory during build with docker-machine

Hi,

I tried a build using a docker-machine based host on OSX.
Bumped the host to 2 GB ram on virtual box and the build keeps failing.

Status: Downloaded newer image for golang:1.6
CGO_ENABLED=1 go build -ldflags="-s" -o ./out/localkube ./cmd/localkube
# k8s.io/minikube/cmd/localkube
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: fork/exec /usr/bin/gcc: cannot allocate memory

It would be great to make the build without docker and/or to reduce the needed RAM.

-sebastien

Mechanism to notify users of cluster upgrades

When we release new cluster components, we need a mechanism to notify users that the upgrades are available.

cc @vishh

Support serviceaccounts

Without them DNS addon won't work (and lots of other things as well)
It is not an issue if we keep relying on weave in localkube.

Use glog for localkube logging and add a --v flag

Allow for privileged nodes

Ran into an issue today helping someone set up Pachyderm on localkube around privileged nodes. We should probably support these in minikube.

Provide installation script

Later we'll have full .dmg/deb/msi installers.

Pass conformance tests

https://coreos.com/kubernetes/docs/latest/conformance-tests.html

Or document known failures that we decide we can live with.

Multiple platform support

Server platforms for linux/amd64, linux/arm, linux/arm64, linux/ppc64le
Client platforms for windows/amd64, darwin/amd64 + the above

@dlorenc @vishh

Remove localkube dep on weave

Bundle the localkube binary with the release

Rather than fetching it from github, we should include a fixed copy of the localkube binary that will run in the VM. This can either be transferred in from the minikube CLI or baked into the VM image.

Add a minikube logs command to fetch all logs from the VM

This should support all relevant logs from inside the VM:
localkube, docker, etc.

Users should only run this to help debug issues.

Switch build.sh to a Makefile

Once the localkube merge lands (#39 ), the build is getting complicated enough to deserve a real makefile. We'll want to building localkube for a different OS than minikube in most cases, with different sets of flags (static vs. dynamic linking, etc.).

Unable to do port forwarding: socat not found

I'm trying to forward local port 5000 to a pod port 5000 with kubectl:

kubectl port-forward --namespace kube-system kube-registry-v0-4icv2_kube-system 5000:5000 &

But I get the following error:

an error occurred forwarding 5000 -> 5000: error forwarding port 5000 to pod kube-registry-v0-4icv2_kube-system, uid : unable to do port forwarding: socat not found

Labels for this repo

I just changed colors for the cla: xx labels to match with the official Kubernetes repo.

In this issue we should consider which labels we need.
Examples:

• priority/P[0-3]
• area/(localkube|minikube|dependencies|build-release|testing|??)
• help-wanted
• lgtm
• release-note
• machine/(windows|osx|linux|arm)
• kind/(bug|feature|documentation|support|??)

Also, we should remove those labels we aren't using, but are there by default
@dlorenc @vishh

Setup travis CI

DNS not working (or not enabled)?

Using localkube, SkyDNS was enabled however it does not seem to be setup on minikube:

[ root@curl-131556218-gq56k:/ ]$ nslookup kubernetes.default
Server:    10.1.30.3
Address 1: 10.1.30.3

nslookup: can't resolve 'kubernetes.default'
[ root@curl-131556218-gq56k:/ ]$

Is it not enabled? It seems like something that is quite commonly used. Is it a bug or me doing something wrong?

Release version 0.1.0

Once we've fixed all the issues in the 0.1 milestone we should do a build and release. We'll have to build and tag an ISO as well as the minikube binaries. Right now it's only been tested on Mac and Linux, so we should stick to those. Windows support can come later.

Localkube should vendor kube-dns instead of the current kube2sky solution

Ref: kubernetes/kubernetes#23930

Support for Windows

Minikube should eventually run on Windows.

certificate signed by unknown authority

I was using minikube, then I started using an AWS cluster then switched back to minikube and now I get this :(

➜  minikube git:(master) kubectl cluster-info
error: couldn't read version from server: Get https://192.168.99.100:443/api: x509: certificate signed by unknown authority

I don't really understand what's going on here?

Update vendored libmachine.

Automatically update the kubeconfig after starting a cluster.