crc-org / crc

CRC is a tool to help you run containers. It manages a local OpenShift 4.x cluster, Microshift or a Podman VM optimized for testing and development purposes

CRC - Runs Containers

• Introduction
• Usage data
• Documentation
• Community

Introduction

crc is a tool to run containers. It manages a local OpenShift 4.x cluster, or an OKD cluster VM optimized for testing and development purposes.

If you are looking for a solution for running OpenShift 3.x, you will need tools such as oc cluster up, Minishift or CDK.

Usage data

The first time CRC is run, you will be asked to opt-in to Red Hat’s telemetry collection program.

With your approval, CRC collects pseudonymized usage data and sends it to Red Hat servers to help improve our products and services. Read our privacy statement to learn more about it. For the specific data points being collected, see Usage data.

Manually configuring usage data collection

You can manually change your preference about usage data collection by running crc config set consent-telemetry <yes/no> before the next crc start.

Documentation

Getting CRC

CRC binaries with an embedded OpenShift disk image can be downloaded from this page.

Using CRC

The documentation for CRC is currently hosted by GitHub Pages.

See the CRC Getting Started Guide.

Building the documentation

You can find the source files for the documentation in the docs directory.

To build the formatted documentation, install podman then use the following:

$ git clone https://github.com/crc-org/crc
$ cd crc
$ make build_docs

This will create a docs/build/master.html file which you can view in your browser.

Developing CRC

Developers who want to work on CRC should visit the Developing CRC document.

Community

Contributions, questions, and comments are all welcomed and encouraged!

You can reach the community by:

• Joining the #codeready channel on Freenode IRC

If you want to contribute, make sure to follow the contribution guidelines when you open issues or submit pull requests.

crc's Issues

Add Gherkin tests for testing CLI functionality

I have done quick spike last week whether it would be possible to port parts of Minishift integration tests into CRC and it went successfully. For that purpose I have used the shell runner which was part of Minishift tests, which opens shell instance and runs all test commands in that instance.

It needs some polishing and few additions like - how to secure that CRC binary will be first on the path, but overall it should not take much time and is nearly ready. WDYT @LalatenduMohanty, @gbraad ?

Add basic pre-flight checks and setup (macOS / VirtualBox)

Depends on: #9

Which checks are needed for macOS (VB);

• virtualbox installed
• virtualbox usable
• ... and so on

Update the project readme

Implement status command

Find a way to disable the 'Monitoring' section on the console/dashboard

Since most of the sections for monitoring and the link to metrics do not work, it is probably best to find a way to remove/disable this section from the dashboard.

Add basic pre-flight checks and setup (Windows / Virtualbox)

Depends on #9

Implement pre-flight checks and setup related to use of Virtualbox on Windows

• Check if Virtualbox installed
• Check if configured
• ...

Makefile to compile the existing source code

Add check for boilerplate in each files

Need to add check for boilerplate around the licensing in our CI for each files

/*
Copyright (C) 2019 Red Hat, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

[Epic] Native installers

Installers for platforms to handle the setup and integration with the target platform.

• Windows (#1516)
• macOS (#1516)
• Linux

Getting started guide

We need a Getting Started Guide to help the user to get up and running in no time.

• installation of the needed crc-driver-libvirt driver plugin
  • actually handled by the crc setup command
• basic usage crc start
• ... note: at this point no other commands exist yet (see command roadmap #70 )
• use with oc (export of kubeconfig)

Documentation structure

Since we know we will release alongside of OpenShift our release schedule is known, however at this point it is not clear what kind of documentation is expected and where this is published. I would suggest to focus first on a getting started guide and then follow-up with detailed documentation.

• Getting started guide #65
• contribution guidelines
• test strategy (needs input from @tsedmik @jsliacan)
• ...

Note: It is best to split this out in several expected tasks. @robin-owen, please do so where you see fit.

Add a contribution guideline

Add a contribution guideline similar to Minishift.

[1] https://github.com/minishift/minishift/blob/master/CONTRIBUTING.adoc

Modify KVM driver to create/use created network overlay

Allow the creation of the network using static MAC address and a fixed IP address.

So at this moment we assume the VM will allows uses the same MAC address and the expected IP address is 192.168.126.11.

Current solution is to have the pre-flight/setup code handle this #6

Automated generation of images

Auto generate images non-interactive, as snc*, based on either tagged release or master.

• generate image => https://github.com/praveenkumar/snc/blob/master/createdisk.sh
• use branch/tag indicator
• collect and store metadata #34
• publish artifacts

Add documentation around how to build CRC for development

We need a documentation around steps around how build CRC locally, so that anyone can compile the code locally i.e. developing.adoc. It should be in the same line as https://docs.okd.io/latest/minishift/contributing/developing.html

[Spike] Investigate if posible to run coreDNS on another port

When running coredns on a different port we encounter issues when Libvirt's default network is enabled; dnsmasq claims port 53 on 0.0.0.0. Although socket reuse is enabled, coredns does not bind correctly and therefore fails to run.

When running on any other port than 53, we can not use the libc resolver, as unlike macOS, it will not perform queries on ports other than 53. This spike is to investigate if anorther way is possible to enable running both dnsmasq and coredns (like other port), but using iptables, etc, etc.

Add basic command structure

Add CI to create disk image

Disk image which is consumed by crc should have a CI in place and the scripts for this need to placed either in this repo or in a separate one.

Add CI for make, make cross and make test

We need to add e.g. travis for test our PRs for make, make cross

• Add Makefile
• Travis integration

Modify VirtualBox driver for use with disk image

Focus is on the macOS implementation for this task.

Generate documentation

We need to generate documentation from the source files. We prefer to use a container for this, however... what will this look like? What source format will be used? Etc.

I suspect we will go with AsciiDoc again, but hopefully with a lightweight container that can be generated easily (container pipeline?)

Add go dependency management system

To implement make cross build we need a third party dependency, refer https://github.com/code-ready/crc/pull/20/files#r265006752. So planning to add https://github.com/golang/dep to the project.

Commands can be configured to print output in json

CRC should be able to give command output in json if required.

Add basic pre-flight checks and setup (macOS / Hyperkit)

Currently impeded due to issue with the image when running on Hyperkit, see crc-org/osp4#9, openshift/os#376

Depends on #9

• check if hyperkit available
• check if system correcttly configured
• ...

Add basic pre-flight checks and setup (Linux / Libvirt)

Depends on: #9

Which checks are needed for Linux; libvirt?

Add basic pre-flight checks and setup (Windows / Hyper-V)

Depends on: #9

Put vendor directory inside the repo instead always fetch using go mod.

We should have vendor directory inside our repo which helpful for us to do local make just after fetching the repo and also in case some upstream repo which we depend on disappear (like we saw during the minishift development) have it local copy helps.

Add version sub-command

Implement stop command

Retrigger certificate rotation/generation for internal cluster communication

Blocked by; openshift/api#199

At the moment, this renders the cluster install useless for distribution, as the expired certificates prevent communication.

Add basic abstractions

Add abstractions for logging and output print (from structure)

possibly print json -> #13

Modify machine-drivers

Modify our fork to:

• remove the ISO usage
• remove provisioning
• remove unneeded drivers

Add config abstraction

For use with #6

Adding DNS solution as part of pre-, post- startup

This depends on #44. This would be to provide a solution for resolving hostnames that are needed from the cluster. For instance, it is possible to embed coreDNS as part of the binary and fork off duirng the pre-start.

[EPIC] integrate pre-flight and setup code

Introduce pre-flight and setup code for all platforms and hypervisors we support:

Stage 1

• Linux (Libvirt) #6
• macOS (VirtualBox) #7

Stage 2

• Windows (Hyper-V) #8
• macOS (Hyperkit) #46

Stage 3

* [ ] Linux (VirtualBox) #47
* [ ] Windows (VirtualBox) #48

Allow the use of pre- and post- hooks during the VM startup

This is used to start dependent components that are needed for the cluster to operate correctly. Eg. on macOS and Windows we need to provide a DNS solution that resolves addresses for the OpenShift cluster domain to the VM. For this to work, we first need to fork a DNS process and after the IP address is known (unless statically provided) we need to set the IP address in the zonefile for this domain.

Implement

• pre-hook before VM startup
• post-hook after VM startup
  • pass struct with basic VM info (hostname, IP, etc)

Add basic pre-flight checks and setup (Linux / Virtualbox)

Depends on #9

Implement pre-flight checks and setup related to use of Virtualbox on Linux

• Check if Virtualbox installed
• Check if configured
• ...

[EPIC] Machine-drivers integration

Stage 1

• modify forked machine library #51
• Modify forked KVM driver
  • modify disk image use #49
  • modify network use #50
• modify forked VirtualBox driver
  • modify disk image use #52

DNS queries not handled in timely fashion

The cluster can not always be reached by oc. The following can be seen:

$ dig api.test1.tt.testing                

; <<>> DiG 9.11.5-P4-RedHat-9.11.5-4.P4.fc29 <<>> api.test1.tt.testing
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46459
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;api.test1.tt.testing.        IN    A

;; ANSWER SECTION:
api.test1.tt.testing.    0    IN    A    192.168.126.11

;; Query time: 2092 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: do mrt 21 12:57:10 CST 2019
;; MSG SIZE  rcvd: 65

$ oc get pods --all-namespaces > /tmp/out
No resources found.
Unable to connect to the server: dial tcp 173.230.157.229:6443: i/o timeout

The DNS request takes about 3 seconds to be handled.
When this happens, dnsmasq on the host reports:

Maximum number of concurrent DNS queries reached (max: 150)

Modify KVM driver for use with disk image

Logging framework for CRC

We need a logging framework for CRC which will enable us to collect the logs in a log file. This will be beneficial for many reasons, such as

• It should help system tray users when CRC needs to notify about some error and user should be able to able to find the details logs.
• It would help issue tracking and debugging for failures easier.
• It will simplify the code in log run i.e. we do not need to add print statements for printing it to console.

Here are some examples of logging framework

[1] https://github.com/sirupsen/logrus
[2] https://github.com/uber-go/zap

Cluster bundle handling code for virtual machine creation

Depends on #53 and #34

• extract logic (pkg/crc/machine/bundle)
• crc start passing selected bundle to driver (cmd/start)
• modify the hypervisor struct to contain bundle location (pkg/crc/machine)
• driver calling extract logic on (PreCreate, crc-driver-libvirt)

Setup single-VM during/post installer

Fix typo in logging package

the logging.ErrorF() func should be calling logrus.Errorf but is calling logrus.Error

https://github.com/code-ready/crc/blob/a5ed94364b04a6c810b2ef0bbaa5621872ab70ce/pkg/crc/logging/logging.go#L123-L125

Metadata and filenames describing the OpenShift images

When images are generated for nightly, we need to generate a metadata file containing the rhcos version, openshift release, etc. We also need a fixed naming scheme, as these are needed when unpacking from cache to the vm storage.

Add errors abstraction

Had already worked on this as part of the #12, but can already push this as it solves some of the issues with other PRs

Implement delete command

Put a testsuite file which runs on the openshift CI and also our going to be CRC CI

As of now we are not able to run the complete openshift/conformance/parallel as part of testsuite so we have identified around ~130 test cases which actually passed on the openshift CI for libvirt provider. We need to keep that test file in our repo so that we have control over to add more tests over period of time.

This file we are going to use in the openshift CI side mostly in the https://github.com/openshift/release/blob/master/ci-operator/templates/openshift/installer/cluster-launch-installer-libvirt-e2e.yaml template.

Handle pull-secret

Add crc base dir as ~/.crc