Operator to manage the lifecycle of Quay.
This repository contains the functionality to provision a Quay ecosystem. Quay is supported by a number of other components, and thus a CustomResourceDefinition called QuayEcosystem is used to define the entire architecture.
The following components are supported to be maintained by the Operator:
- Quay Enterprise
- Redis
- PostgreSQL
The Quay Operator can be deployed on OpenShift or Kubernetes platforms. Quay recommends that it by deployed in a namespace called quay-enterprise, however support is available for deploying the operator to a namespace of your choosing. The steps below illustrate the steps necessary to deploy to an OpenShift or Kubernetes environment.
When choosing a namespace other than quay-enterprise, the namespace field in the deploy/cluster_role_binding.yaml must be updated with the new namespace otherwise permission issues will occur when deploying in OpenShift.
The steps described below assume the namespace that will be utilized is called quay-enterprise.
$ oc new-project quay-enterprise
Deploy the cluster resources. Given that a number of elevated permissions are required to resources at a cluster scope the account you are currently logged in must have elevated rights
$ oc create -f deploy/crds/redhatcop.redhat.io_quayecosystems_crd.yaml
$ oc create -f deploy/service_account.yaml
$ oc create -f deploy/cluster_role.yaml
$ oc create -f deploy/cluster_role_binding.yaml
$ oc create -f deploy/role.yaml
$ oc create -f deploy/role_binding.yaml
$ oc create -f deploy/operator.yaml
Note: When running on OpenShift 3.x, an alternate Custom Resource Definition (CRD) than the file specified MUST be used. Otherwise an OpenAPI error will be produced
$ oc create -f deploy/crds/redhatcop.redhat.io_quayecosystems_crd-3.x.yaml
The steps described below assume the namespace that will be utilized is called quay-enterprise.
$ kubectl create namespace quay-enterprise
Deploy the cluster resources. Given that a number of elevated permissions are required to resources at a cluster scope the account you are currently logged in must have elevated rights
$ kubectl -n quay-enterprise create -f deploy/crds/redhatcop.redhat.io_quayecosystems_crd.yaml
$ kubectl -n quay-enterprise create -f deploy/service_account.yaml
$ kubectl -n quay-enterprise create -f deploy/cluster_role.yaml
$ kubectl -n quay-enterprise create -f deploy/cluster_role_binding.yaml
$ kubectl -n quay-enterprise create -f deploy/role.yaml
$ kubectl -n quay-enterprise create -f deploy/role_binding.yaml
$ kubectl -n quay-enterprise create -f deploy/operator.yaml
The Quay Operator can also be deployed as a Helm chart.
Create a new OpenShift project or Kubernetes Namespace
$ kubectl create namespace quay-enterprise
To install the chart run the following command:
Add the helm repository:
$ helm repo add quay-operator https://redhat-cop.github.io/quay-operatorInstall the chart
$ helm install quay-operator quay-operator/quay-operatorCreate a pull secret to retrieve Quay images from quay.io. If unsure what to use for the pull secret see Accessing Red Hat Quay (formerly Quay Enterprise) without a CoreOS login.
$ oc create secret generic redhat-pull-secret --from-file=".dockerconfigjson=<location of docker.json file>" --type='kubernetes.io/dockerconfigjson'
Create a custom resource to deploy the Quay ecosystem. The following is an example of a QuayEcosystem custom resource to support a deployment of the Quay ecosystem
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
You can also run the following command to create the QuayEnterprise custom resource
$ oc create -f deploy/crds/redhatcop_v1alpha1_quayecosystem_cr.yaml
This will automatically create and configure Quay Enterprise, PostgreSQL and Redis. The credentials that can be used to access Quay Enterprise is described in the following section.
There are several locations within the Quay ecosystem of tools where sensitive information is stored/used. The Operator supports credentials either provided by the user as Secrets, or using defaults.
During the installation process, a superuser is created in Quay Enterprise with administrative rights. The credentials for this user can be specified or leverage the following default values:
Username: quay
Password: password
Email: [email protected]
To specify an alternate value, create a secret as shown below:
oc create secret generic <secret_name> --from-literal=superuser-username=<username> --from-literal=superuser-password=<password> --from-literal=superuser-email=<email>
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
superuserCredentialsSecretName: <secret_name>
imagePullSecretName: redhat-pull-secret
A dedicated deployment of Quay Enterprise is used to manage the configuration of Quay. Access to the configuration interface is secured and requires authentication in order for access. By default, the following values are used:
Username: quayconfig
Password: quay
While the username cannot be changed, the password can be defined within a secret. To define a password for the Quay Configuration, execute the following command to create a secret:
oc create secret generic <secret_name> --from-literal=config-app-password=<password>
Reference the name of the secret in the QuayEcosystem custom resource as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
configSecretName: <secret_name>
imagePullSecretName: redhat-pull-secret
Note: The superuser password must be at least eight characters.
The PostgreSQL relational database is used as the persistent store for Quay Enterprise. PostgreSQL can either be deployed by the operator within the namespace or leverage an existing instance. The determination of whether to provision an instance or not within the current namespace depends on whether the server property within the QuayEcosystem is defined.
The following options are a portion of the available options to configure the PostgreSQL database:
| Property | Description |
|---|---|
image |
Location of the database image |
volumeSize |
Size of the volume in Kubernetes capacity units |
Note: It is important to note that persistent storage for the database will only be provisioned if the volumeSize property is specified when provisioned by the operator.
Define the values as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
database:
volumeSize: 10Gi
The credentials for accessing the server can be specified through a Secret or when being provisioned by the operator, leverage the following default values:
Username: `quay`
Password: `quay`
Root Password: `quayAdmin`
Database Name: `quay`
To define alternate values, create a secret as shown below:
oc create secret generic <secret_name> --from-literal=database-username=<username> --from-literal=database-password=<password> --from-literal=database-root-password=<root-password> --from-literal=database-name=<database-name>
Reference the name of the secret in the QuayEcosystem custom resource as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
database:
credentialsSecretName: <secret_name>
Instead of having the operator deploy an instance of PostgreSQL in the project, an existing instance can be leveraged by specifying the location in the server field along with the credentials for access as described in the previous section. The following is an example of how to specify connecting to a remote PostgreSQL instance
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
database:
credentialsSecretName: <secret_name>
server: postgresql.databases.example.com
Quay supports multiple storage backends (configured as an array). The quay operator supports aiding in the facilitation of certain storage backends. The following backends are currently supported:
Please refer to the Registry Storage Documentation for the options available.
Quay provides the capability to create container image repositories that exactly match the content of external registries. This functionality can be enabled by setting the enableRepoMirroring: true as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
enableRepoMirroring: true
The following additional options are also available:
repoMirrorTLSVerify- Require HTTPS and verify certificates of Quay registry during mirrorrepoMirrorServerHostname- URL for use by the skopeo copy commandrepoMirrorEnvVars- Environment variables to be applied to the repository mirror containerrepoMirrorResources- Compute resources to be applied to the repository mirror container
Files related to the configuration of Quay are located in the /conf/stack directory. There are situations for which additional user defined configuration files need to be added to this directory (such as certificates and private keys). The Quay Operator supports the injection of these assets within the configFiles property in the quay property of the QuayEcosystem object where one or more assets can be specified.
Two types of configuration files can be specified by the type property:
config- Configuration files that will be added to the/conf/stackdirectoryextraCaCerts- Certificates to be trusted container
Configuration files are stored as values within Secrets. The first step is to create a secret containing these files. The following command illustrates how a private key can be added:
oc create secret generic quayconfigfile --from-file=<path_to_file>
With the secret created, the secret containing the configuration file can be referenced in the QuayEcosystem object as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
configFiles:
- secretName: quayconfigfile
By default, the config type is assumed. If the contents of the secret contains certificates that should be added to the extra_ca_certs directory, specify the type as extraCaCert as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
configFiles:
- secretName: quayconfigfile
type: extraCaCert
Individual keys within a secret can be referenced to fine tune the resources that are added to the configuration using the files property as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
configFiles:
- secretName: quayconfigfile
files:
- key: myprivatekey.pem
filename: cloudfront.pem
- key: myExtraCaCert.crt
type: extraCaCert
The example above assumes that two files have been added to a secret called quayconfigfile. The file myprivatekey.pem that was added to the secret will be mounted in the quay pod at the path /conf/stack/cloudfront.pem since it is a config file type and specifies a custom filename that should be projected into the pod. The myExtraCaCert.crt file will be added to the Quay pod within at the path /conf/stack/extra_certs/myExtraCert.crt
Note: The type property within files property overrides the value in the configFiles property.
The operator by default is configured to complete the automated setup process for Quay. This can be bypassed by setting the skipSetup field to true as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
skipSetup: true
Quay, as a secure registry, makes use of SSL certificates to secure communication between the various components within the ecosystem. Transport to the Quay user interface and container registry is secured via SSL certificates. These certificates are generated at startup with the OpenShift route being configured with a TLS termination type of Passthrough.
SSL certificates can be provided and used instead of having the operator generate certificates. Certificates can be provided in a secret which is then referenced in the QuayEcosystem custom resource.
The secret containing custom certificates must define the following keys:
tls.cert- All of the certificates (root, intermediate, certificate) concatinated into a single filetls.key- Private key as for the SSL certificate
Create a secret containing the certificate and private key
oc create secret tls custom-quay-ssl --key=tls.key=<ssl_private_key> --cert=tls.cert=<ssl_certificate>
The secret containing the certificates are referenced using the sslCertificatesSecretName proprety as shown below
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
sslCertificatesSecretName: custom-quay-ssl
Quay makes use of an OpenShift route to enable ingress. The hostname for this route is automatically generated as per the configuration of the OpenShift cluster. Alternatively, the hostname for this route can be explicitly specified using the hostname property under the quay field as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
hostname: example-quayecosystem-quay-quay-enterprise.apps.openshift.example.com
imagePullSecretName: redhat-pull-secret
Support is available to access Quay through a number of OpenShift and Kubernetes mechanisms for ingress. When running on OpenShift, a Route is used while a LoadBalancer Service is used.
The type of external access can be specified by setting the externalAccessType using one of the available options in the table below:
| External Access Type | Description | Notes |
|---|---|---|
Route |
OpenShift Route | Can only be specified when running in OpenShift |
LoadBalancer |
LoadBalancer Service | |
NodePort |
NodePort Service | A dns based hostname or IP address must be specified using the hostname property of the quay resource |
An example of how to specify the externalAccessType is shown below
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
externalAccessType: LoadBalancer
By default, NodePort type Services are allocated a randomly assigned network port between 30000-32767. To support a predictive allocation of resources, the NodePort services for Quay and Quay Config can be define using the nodePort and configNodePort as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
externalAccessType: NodePort
nodePort: 30100
configNodePort: 30101
During the development process, you may want to test the provisioning and setup of Quay Enterprise server. By default, the operator will use the internal service to communicate with the configuration pod. However, when running external to the cluster, you will need to specify the ingress location for which the setup process can use.
Specify the configHostname as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
configHostname: example-quayecosystem-quay-config-quay-enterprise.apps.openshift.example.com
imagePullSecretName: redhat-pull-secret
In order to conserve resources, the configuration deployment of Quay is removed after the initial setup. In certain cases, there may be a need to further configure the quay environment. To specify that the configuration deployment should be retained, the keepConfigDeployment property within the Quay object can can be set as true as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
keepConfigDeployment: true
By default, the operator managed Redis instance is deployed without a password. A password can be specified by creating a secret containing the password in the key password. The following command can be used to create the secret:
oc create secret generic <secret_name> --from-literal=password=<password>
The secret can then be specified within the redis section using the `` as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
redis:
credentialsSecretName: <secret_name>
imagePullSecretName: redhat-pull-secret
Clair is a vulnerability assessment tool for application container. Support is available to automatically provision and configure both Clair and the integration wtih Quay. A property called clair can be specified in the QuayEcosystem object along with enabled: true within this field in order to deploy Clair. An example is shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
clair:
enabled: true
imagePullSecretName: redhat-pull-secret
Clair routinely queries CVE databases in order to build its own internal database. By default, this value is set at 500m. You can modify the time interval between checks by setting the updateInterval property as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
clair:
enabled: true
imagePullSecretName: redhat-pull-secret
updateInterval: "60m"
The above configuration would have Clair update every 60 minutes.
Each of the following components expose a set of similar properties that can be specified in order to customize the runtime execution:
- Quay
- Quay Configuration
- Quay PostgreSQL
- Redis
- Clair
- Clair PostgreSQL
As referenced in prior sections, an Image Pull Secret can specify the name of the secret containing credentials to an image from a protected registry using the property imagePullSecret.
There may be a desire to make use of an alternate image or source location for each of the components in the Quay ecosystem. The most common use case is to to make use of an image registry that contains all of the needed images instead of being sourced from the public internet. Each component has a property called image where the location of the related image can be referenced from.
The following is an example of how a customized image location can be specified:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
image: myregistry.example.com/quay/quay:v3.2.0
Compute Resources such as memory and CPU can be specified in the same form as any other value in a PodTemplate. CPU and Memory values for Requests and Limits can be specified under a property called resources.
Note: In the case of the QuayConfiguration deployment, configResources is the property which should be referenced underneath the Quay property.
The following is an example of how compute resources can be specified:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
resources:
requests:
memory: 512Mi
Readiness and Liveness Probes can be specified in the same form as any other value in a PodTemplate.
The following is how a readinessProbe and livenessProbe can be specified:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
livenessProbe:
initialDelaySeconds: 120
httpGet:
path: /health/instance
port: 8443
scheme: HTTPS
readinessProbe:
initialDelaySeconds: 10
httpGet:
path: /health/instance
port: 8443
scheme: HTTPS
Note: If a value for either property is not specified, an opinionated default value is applied.
It may be desired that components of the QuayEcosystem may need to be deployed to only a subset of available nodes in a Kubernetes cluster. This functionality can be set on each of the resources using the nodeSelector property as show below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
nodeSelector:
node-role.kubernetes.io/infra: true
Each of the core components consist of Kubernetes Deployments. This resource supports the method in which new versions are released. This operator supports making use of the RollingUpdate and Recreate strategies. Either value can be defined by using the deploymentStrategy property on the desired resource as shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
deploymentStrategy: RollingUpdate
Note: The absence of a defined value will make use of the RollingUpdate strategy
In addition to environment variables that are automatically configured by the operator, users can define their own set of environment variables in order to customize the managed resources. Each core component includes a property called envVars where environment variables can be defined. An example is shown below:
apiVersion: redhatcop.redhat.io/v1alpha1
kind: QuayEcosystem
metadata:
name: example-quayecosystem
spec:
quay:
imagePullSecretName: redhat-pull-secret
envVars:
- name: FOO
value: bar
Note: Environment variables for the Quay configuration pod can be managed by specifying the configEnvVars property on the quay resource
Caution: User defined environment variables are given precedence over those managed by the operator. Undesirable results may occur if conflicting keys are used.
To resolve issues running, configuring and utilzing the operator, the following steps may be utilized:
The QuayEcosystem custom resource will attempt to provide the progress of the status of the deployment and configuration of Quay. Additional information related to any errors in the setup process can be found by viewing the log messages of the config pod as shown below:
oc logs $(oc get pods -l=quay-enterprise-component=config -o name)
Execute the following steps to develop the functionality locally. It is recommended that development be done using a cluster with cluster-admin permissions.
Clone the repository, then resolve all dependencies using go mod
$ export GO111MODULE=on
$ go mod vendor
Using the operator-sdk, run the operator locally
$ operator-sdk up local --namespace=quay-enterprise
Execute the following steps to upgrade an existing deployment to a new version without upgrading the operator:
oc edit quayecosystem/quayecosystem
Find and update the following entries:
image: quay.io/redhat/clair-jwt:vX.X.X
image: quay.io/redhat/quay:vX.X.X
Once saved, the operator will automatically apply the upgrade.
Note: If you used a different name than QuayEcosystem for the custom resource to deploy your Quay ecosystem, you will have to replace the name to fit the proper value
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent