sabre1041 / openshift-flexvolume-cifs

FlexVolume driver for access CIFS based shares

openshift-flexvolume-cifs

FlexVolume driver for access CIFS based shares

Overview

FlexVolume's provide a method for users to create their own drivers for adding options for utilizng persistent storage in an OpenShift and Kubernetes environment. This driver allows for the creation of a PersistentVolume or direct mount to access storage that is exposed by a CIFS share.

Setup

Utilize either one of the following methods to configure FlexVolume support in your environment

Manual setup

Installation and configuration of the CIFS FlexVolume driver can be completed manually or in an automated fashion using Ansible.

1. Install required packages

yum install cifs-utils

2. Install the driver

The cifs driver needs to be placed on all nodes that applications leveraging the FlexVolume driver will be mounted. The drivers that are used to support FlexVolume's are placed within the /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ directory. Within this directory, drivers are stored by vendor name and driver name in the format <vendor>~<driver>. Tehe vendor for this driver is openshift.io and the driver name is called cifs. Create a directory called /usr/libexec/kubernetes/kubelet-plugins/volume/exec/openshift.io~cifs and place the cifs driver executable within it

3. Restart the OpenShift Services

On all nodes the driver was configured on, restart the Atomic Node service

systemctl restart atomic-openshift-node

Ansible based environment setup

The installation process for the FlexVolume driver can be automated through the use of an Ansible playbook located in the ansible directory.

Configure the inventory

An example inventory file called inventory is available in the ansible directory. The inventory groups align with the group names utilized in the OpenShift installation meaning that inventories created for existing environments can be utilized. Otherwise, fill out the nodes group with the instances that are desired to be configured with the CIFS FlexVolume driver.

Run the playbook

Run the Ansible playbook executing the following command within the ansible directory.

ansible-playbook -i inventory setup-openshift.yml

Implementation

The following steps will not only walk through some of the available options provided by the CIFS driver, but showcase how one can utilize it in your environment.

Secrets

In many cases, access to CIFS shares will be protected and require authentication. Credentials to access CIFS volumes can be placed in a secret and then referenced in the FlexVolume definition.

A secret can either be manually created or an OpenShift template can be processed to create a secret.

To manually create the required secret, create a file called flexvolume-cifs-secret.yml with the following contents

apiVersion: v1
kind: Secret
metadata:
  name: cifs-credentials
type: openshift.io/cifs
stringData:
  username: <username>
  password: <password>

Replace <username> and <password> with the actual values

Create or utilize an existing namespace and apply the previously created secret file

oc apply -f secret.yml

Instead of manually creating a secret file and then applying it to the cluster, an existing template file called credentials-secret-template.yml can be utilized from the examples folder.

The template requires that a username and password be provided as parameters.

Execute the following command to instantiate the template

oc process -p USERNAME=<username> -p PASSWORD=<password> -f examples/cifs-secret-template.yml | oc apply -f-

A new secret called cifs-credentials will be created

NOTE: In addition to username and password, the domain associated with the CIFS share can also be specified. If desired, add the domain key to the secret.

A credentials file will be located one directory above the location where the volume is mounted in the format `.<MOUNT_NAME>.credentials

Security Context Contraints

By default, all pods created by typical end users make use of the restricted Security Context Constraint. FlexVolumes are not one of the allowed volume types. Instead of modifying the existing restricted SCC, it is recommended that a custom SCC be created instead. An example of an SCC based upon the restricted SCC with support for FlexVolume's is provided in the examples folder called restricted-flexvolume-scc.yml.

As a cluster administrator, apply the SCC to the cluster.

oc apply -f examples/restricted-flexvolume-scc.yml

With the SCC in place, grant the service account that will be used by applications access to the newly created SCC. The following example describes how to grant the default service account to the custom SCC.

oc adm policy add-scc-to-user restricted-flexvolume -z default

FlexVolume Definition

As previously described, FlexVolume definitions can be placed either on a PersistentVolume or directly on the object. An example is defined below:

flexVolume:
  driver: openshift.io/cifs
  fsType: cifs
  options:
    mountOptions: 'file_mode=0770,dir_mode=0770'
    networkPath: //cifsserver/cifsShare
  secretRef:
    name: cifs-credentials
  name: cifs

Be sure to configure the networkPath option with the location of the CIFS sahre. In addition, it is recommended (almost required) that the above mountOptions be specified in order to comply with OpenShift's security model.

Note: The security context mount option context=system_u:object_r:svirt_sandbox_file_t:s0 is used by default in order to grant containers access to the file system the share is mounted on where SELinux is enabled. You may override this with a different context, however it is no recommended.

Existing Implementations

The contents of this repository are not the first attempt at a CIFS FlexVolume driver. However, due to the combination of OpenShift's security requirements, as well as the architecture of the controller-manager, it became necessary for another implementation to be created. Many thanks are due to the following implementations:

• fstab/cifs
• Azure CIFS/SMB FlexDriver for Kubernetes

openshift-flexvolume-cifs's Issues

Plugin does not work with Red Hat CoreOS because of the Python dependency

Hi there,

Thank you for your work on the flexvolume driver! I just wanted to take the time to share that it does not work with OpenShift Container Platform 4 and Red Hat CoreOS. Mainly because Python is not installed there.

openshift-flexvolume-cifs/flexvolume-driver/cifs

Line 14 in 62bd24b

echo $JSON | python -c "import sys, json; print(json.load(sys.stdin).get('$KEY',''))"

The error looks as follows (in case someone Googles it)

$ oc describe pod <pod>

Events:
  Type     Reason       Age               From                              Message
  ----     ------       ----              ----                              -------
  Normal   Scheduled    <unknown>         default-scheduler                 Successfully assigned foo/cifs-app-1-h2kt8 to ocp4-tk9v5-worker-8c6fr
  Warning  FailedMount  9s (x6 over 25s)  kubelet, ocp4-tk9v5-worker-8c6fr  MountVolume.SetUp failed for volume "cifs" : invalid character '/' looking for beginning of value

There is an alternative script on GitHub that uses jq instead of Python and apparently works just fine on RHCOS.

Missguiding Credential order in cifs-secret-template.yml

Hi,
first of all I'd like to thank you for your good work!

It's not really an issue, but I just spent hours finding my mistake: In the example secret file cifs-secret-template.yml you first listed password and then username. This is quite counter intuitive. I absent minded copy pasted the username to password and the password to the username.

Better would be:

stringData:
  username: "${USERNAME}"
  password: "${PASSWORD}"

Cheers!

Mounted CIFS volume on host won't mount in pod

running into an issue with CIFS and FlexVolume. OpenShift Enterprise 3.10. We've deployed the driver across our cluster and are testing the code in https://github.com/sabre1041/openshift-flexvolume-cifs/blob/master/examples/application-example.yml

Our CIFS mount options were changed to use version 2.0. We're seeing the following error we are seeing is:

Unable to mount volumes for pod "cifs-app-5-b4vvs_aperio-cropper(de6b7124-7362-11e9-b472-02a62fa63878)": timeout expired waiting for volumes to attach or mount for pod "aperio-cropper"/"cifs-app-5-b4vvs". list of unmounted volumes=[cifs]. list of unattached volumes=[cifs default-token-xbppg]

when we login into the host and do a mount command, we see the CIFS volume mounted. Trying to figure this one out.

We'd appreciate any pointers you can provide.

Thanks

Marco Hernandez

Active Directory Authentication

Does the solution support Active Directory Authentication (using keytab and so on) for authorising access to smb share?

application-template

Hi!
Is it an error "value" field in part of application-template.yml:
description: Name of the secret containing the CIFS credentials.
displayName: CIFS Secret
name: SECRET
required: true
value: "file_mode=0770,dir_mode=0770"
I think it should be "cifs-credentials" instead of "file_mode=0770,dir_mode=0770"