How to install openstack cloud provider in k8s

Openstack Cloud Provider for kubernetes with relevant plugins allow to use Openstack load balancers, Keystone authentication and Cinder volumes in Kubernetes cluster. For more info, see https://github.com/kubernetes/cloud-provider-openstack

Step-by-step guide

Install Openstack Cloud Provider

Create cloud-config secret in kube-system namespace from cloud.conf file

 vi cloud.conf
 [Global]
 auth-url=https://openstack.example.com:5000/v3/
 username=<openstack-user>
 password=********
 region=REGION
 tenant-id=0ede22d109494dc3972c22693ca14883
 domain-id=714d120017064be885d94142538153d7
  
 kubectl -n kube-system create secret generic cloud-config --from-file=cloud.conf

Download Openstack Cloud Provider manifest from https://github.com/kubernetes/cloud-provider-openstack/blob/master/manifests/controller-manager/openstack-cloud-controller-manager-ds.yaml and apply it

 cat openstack-cloud-controller-manager-ds.yaml
 apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: cloud-controller-manager
   namespace: kube-system
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
   name: system:cloud-controller-manager
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: cluster-admin
 subjects:
 - kind: ServiceAccount
   name: cloud-controller-manager
   namespace: kube-system
 ---
 apiVersion: apps/v1
 kind: DaemonSet
 metadata:
   name: openstack-cloud-controller-manager
   namespace: kube-system
   labels:
     k8s-app: openstack-cloud-controller-manager
 spec:
   selector:
     matchLabels:
       k8s-app: openstack-cloud-controller-manager
   updateStrategy:
     type: RollingUpdate
   template:
     metadata:
       labels:
         k8s-app: openstack-cloud-controller-manager
     spec:
       nodeSelector:
         node-role.kubernetes.io/master: ""
       securityContext:
         runAsUser: 1001
       tolerations:
       - key: node.cloudprovider.kubernetes.io/uninitialized
         value: "true"
         effect: NoSchedule
       - key: node-role.kubernetes.io/master
         effect: NoSchedule
       serviceAccountName: cloud-controller-manager
       containers:
         - name: openstack-cloud-controller-manager
           image: docker.io/k8scloudprovider/openstack-cloud-controller-manager:latest
           args:
             - /bin/openstack-cloud-controller-manager
             - --v=1
             - --cloud-config=$(CLOUD_CONFIG)
             - --cloud-provider=openstack
             - --use-service-account-credentials=true
             - --address=127.0.0.1
           volumeMounts:
             - mountPath: /etc/kubernetes/pki
               name: k8s-certs
               readOnly: true
             - mountPath: /etc/ssl/certs
               name: ca-certs
               readOnly: true
             - mountPath: /etc/config
               name: cloud-config-volume
               readOnly: true
             - mountPath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
               name: flexvolume-dir
           resources:
             requests:
               cpu: 200m
           env:
             - name: CLOUD_CONFIG
               value: /etc/config/cloud.conf
       hostNetwork: true
       volumes:
       - hostPath:
           path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
           type: DirectoryOrCreate
         name: flexvolume-dir
       - hostPath:
           path: /etc/kubernetes/pki
           type: DirectoryOrCreate
         name: k8s-certs
       - hostPath:
           path: /etc/ssl/certs
           type: DirectoryOrCreate
         name: ca-certs
       - name: cloud-config-volume
         secret:
           secretName: cloud-config
 ---
 kubectl apply -f openstack-cloud-controller-manager-ds.yaml

Install Cinder CSI Plugin

Download Cinder CSI plugin manifests from https://github.com/kubernetes/cloud-provider-openstack/tree/master/manifests/cinder-csi-plugin to cinder-csi-plugin folder - all except csi-secret-cinderplugin.yaml because we already created cloud-config secret.

Apply them

 kubectl apply -f cinder-csi-plugin

Create cinder storage class

 vi cinder.yaml
 apiVersion: storage.k8s.io/v1
 kind: StorageClass
 metadata:
   name: cinder
 provisioner: cinder.csi.openstack.org
  
 kubectl apply -f cinder.yaml

To request permanent storage from cinder, create permanent volume claim (PVC) and refer to it in container volumes. For example,

 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: cinder-volume-claim
 spec:
   resources:
     requests:
       storage: '10Gi'
     limits:
       storage: '20Gi'
   accessModes:
     - ReadWriteOnce
   storageClassName: cinder
 ---
 apiVersion: v1
 kind: Pod
 metadata:
   name: nginx
 spec:
   containers:
   - image: nginx
     imagePullPolicy: IfNotPresent
     name: nginx
     ports:
     - containerPort: 80
       protocol: TCP
     volumeMounts:
       - mountPath: /var/lib/www/html
         name: cinder-volume
   volumes:
   - name: cinder-volume
     persistentVolumeClaim:
       claimName: cinder-volume-claim
       readOnly: false