[SOLVED] Kubernetes PVC with ReadWriteMany on AWS

Kubernetes PVC with ReadWriteMany on AWS

I want to setup a PVC on AWS, where I need ReadWriteMany as access mode. Unfortunately, EBS only supports ReadWriteOnce.

How could I solve this?

I have seen that there is a beta provider for AWS EFS which supports ReadWriteMany, but as said, this is still beta, and its installation looks somewhat flaky.
I could use node affinity to force all pods that rely on the EBS volume to a single node, and stay with ReadWriteOnce, but this limits scalability.

Are there any other ways of how to solve this? Basically, what I need is a way to store data in a persistent way to share it across pods that are independent of each other.

Solution

Using EFS without automatic provisioning

The EFS provisioner may be beta, but EFS itself is not. Since EFS volumes can be mounted via NFS, you can simply create a PersistentVolume with a NFS volume source manually -- assuming that automatic provisioning is not a hard requirement on your side:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-efs-volume
spec:
  capacity:
    storage: 100Gi # Doesn't really matter, as EFS does not enforce it anyway
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  mountOptions:
    - hard
    - nfsvers=4.1
    - rsize=1048576
    - wsize=1048576
    - timeo=600
    - retrans=2
  nfs:
    path: /
    server: fs-XXXXXXXX.efs.eu-central-1.amazonaws.com

You can then claim this volume using a PersistentVolumeClaim and use it in a Pod (or multiple Pods) as usual.

Alternative solutions

If automatic provisioning is a hard requirement for you, there are alternative solutions you might look at: There are several distributed filesystems that you can roll out on yourcluster that offer ReadWriteMany storage on top of Kubernetes and/or AWS. For example, you might take a look at Rook (which is basically a Kubernetes operator for Ceph). It's also officially still in a pre-release phase, but I've already worked with it a bit and it runs reasonably well. There's also the GlusterFS operator, which already seems to have a few stable releases.