Storage#

This section describes how to configure storage to serve as backend for volumes or storage classes in your EKS cluster.

Cluster Storage#

We will present two different options for the cluster storage, allowing you to choose the one that best fits your needs.

• EFS - Amazon Elastic File System
• NFS File Share (backed-up by an S3 File Gateway)

Option 1. EFS - Amazon Elastic File System#

• Before creating the storage class, verify that an Amazon EFS Container Storage Interface (CSI) Driver is deployed to the previously created Amazon EKS cluster. Also, verify that an AWS Identity and Access Management (IAM) OpenID Connect (OIDC) provider exists for the cluster. To check this topic, follow this link ⧉.

  • Create an IAM policy and role

    1. Create an IAM policy

       • Download the IAM policy that allows the CSI driver's service account to make calls to AWS APIs. This IAM policy is designed for the AWS EFS CSI (Container Storage Interface) driver, which allows Kubernetes to manage Amazon EFS (Elastic File System) volumes. This policy grants permissions needed for EFS operations from within a Kubernetes cluster.

       curl -o iam-policy-example.json https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/master/docs/iam-policy-example.json
       

       • Create the policy based on the previously downloaded json.

       aws iam create-policy  --policy-name AmazonEKS_EFS_CSI_Driver_Policy --policy-document file://iam-policy-example.json
       

    2. Create an IAM role and attach the IAM policy to it.

       • First, you need to find the cluster's OpenID Connect (OIDC) provider URL. This is possible through the following command:

       aws eks describe-cluster --name my-cluster --query "cluster.identity.oidc.issuer" --output text
       

       The output must be as follows:

       https://oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE

       • Create the IAM role, granting the Kubernetes service account the AssumeRoleWithWebIdentity action. This action allows the role to be assumed by an entity using a Web Identity Federation token (in this case, an OIDC token from Kubernetes).

       • Copy the following contents to a file named trust-policy. Replace 111122223333 with your account ID. Replace EXAMPLED539D4633E53DE1B71EXAMPLE and region-code with the values returned in the previous step.

         {
             "Version": "2012-10-17",
             "Statement": [
                 {
                     "Effect": "Allow",
                     "Principal": {
                         "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
                     },
                     "Action": "sts:AssumeRoleWithWebIdentity",
                     "Condition": {
                         "StringEquals": {
                             "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:kube-system:efs-csi-controller-sa"
                         }
                     }
                 }
             ]
         }
         

       • Now, create the role.

         aws iam create-role
         --role-name AmazonEKS_EFS_CSI_DriverRole
         --assume-role-policy-document file://"trust-policy.json"
         

       • Attach the IAM policy to the role with the following command. Replace 111122223333 with your account ID.

       aws iam attach-role-policy
       --policy-arn arn:aws:iam::111122223333:policy/AmazonEKS_EFS_CSI_Driver_Policy
       --role-name AmazonEKS_EFS_CSI_DriverRole
       

       • Create a Kubernetes service account that is annotated with the ARN of the IAM role that you created.

       • Create a file with the following contents named efs-service-account.yaml and replace 111122223333 with your account ID.

         ---
         apiVersion: v1
         kind: ServiceAccount
         metadata:
           labels:
             app.kubernetes.io/name: aws-efs-csi-driver
           name: efs-csi-controller-sa
           namespace: kube-system
           annotations:
             eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/AmazonEKS_EFS_CSI_DriverRole
         

       • Run the following command to create the storage class. The Kubernetes service account named efs-csi-controller-sa is annotated with the IAM role that you created named AmazonEKS_EFS_CSI_DriverRole.

         kubectl apply -f efs-service-account.yaml
         

  • Install the Amazon EFS driver

    • There are three ways to install the driver: Helm, Manifest (private registry), or Manifest (public registry). We used the last option.

      1. Download the manifest through the following command:

         kubectl kustomize
         "github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/?ref=v1.4.0" > public-ecr-driver.yaml
         

      2. Edit the file and remove the lines that are responsible for the creation of the storage class named efs-csi-controller-sa because it was created in the previous step. After that, apply the manifest:

         kubectl apply -f public-ecr-driver.yaml
         

  • Create an Amazon EFS file system

    • The Amazon EFS CSI driver supports the Amazon EFS access points, which are application-specific entry points into an Amazon EFS file system that make it easier to share a file system between multiple points.

    • Below are the different steps the you need to create an Amazon EFS file system for the previously created cluster.

      1. Get the VPC (Virtual Private Cloud) ID where the cluster is in.

         vpc_id=$(aws eks describe-cluster --name my-cluster --query "cluster.resourcesVpcConfig.vpcId"   --output text)
         

      2. Get the CIDR range for the cluster's VPC.

         cidr_range=$(aws ec2 describe-vpcs --vpc-ids $vpc_id --query "Vpcs[].CidrBlock" --output text)
         

      3. Create a security group with an inbound rule that allows inbound NFS traffic for the Amazon EFS mount points.

         • Create a security group.

           security_group_id=$(aws ec2 create-security-group --group-name MyEfsSecurityGroup --description "My EFS security group" --vpc-id $vpc_id --output text)
           

         • Create an inbound rule that allows inbound NFS traffic from the CIDR for the cluster's VPC.

           aws ec2 authorize-security-group-ingress --group-id $security_group_id --protocol tcp --port 2049 --cidr $cidr_range
           

      4. Create an Amazon EFS file system for the previously created cluster.

         • Create a file system.

           file_system_id=$(aws efs create-file-system  --region region-code --performance-mode generalPurpose --query 'FileSystemId' --output text)
           

         • Create mount targets. First you need to find the ip address of the cluster nodes. This can be done through the following command:

           kubectl get nodes
           

           The output needs to be as follows:

           

         • After that, determine the ids of the subnets in the VPC and which Availability Zone the subnet is in.

           aws ec2 describe-subnets
           --filters "Name=vpc-id,Values=$vpc_id" --query 'Subnets[*].{SubnetId: SubnetId,AvailabilityZone: AvailabilityZone,CidrBlock:CidrBlock}' --output table --region region-code
           

           

         • Finally, add mount targets for the subnets that the nodes are in. Through this link ⧉ you can find out in which CidrBlock the node's ip address is in.

           aws efs create-mount-target
           --file-system-id $file_system_id
           --subnet-id subnet-EXAMPLEe2ba886490
           --security-groups $security_group_id --region region-code
           

         • To ensure that everything was created correctly, you can deploy the sample application with dynamic provisioning presented at the end of this guide ⧉.

         • After that, you can create a storage class. Take what follows as an example:

           

         • Replace the fileSystemId value with the previously created file system.

Option 2. NFS#

1. Create AWS Storage Gateway

   1. Open the AWS Storage Gateway console ⧉ and select Create Gateway.

   2. Gateway Settings

      • For Gateway name, enter a name for the gateway.
      • For Gateway time zone, select the local time zone for the part of the world where you want to deploy the gateway.

   3. Gateway Options

      • For Gateway Type, select Amazon S3 File Gateway.

        

   4. Platform Options

      • For Host platform, select the platform on which the gateway is supposed to be deployed. On our side, we chose Amazon EC2. The gateway must be created in the same Virtual private cloud (VPC) as the cluster, and the selected VPC subnet can be one that is available in the VPC.

      • After filling out all the fields with the correct data, select Launch Instance. This step can take up to two minutes. Select Next when the previous task is completed.

        

   5. Connect to AWS

      • For Endpoint options you can select VPC hosted, but you must have created all the inherent resources in advance. Otherwise, continue with Publicly accessible.

      • For Gateway connection options, continue with the default options.

        

   6. Review and activate

      • In this step, you can edit the prefilled settings. After proceeding, all data cannot be changed.

   7. Configure gateway

      • This is the last step and all configurations can be left with the default values. Select Next to finish the gateway configuration.

        

   8. Now, on the Gateway overview dashboard, the resource is available. You may need to wait a few minutes for the status to turn green. At this point, the gateway is ready to use.

      With the gateway ready, it is time to create the file shares. Besides having the gateway up and running, a S3 bucket resource needs to be created for each file share. Buckets are containers for data stored in S3.

      

2. S3 bucket creation

   1. To create this resource, go to this dashboard ⧉ and select Create bucket. A page similar to the one below comes up. Fill out the bucket name and make sure that the AWS Region is the same as in the previous resources. All other configurations can be left with the default values.

      

   2. If all goes well, the bucket is created and ready to use.

      

3. File share creation

   1. File share settings

      • For Gateway, select the previously created gateway.

      • For Amazon S3 location, select S3 bucket name and fill out the corresponding field with its name.

        

      • By default, the file share name is prefilled with the S3 bucket name, but it can be changed to a different one.

      • For now, leave the other configurations with the default values and select Next.

        

   2. Amazon S3 storage settings

      • Keep the configurations with the default values and proceed.

        

   3. File access settings

      • Keep the configurations with the default values and proceed.

      

      

   4. Review and create

      • Check that all settings are correct and proceed with creating the file share.

   5. File share is ready to use when its status changes to Available. It may take a few minutes between creation and this status update.

      

EKS and Database Shared Storage#

The EKS cluster and the SQL Server must have a shared storage space that allows both to read and write data, ensuring seamless access and interaction between them.