Seamless EKS-S3 Integration with Amazon EKS Pod Identity

Navigating permissions in Amazon Elastic Kubernetes Service (EKS) for applications to interact with AWS services, such as Amazon S3, can be complex. Amazon EKS Pod Identity simplifies this task, enabling seamless and secure access. This guide demonstrates how to use Amazon EKS Pod Identity to grant an application in EKS the necessary permissions to work with an Amazon S3 bucket.

Prerequisites

Before starting, ensure you have:

• An active AWS account with access to EKS and S3.
• A running Amazon EKS cluster, version 1.24 or higher.
• AWS CLI installed and configured.
• kubectl set up for your EKS cluster.
• Docker installed for application containerization.
• A Python environment for script testing.
• An existing Amazon S3 bucket.
• Basic knowledge of Kubernetes.

Alright, let’s dive in and set the stage for secure, seamless operations between your Kubernetes pods and S3 buckets!

Define IAM Role Permissions

To run applications smoothly, I first need to create an IAM role for permission to access my S3 bucket. This involves setting up the role and configuring its trust policy to allow pods.eks.amazonaws.com as the principal. I use a specific IAM template for this setup.

Replace "k8s-eks-s3-bucket" with the name of your S3 bucket.

Install EKS Pod Identity Agent

After the IAM role is ready, I then set up the Amazon EKS Pod Identity Agent in my existing EKS cluster. For beginners, Amazon’s Getting started with Amazon EKS is a great resource to learn about setting up a cluster.

First, I go to the Amazon EKS dashboard and select my cluster. On the cluster page, I click the Add-ons tab and then choose ‘Get more add-ons’. Here, I add the Amazon EKS Pod Identity Agent. On the following page, I stick with the default settings and click ‘Next’. After reviewing the add-on configuration, I click ‘Create’. In a few minutes, the add-on is active in my cluster. The final step is linking the IAM role to my Kubernetes pods.

Create a Namespace and Service Account in Kubernetes

Namespaces in Kubernetes allow you to partition cluster resources between multiple users. To create a new namespace:

Open your terminal and run the following kubectl command to create a new namespace:

kubectl create namespace ekspods3ns

Replace ekspods3ns with the desired name for your namespace.

Verify that the namespace is created:

kubectl get namespaces

This command will list all the namespaces in your cluster, including the one you just created.

A service account provides an identity for processes that run in a Pod. To create a service account within your newly created namespace:

Run the following kubectl command:

kubectl create serviceaccount ekspods3svc --namespace ekspods3ns

Replace ekspods3svc with the name you want for the service account, and ekspods3ns with the name of your namespace.

Verify the service account is created:

kubectl get serviceaccounts --namespace ekspods3ns

This will list the service accounts in the specified namespace.

Map IAM Role to Kubernetes Service Account

After adding the Amazon EKS Pod Identity add-on, namespace, and service account to my cluster, I link the IAM role to my Kubernetes pods.

I go to the Access tab in my EKS cluster and select ‘Create Pod Identity association’ under Pod Identity associations. Here, I use the IAM role I made earlier and select my Kubernetes namespace and service account. Finally, I click Create.

Testing the EKS-to-S3 Integration

Now, when I deploy my application pod in the Amazon EKS cluster, it will already be set up with EKS Pod Identity and the necessary IAM permissions.

It’s time to test the integration by deploying a Kubernetes job. This job will utilize the Docker image I’ve created and pushed to Docker Hub, which contains a script to upload a file to an S3 bucket.

• Clone the Git repository containing the Kubernetes manifest file (ekspods3-job.yaml).
• Open your terminal and navigate to the directory where you cloned the Git repository.
• Edit the ekspods3-job.yaml file and replace the AWS_S3_BUCKET_NAME value with the name of your S3 bucket.

Apply the manifest file to your EKS cluster using the following command:

kubectl apply -f ekspods3-job.yaml

You can check the status of the job by running:

kubectl get jobs -n ekspods3ns

Expected Outcome

The job will create a pod in your EKS cluster that runs the Docker container. This container will execute the script to upload a file to the specified S3 bucket. You can verify the file upload by checking the specified bucket in the AWS S3 console or using the AWS CLI.

Conclusion

Amazon EKS Pod Identity is a powerful tool for simplifying AWS service access in Kubernetes applications. By following these steps, you can securely and efficiently manage access to AWS services, like S3, in your EKS environment.