CIS Hardened Image Level 1 on EKS-Optimized Amazon Linux 2

In my current project, I am handling a US customer who is completely focused on working with LLM and security. Their clients are expected to give us the data sources, and we provision from Greengrass, which works as an agent running on the end client to fetch the customer false alarms. It's primarily focused on the SOC 2 side, where data goes to the SOC dashboard or SOC data source, leveraging LLM to filter how many alarms are false positives versus true positives. In doing this, the NOC engineers observing the SOC 2 dashboard do not have to worry about how many are true positives; our LLM API or tool filters it out and indicates how many true positives and false positives are displayed on the dashboard. This significantly eases the burden on the NOC engineers.

We are utilizing Amazon EKS  for our application because most components are LLM and require certain GPUs, so we have specific managed node groups and we also use Carpenter along with HPA in place. Whenever the need arises, process queues are listed in Redis  for elastic cache, and all the lists are processed and read from Redis  and handed over to the nodes dynamically using HPA. We leverage HPA and Carpenter within Amazon EKS  for scaling or scaling out.

Amazon EKS is simple due to its support for diverse AWS  tools and various integrations, significantly influencing my application development and management processes. There is the aspect of trust relationships and permissions. Every service we create involves setting a role, and all authentication processes link to a policy, deciding access. This seamless process extends across accounts thanks to ARN (Amazon Resource Names). With the security guardrails in place, services are accessed effortlessly while maintaining high security, giving me confidence in its management capabilities.

The HPA feature within Kubernetes  is one aspect I appreciate about Amazon EKS; it's beneficial for scalability. The managed node groups or dedicated node groups with GPU capacity allow us to scale in or out depending on our capacity planning, and Carpenter, which we have provisioned for scaling, contributes to that. These features are not available by default in EC2s or ECS, where you lack control over the nodes. Many people attempt to adopt Lambda functions or serverless solutions, but that approach does not suffice for GDPR and HIPAA compliance as it demands a solid identity of the node and clarity on where it is being provisioned. Hence, we cannot depend on any of the Lambda or serverless components which may be unstable, and this often leads to increased billing due to the dynamic nature of scheduled procedures that require a standalone node rather than a transient one.

When it comes to the integration with IAM , I have two thoughts on the authentication process of Amazon EKS. Previously, when I began using AWS , there was something called the AWS auth config map within Amazon EKS. Initially, only the person who creates the cluster has access to it. Now, if you engage in enterprise roles—as I did while working within the UK for Santander—it presents challenges. Essentially, earlier only the creator had the master role for accessing the node and the onboarding process was rather manual, with companies relying on ServiceNow  and other tools for onboarding new joiners or team members. Now, EKS API or EKS CTL has many default settings that are not enabled, you need to enable these. Most clusters are created using Terraform , and you need to create a role that can manage cross-account access, as many customers don't operate with a single account.

My previous employers, such as Fidelity Investment, Nokia, and Ford, have worked across multiple accounts, necessitating single sign-on. This setup allows for cross-account access to the cluster by employing EKS CTL APIs, which leverage single sign-on to onboard team members. As such, once the role has access to the cluster, it can onboard users as a dev user, admin, or tester, simplifying the onboarding process. This way, previously manual tasks can be automated, which is a significant improvement. Earlier, we had to make changes to the config map to onboard users, but with EKS CTL API, this integration between EKS, Kubernetes  service, and the cloud side is improved tremendously, alleviating many worries.

Self-healing nodes assist in minimizing administrative burdens in my projects. Coming from a telecom background where I've worked over seven years, I'm familiar with a service called SON—self-organizing and self-healing functionality. At a logical level, these are the layers we interact with, but AWS handles the physical layer through their software components. For instance, if one node is not ready and you enable the auto mode feature, AWS manages that for you—IAM  upgrades or any nodes malfunctioning. I've seen these features in the UI; I've enabled them, and every 10 or 15 days, patches roll out. I can check them via AWS Inspector  to see if there are any node-level patches or AMI level patches necessary. AWS takes care of these issues automatically. I appreciate that I don't need to manually check the dashboard and apply upgrades one by one, which is a significant improvement.

I measure the impact of Amazon EKS on the organization's management of complex workloads in terms of effectiveness and efficiency through my background in development and systems. Initially, I spent five years as a Java developer before transitioning to DevOps. With my understanding of end-to-end application architecture, I assess workloads based on system and application planning. For example, when I worked on a data lake product in Fidelity Investment, I observed that the cloud onboarding process, including Amazon EKS, had roadmaps extending over five years—from 2019 to 2024. I understand the nuances of enterprise or legacy applications and any system-related complexities. It all boils down to two components: system planning and application planning. Initially, we identify the type of application—whether it is database-related or has high GPU demands. Most applications today involve GPUs, which tend to incur high costs, and often customers are unaware of how to handle dynamic workloads effectively. It's crucial to assess not just one part (system), but various elements CPU, memory, and IOPS since the underlying hardware interacts with those components regardless of domain. First, we need to evaluate the application's requirements—such as its dependency on node storage. With EKS, Kubernetes provides solutions CSI , CNI, and CRI. By understanding the application's demands, I can apply the right Kubernetes configurations for performance optimization, such as taking advantage of Amazon EKS's ability to adjust the container network interface settings to suit the client's workload requirements. This loose coupling allows us to optimize our resources irrespective of whether we're using on-prem or cloud environments.

It has been since 2019 that I started using Amazon EKS. At that time, it was completely new, and many people were not using it just yet; it started from version 1.21, and right now we are on 1.33. Recently, 1.34 has been launched, but it's not yet available in the service catalog; we can see only 1.33. A lot of improvements have been made.

We had numerous add-ons to install manually because Kubernetes is a completely different service than AWS cloud provider, and everyone has opted to use it. After opting, there is an identity that you have to maintain—one at Kubernetes level and one at the AWS provider level. You have to maintain one identity at IAM  level and one within the cluster, Amazon EKS. A few things do not make sense within the add-ons, many of the secret providers that read the secret from Secrets Manager and then mount it as a volume. We use a service called EBS CSI  driver, which reads the secrets or sensitive data from Secrets Manager and then mounts it as a volume to the pod at runtime. However, that doesn't have a dynamic feature where, if any changes happen in the secrets, it can read and populate in the environment.

Sometimes consider your RDS  password or OpenSearch  password rotates. Amazon EKS doesn't have that feature to read the dynamic one and consider that the password has changed overnight; there is no functionality from the provider to see the changes and then restart the pod or fetch the new value. This often leads to downtime of 12 or even 6 hours, depending on when you realize it, so that needs improvement.

Nonetheless, mostly on the add-on side, they have developed a lot; earlier we were installing them manually, but now with EKS auto mode, many things VPC CLI and pod identity service—around four plugins—are installed by default, which is a good thing. However, I believe there should be some solution that is self-contained, covering generic use cases.

With the 1.33 release, they have addressed most of my earlier concerns, but I am still looking for some improvements, particularly in CloudWatch monitoring. In IT, we manage two aspects: either the system or the application. Currently, the application logs and monitoring are not very robust in CloudWatch; you can only find things if you are familiar with them. Fortunately, we are familiar, as most of the monitoring involves two types of databases: one is a time series for monitoring data, and the other is an indexing solution for a streaming service. This means we need to get the logs from each node, index them, and populate them on a screen. That part remains a separate service, but if they managed it within Amazon EKS service, where the monitoring is consolidated in one place, you wouldn't need to rely on Prometheus, Grafana , or different services. It would be advantageous to have a consolidated platform for EKS, as Kubernetes is leveraged; monitoring and logging should also be integrated simply by enabling parameters or tags. This would create a self-contained platform where people can onboard and start using it. Currently, I still need to enable logging and monitoring among other things myself; that shouldn't be the case after six or seven years in the market.

On a scale from 1 to 10, I would rate Amazon EKS tech support an eight. Some individuals have a deep understanding of the services and can identify potential bottlenecks, especially with load balancer endpoints and certificate management. The shift from NGINX  to AWS load balancers has diminished many previous issues. However, not every support engineer meets the same level of expertise, hence why I rate it a solid eight, which I consider decent.

I have been using Amazon EKS for seven years.

Amazon's customer support has its merits; it is good overall. However, when it comes to enterprise licenses, the quality declines significantly. Startups may not recognize this at first, engaging with support during their initial phase, but they soon discover the lack of expert guidance and the costs associated with it—it's quite expensive.

Positive

I have experience with other Kubernetes engines, such as those from Oracle, Azure , and Google. For instance, I find Red Hat OpenShift  to be an excellent competitor. Many have transitioned to Azure  due to the hosting incentives it offers for running open AI models. While people explore Amazon EKS or other Kubernetes solutions, the simplicity in moving between platforms remains an advantage since your manifests mostly stay the same, needing only minor adaptations to services. My experience with Red Hat OpenShift  tells me it's a robust solution with competitive attributes against AWS.

In terms of ROI, the return on investment, I have clear examples. When we began, we utilized managed nodes for applications needing dynamic processes. Some applications simply required pod creation once data was queued for processing, allowing the node to remain free afterward. For a customer who was not cost-effective—we had provisioned a node that saw little CPU usage but substantial memory consumption—we implemented effective resource quotas and HPA. This setup enables the system to utilize resources dynamically based on the actual demand observed by reading metrics from the Prometheus adapter. Notably, the adapter isn't an out-of-the-box feature provided by Prometheus; you need to create your own adapter for it. Using tools HPA and Carpenter allowed us to scale resources based on requirements. Initially, not having them resulted in an unoptimized solution. However, with these tools in place, we witnessed a reduction of costs by approximately a third—if it was $100 beforehand, we brought costs down to $25.

Regarding the pricing aspect and the licensing cost of Amazon EKS, sometimes it is not clear. Most discussions revolve around the data transfer costs from one region to another, and there are certain concerns regarding GPU nodes. However, if you optimize your node usage, with tools such as Kubecost, you can analyze how effectively you utilize your nodes. If you manage to optimize usage, you won't face steep costs. Otherwise, the cloud provider will certainly benefit from inefficient usage. Ultimately, it's not out of the box—if you want to monitor costs effectively, applying separate tools and acting accordingly in advance is essential.

I notice key differences between Amazon EKS and its competitors, analyzing both pros and cons. The seamless integration is sometimes lacking in other offerings. When managing software in platforms Kubernetes—including EKS, AKS, GKE , Rancher  Kubernetes, and Oracle's Kubernetes engine—I've faced specific challenges, particularly with user management in Oracle's solution, which isn't as seamless as it is in Amazon EKS. Comparatively, OpenShift from Red Hat has notable strengths. Oracle is making improvements, especially with its longstanding database solutions. For cloud providers, though, OS from Red Hat is a formidable competitor, offering robust out-of-the-box solutions around resource limits and dashboard configurations that do not require command-line interventions.

The review suggests that people considering Amazon EKS should heed some recommendations. They often attempt to enforce infrastructure as code with tools Terraform  or HashiCorp to maintain workspace and all. I advise using services within a single environment, especially for LLM applications. It's prudent to have multiple LLM sources across various cloud providers while utilizing the same keys within your AWS environment.

My second piece of advice is to establish a separate CI/CD platform independent of AWS. This keeps things loosely coupled; with minimal tweaks in CI/CD pipelines, you can seamlessly migrate from one platform to another—say from EKS to AKS to GKE  or OpenShift—thus keeping the focus on feature development rather than migration headaches. This leads to a modular approach in your code and infrastructure, ensuring that only the cloud provider specifics require adjustments.

Overall rating: 8 out of 10

Public Cloud

Amazon Web Services (AWS)

We use Amazon EKS  mostly for deploying production workloads, and there are multiple AI models that we run on Amazon EKS .

My favorite feature of Amazon EKS is the ecosystem that it provides, including the integration with S3 , along with EBS, and the networking that is smooth to run Kubernetes .

I have experience with Azure , and in comparison to Azure , a downside of Amazon EKS is that even if you want to deploy a dev workload or do some experimentation, we have to pay the charges for the control panel with no free option.

Additionally, I have faced many issues while configuring the node groups and the whole configurations; bringing up the nodes was a bit hectic, and I was not able to determine which node was failing and for what reason.

Specifically, the pricing for the control panel of Amazon EKS is hefty, and there is no cost-cutting that can be done on that side.

I have been using Amazon EKS in my career for four years overall.

Amazon EKS is pretty stable, and I have not seen any lagging, crashing, or downtime.

Amazon EKS is good in terms of scaling.

I have contacted technical support and customer support for Amazon EKS.

Positive

I have used Azure EKS, Civo, and also experimented with GCP as alternatives to Amazon EKS. If I have to rank between all three of them, GCP comes at number one, Amazon EKS comes at number two, and Azure EKS service comes at number three.

On the costing part of Kubernetes , Azure is beating Amazon EKS since I can do some experimentation without paying for the control panel; I can just pay for the node groups, which is an area where Amazon can improve.

From my point of view, the initial deployment of Amazon EKS is difficult.

I had to configure many components, such as IAM  policies and other things; it was not a simple click-through process. The major issue is that there is no single point where I can see all the logs; while there is CloudWatch, it is not easily accessible, and you have to go through a hectic process to search and find information. There is nothing where you can go and click to get all the logs in a single place.

Amazon EKS requires maintenance on my end to continue functioning; we have to do upgradation from time to time since every Kubernetes version is only valid for one year.

On a scale from one to ten, I rate Amazon EKS a seven out of ten.

Public Cloud

Amazon Web Services  (AWS )

Amazon EKS  is used for running any type of application, and it is one of the more stable platforms. We use everything in AWS , CloudFront, and we probably have a hundred applications in AWS .

Several of our platforms have transitioned to Kubernetes  because it is more economical than some other methods, and we have lots of apps that use Kubernetes .

The deployment of Amazon EKS  took time mainly because of the amount of stuff we had to move, such as old legacy applications and systems, which need changes every few years, and AWS is just better.

We have several applications that run in Amazon EKS.

The best features of Amazon EKS include Kubernetes, as several of our platforms have transitioned to Kubernetes because it is more economical than some other methods, and we have lots of apps that use Kubernetes.

The support for AWS tools integration with Amazon EKS has influenced our management process significantly. The cloud affects everything, with probably 80% of our stuff in the cloud, mainly in AWS.

The self-healing nodes with Amazon EKS help minimize administrative burdens, and while I'm not a system admin myself, I do disaster recovery testing and resiliency assessments for cloud applications.

Improvements could include better support and pricing, which is always important.

There are definitely areas for improvement with Amazon EKS.

My experience with AWS products spans the last six, seven years.

Amazon EKS is stable.

Amazon EKS scales effectively.

I would rank their support close to a 10; they are very responsive.

Positive

The initial setup with Amazon EKS is straightforward, and all of our teams are involved with AWS. Some use EC2  containers, and I've seen everything there is to see in AWS.

Amazon EKS is easy to use, and you don't experience the problems that you encounter with some solutions like EC2  containers or S3  buckets.

The support for AWS tools integration with Amazon EKS has influenced our management process significantly. The cloud affects everything, with probably 80% of our stuff in the cloud, mainly in AWS, though we have some in Google Cloud  and Azure .

In my role, I don't set anything up; I set up test disaster scenarios and have teams practice against those scenarios. We have encountered challenges during that migration process, and we utilize our AWS dedicated person whenever something arises that we can't handle or don't understand, so we can seek their help.

Amazon EKS helps us manage complex workflows effectively.

On a scale of 1-10, I rate Amazon EKS a 10.

Public Cloud

Amazon Web Services (AWS)

I'm actually working for a company that uses AWS  as a cloud platform, and for our clients, we use Amazon EKS . We utilize multiple clusters and other requirements, making Amazon EKS  our choice for deployment service or orchestration service.

The usual use case for Amazon EKS is to deploy an application intended for heavy user load and traffic. In technical terms, there are multiple services to choose from, but we choose Amazon EKS for its orchestration, load balancing, and auto-scaling capabilities. With this service, you don't have to worry about manual auto-scaling or manual load balancing. Before Kubernetes , manual intervention was needed for scaling applications, leading to potential crashes if capacity was exceeded. Amazon EKS alleviates those concerns with its auto-scaling feature, where predefined thresholds automatically trigger the launching of additional resources to handle increased traffic. Also, Amazon EKS allows configurations such as minimum and maximum server requirements, ensuring scalability while minimizing costs.

The features of Amazon EKS that I find most valuable include load balancing, auto-scaling, networking, security, and scalability.

Scalability in Amazon EKS refers to the ability to automatically scale up or down your application based on traffic needs. For instance, if you initially expect 10 users but suddenly have 20, Amazon EKS automatically handles the scaling, thereby preventing application crashes and maintaining service availability.

Reliability is crucial when running an application on Amazon EKS, as it ensures your application never crashes. With Amazon EKS, you don't manage the infrastructure yourself; Amazon takes care of it all. You simply need to deploy your container, select the required configurations, and Amazon EKS handles the rest without requiring you to manage the underlying resources.

I have utilized Amazon EKS's integration with IAM , which stands for identity and access management. IAM  restricts access to services, ensuring only authorized personnel can access certain capabilities. This prevents mistakes or unauthorized actions, maintaining security throughout the platform.

The support for AWS  tools integration in Amazon EKS influences our application development and management significantly. With integrated features related to security, scalability, and billing, we ensure the efficiency of our processes. At my company, we manage around 600 clusters on Kubernetes  and emphasize reliability by integrating Amazon EKS with various third-party applications. This integration aids in deployment, security, and ultimately, efficiency, as it ensures that applications remain available and perform efficiently.

One area of Amazon EKS that could be improved is the manual process for adjusting the number of nodes. When I've already defined configurations in Docker  or YAML files, it seems unnecessary to go back and make similar adjustments in the console.

I have been working with Amazon EKS for 4.7 years.

I do not often communicate with the technical support and customer service of Amazon EKS.

Negative

Currently, I am using GKE  in Google Cloud , which is similar to Amazon EKS. The differences between GKE , Amazon EKS, and AKS mainly come down to minor functional variations; overall, they provide similar capabilities.

Regarding the pricing and licensing of Amazon EKS, I am not entirely certain, but from my perspective, it's somewhat comparable to AWS's compute instances. While it may be on the pricier side due to being a managed service provided by Amazon, the features and functionalities justify the cost, especially for applications requiring reliability and scalability.

I participate in the setup and deployment of Amazon EKS, though I don't do it directly through the console. I use a third-party application called Argo CD, which allows me to deploy Kubernetes applications without accessing the Amazon console directly, making the process efficient and straightforward.

On a scale of one to ten, I rate Amazon EKS a nine out of ten.

Public Cloud

Amazon Web Services (AWS)

Our usual use cases for Amazon EKS  include IoT applications for edge computing devices, where we deploy some of our proprietary IoT applications to edge devices running in multiple locations, and artificial intelligence deployment to multiple systems, with a couple of them purely on the cloud where we manage bundled infrastructures into Amazon EKS  for several proprietary customers.

The features and capabilities of Amazon EKS that I have found most valuable include the ease of deployment and the interesting part being the cost, which is not as expensive as setting up other cloud infrastructure.

Amazon EKS's support for AWS  tools integration has influenced our application development and management process by being quite easy, with the integration being straightforward. Whenever issues arise, we talk to the support team who provide us with documentation, which is how we basically sort out most of those issues.

Amazon EKS's self-healing nodes help minimize administrative burdens in my organization by being wonderful and seamless, as it reduces the need for a whole lot of people on the team to handle issues, and it has really been seamless for us.

An area of Amazon EKS that could be improved in the future is its use for edge computing, which has been a small issue for us, especially since most of our recent work has been on edge computing applications such as Raspberry Pi and Jetson. If they could integrate things such as K3s, that would really be helpful as K8s feels a little bit bulky for edge computing deployment.

I have been working with Amazon EKS since last year, when we started moving some of our solutions to AWS  EKS.

My experience with the stability and reliability of Amazon EKS has been very positive, with only a couple of intermittent shutdowns previously, but recently there have been no issues at all.

My impression of Amazon EKS's scalability has been positive, though we have not done very large-scale deployments. Most of what we've done has been on a much smaller but continuous scaling for multiple systems, and there has not been an issue on that aspect so far, although we haven't scaled up to a million or five million devices yet.

I often communicate with Amazon EKS technical support, as they have been our main go-to people.

An example of my interaction with Amazon EKS technical support was during the initial setup when we talked with them, and they provided us an easier route by suggesting how we should bundle our solutions in Docker  for easy deployment.

Positive

Before Amazon EKS, I did not use a different solution for these use cases, as our path has always been with Amazon EKS.

My experience with the initial setup of Amazon EKS was straightforward with no challenges at all on my part, although the interns might complain about some snags. It's basically about studying the documentation.

My setup process involves building the application on GitHub , bundling it in Docker , and connecting with Amazon EKS.

I have seen a return on investment with Amazon EKS for time, as it has helped me save a significant amount of time. However, the cost side has not been as positive since some of the applications run in dollars, leading to complaints from customers and ourselves about the cost, particularly when providing services to customers across Nigeria and some African countries. The return on investment has not been great due to the foreign exchange rate, but for time savings, it has been wonderful in helping with deployment.

My opinion on the pricing and licensing of Amazon EKS is that it is quite varied, especially when doing projects in the African continent. It's quite expensive considering the local currency with respect to the conversions to dollars or euros, and if this could be lowered, it would help more deployments on our side with Amazon EKS.

Before choosing Amazon EKS, I evaluated other options or technologies, including Kubernetes  on AWS, Google Cloud , and Microsoft Azure , but most of our experiences came from AWS, so we stayed with AWS.

I have not utilized Amazon EKS's integration with IAM  solution.

I have not encountered specific benefits using Amazon EKS's automated patching feature for my Kubernetes  clusters, but it has been satisfactory as we haven't actually had many issues with using Kubernetes.

The impact of Amazon EKS on my organization's ability to manage complex workflows effectively has been purely managed by my colleague, and it has been quite seamless with no issues on that particular aspect.

Some of the benefits and positive impact that I have received from Amazon EKS include getting cloud credits through Activate and certain deployments around migration, which have been quite beneficial, along with business support credit and support during certain issues. During the initial times of integrations and migrations, AWS connected us with more specialists in different locales with much more experience while also paying for their services.

Based on my overall experience with Amazon EKS, I would rate it an eight out of ten due to the lack of K3s from Rancher.

Amazon Web Services (AWS)