Slack recently disclosed the architecture of its internal compute orchestration platform, epitomized by "Bedrock", based on AWS Elastic Kubernetes Service (EKS) and Karpenter.
The "Bedrock" platform, encapsulated within a single YAML file, streamlines container deployment and management, simplifying the deployments for Slack's internal developers. Leveraging tools like Jenkins, FQDN service discovery, and the Nebula overlay network, Bedrock ensures efficient operations, with over 80% of Slack applications now operating seamlessly on this innovative framework, resulting in enhanced testing precision and refined infrastructure management.
The quest for enhanced operational efficiency led Slack to explore Karpenter. Previously, managing multiple Autoscaling Groups (ASGs) posed challenges, especially with the burgeoning needs of Slack's internal teams. The imperative for frequent updates to EKS clusters housing thousands of worker nodes exacerbated the complexity, throttling the upgrade cadence. To address these challenges, Slack started using Karpenter a cluster autoscaler that automatically provisions new nodes (computers) in the EKS cluster when there are pods that need to be run but cannot be placed on any existing nodes due to resource limitations.
The Karpenter logic is:
- Karpenter monitors your Kubernetes cluster and identifies pods that can't be scheduled on any available nodes. This typically happens when there aren't enough resources (CPU, memory, etc.) on existing nodes to run the pods.
- When Karpenter detects unschedulable pods, it springs into action. It evaluates the resource needs of the waiting pods and picks the optimal type of instance (computer) from Amazon EC2 to run them.
- Karpenter can also be configured with rules to scale down the cluster by terminating unused nodes. This helps to avoid unnecessary costs associated with running idle nodes.
Overall, Karpenter simplifies managing your EKS cluster's size by automating the provisioning and scaling of nodes based on your workload requirements. This can be especially beneficial for workloads with fluctuating resource demands.
Slack implemented Karpenter in a planned, two-phase approach, prioritizing seamless integration and minimal disruption. Initially deployed alongside core deployments and applications, Karpenter underwent meticulous validation for several applications. Subsequently, transitioning Karpenter Controller workloads to dedicated ASGs facilitated a comprehensive rollout across Slack's extensive fleet of EKS clusters, comprising thousands of worker nodes. This phased approach minimized disruption and ensured smooth integration.
The architectural prowess of Slack's Bedrock EKS cluster, depicted in the following figure, underpins its operational resilience and efficiency. The combined impact of Bedrock and Karpenter on Slack's EKS architecture is evident. Dynamic instance selection based on pod resource demands and Karpenter's consolidation measures ushered in increased cluster utilization and tangible cost savings. Accelerated node provisioning times further underscored Karpenter's efficacy, ensuring swift adaptation to evolving workload demands and seamless system upgrades.
Slack remains committed to the continuous refinement of its Karpenter environment. Exploring initiatives such as Managed Karpenter and customized kubelets underscores their dedication to innovation and operational excellence. Managed Karpenter, a potential future state, promises further automation and streamlined cluster management. Customized kubelets and lightweight container runtime managers can be tailored to Slack's specific needs, further optimizing resource utilization and security. These efforts pave the way for a future-proof and highly efficient platform built on Amazon EKS.