How Data 360 Optimized Kubernetes Scheduling Architecture, Delivering 13% Cost Savings

Summary

The article discusses how the Data 360 Compute Fabric team at Salesforce optimized Kubernetes scheduling to enhance resource efficiency and reduce costs. By evolving the default kube-scheduler behavior, the team addressed issues such as node fragmentation and the inefficiencies of reactive autoscaling. They implemented a custom scheduler using a MostAllocated approach to consolidate executor pods, which improved CPU and memory utilization by approximately 15% and resulted in a 13% reduction in compute costs. The changes also enhanced operational reliability by reducing node disruption rates and ensuring that Spark applications experienced fewer executor losses and more predictable runtimes.

Key Learnings

• 1Understanding the limitations of the default kube-scheduler in high-scale Spark environments and the need for a custom scheduling strategy.
• 2The trade-off between resource utilization and operational stability, especially in bursty workloads.
• 3The importance of proactive scheduling and intelligent resource placement to minimize costs while maintaining reliability.
• 4How the MostAllocated scoring strategy can effectively eliminate fragmentation and improve resource efficiency.
• 5The impact of scheduling decisions on job-level SLA stability and the overall cost-to-serve.

Who Should Read This

Senior Cloud Engineers with experience in Kubernetes and distributed systems, focusing on optimizing resource utilization and cost management in large-scale data processing environments.

Test Your Knowledge

Topics