Engineering posts about Load Shedding
Curated summaries and key learnings for engineers working with Load Shedding.
Rethinking Distributed Systems for Serverless Performance and Reliability
The article explores the evolution of serverless compute for Apache Spark, addressing long-standing architectural challenges that have hindered performance and reliability. It emphasizes the need for...
Rearchitecting the Workflows control plane for the agentic era
The article discusses the rearchitecting of the Workflows control plane to accommodate a shift towards agent-triggered workflows, necessitated by the increasing demand for durable execution engines...
Building a Distributed Persistent Queue That Scaled AI Workloads 5x Under LLM Rate Limits
The article discusses the engineering of a distributed persistent queue that orchestrates AI workloads and human workflows within strict infrastructure limits. It highlights the challenges of scaling...
Zero-Downtime Patching in Lakebase Part 1: Prewarming
The article discusses the challenges associated with planned maintenance in database systems, particularly focusing on the performance degradation caused by cold restarts. It introduces Lakebase's...
Multi-Cloud Challenges, Intelligent Load Balancing, and AI-Powered Workflows: Databricks at SRECon 2026
The article highlights Databricks' advancements in infrastructure reliability and efficiency as presented at SRECon 2026. It delves into the challenges of multi-cloud operations, particularly...
Scaling Jira cloud Migrations, One Bottleneck at a Time
The article chronicles the Jira Migrations team's journey in scaling their migration platform from handling 20,000 to 50,000 Monthly Paid Enabled Users (PEUs). It discusses the transition from an...
How Data 360 Optimized Kubernetes Scheduling Architecture, Delivering 13% Cost Savings
The article discusses how the Data 360 Compute Fabric team at Salesforce optimized Kubernetes scheduling to enhance resource efficiency and reduce costs. By evolving the default kube-scheduler...
Building Prometheus: How Backend Aggregation Enables Gigawatt-Scale AI Clusters
The article discusses the implementation of backend aggregation (BAG) in Meta's Prometheus AI clusters, highlighting its role in interconnecting thousands of GPUs across multiple data centers. BAG...