Slashing CI Wait Times: How Pinterest Cut Android Testing Build Times by 36%+

Summary

This article discusses Pinterest's approach to reducing CI wait times for Android end-to-end testing by implementing a runtime-aware sharding mechanism. The previous method of sharding tests by package names led to significant imbalances in execution times, causing delays due to the slowest shard. By leveraging historical test duration data, Pinterest developed a new in-house testing platform that optimally distributes tests across shards to ensure similar runtimes. The implementation resulted in a 36% reduction in build times and improved developer productivity by minimizing the impact of flaky tests and optimizing resource allocation on EC2-hosted emulators.

Key Learnings

• 1Implementing runtime-aware sharding can significantly reduce CI build times by balancing test execution across shards.
• 2Historical test duration data is crucial for optimizing test allocation and improving overall CI efficiency.
• 3Transitioning from third-party testing platforms to an in-house solution can provide better control over testing infrastructure and reduce flakiness.
• 4Utilizing a greedy algorithm for test assignment can lead to more predictable runtimes and improved resource utilization.
• 5Continuous monitoring and adjustment of the sharding algorithm based on performance metrics can further enhance developer experience.

Who Should Read This

Senior Software Engineers focusing on CI/CD optimization and test automation strategies

Test Your Knowledge

Topics