How We Designed Our Continuous Integration System to Be More Than 50% Faster
Article Summary
Pinterest's Engineering Productivity team slashed CI build times by over 50% in one week. Here's the architecture that made it possible.
Pinterest's team rebuilt their Continuous Integration system for Optimus, a monorepo with 120+ Java services serving 300 monthly contributors. They moved from running all targets to intelligently selecting only what changed, using Bazel queries and custom tooling.
Key Takeaways
- P50 build times dropped from 52 to 19 minutes (2.7x faster)
- Custom build-collector CLI uses Bazel queries to find minimal target sets
- Custom release rules abstract infrastructure and enable parallel artifact generation
- Language-based monorepos reduce pipeline management overhead
- Dispatcher model parallelizes test and release targets across workers
By running only changed code with Bazel queries and parallelizing releases, Pinterest cut CI times from 52 to 19 minutes at P50 and 69 to 49 minutes at P90.
About This Article
Pinterest's CI pipeline for Optimus had 300 monthly contributors working across 120+ Java services. Every code change triggered all targets to run, which meant expensive network calls to fetch and load dependencies even when most of the code hadn't changed.
Pinterest Engineering built build-collector, a Golang CLI that leverages Bazel's rdeps and kind query functions. It identifies which targets actually need to run based on what files changed, then spreads the work across parallel workers.
After one week of migration, P90 build times dropped from 69 minutes to 49 minutes, a 1.4x improvement. The gains came from running only affected targets and better parallelization.