Leveraging AI to build a faster iOS CI system
Article Summary
Muhieddine EL Kaissi from Thumbtack cut iOS CI time in half using runtime coverage instead of running every test. His Smart Test Selection system proves you don't need faster tests—you need fewer tests.
Thumbtack's iOS team was hitting a wall: 30-40 minute CI waits (60+ on Fridays) for every PR, even one-line fixes. Parallelization had maxed out. With 1,500+ test classes and AI-generated code increasing PR volume, they needed a fundamentally different approach. Enter Smart Test Selection (STS), a coverage-based system that maps changed files to only the tests that actually execute them.
Key Takeaways
- STS uses nightly runtime coverage maps to run only affected tests per PR
- Typical PRs now run ~20 tests instead of 400+, cutting CI time 50%
- Documentation changes skip testing entirely (90% faster), config changes compile-only (65% faster)
- AI tools accelerated development but required 27 iterations with real CI feedback loops
- Rolled out like a product: shadow mode, feature flags, incremental percentages
By running only tests that touch changed code, Thumbtack reduced iOS CI time by 50% and made their pipeline scale with AI adoption instead of breaking under it.
About This Article
Thumbtack's iOS infrastructure team had a scaling problem. With over 1,500 test classes, every pull request ran the full test suite no matter what changed. Snapshot tests took up most of the execution time, and they'd already hit the limits of what node parallelization could do.
Engineer Muhieddine EL Kaissi created a nightly pipeline using Groovy, Ruby, and Bash. It builds a reverse lookup from source files to test classes. When a PR comes in, Jenkins uses that data to run only the tests that are actually affected.
Documentation-only changes skip testing entirely and save 90% of build time. Build config changes run just compile verification and save 65% of time. The system now handles AI-generated PRs and tests without becoming a bottleneck.