Tackling UI Test Execution Time Imbalance for Xcode Parallel Testing
Article Summary
Grab's engineering team discovered that Xcode's parallel testing was secretly sabotaging their CI pipeline. Tests from the same class were forced onto the same simulator, creating massive execution time imbalances.
Grab's iOS team faced a critical bottleneck: their UI test suite was taking too long in CI, and Xcode's parallel testing wasn't helping as much as expected. The culprit? An undocumented constraint that grouped all tests from the same class onto a single simulator, leaving other simulators idle while one struggled with the workload.
Key Takeaways
- Xcode groups tests by class name, forcing them onto one simulator
- Test time imbalance reduced to 0.4 minutes after implementing the fix
- Solution: inject random tokens into class names to trick Xcode's grouping logic
- Swizzle XCTestSuite.init to remove tokens and run tests properly
By manipulating test names with random tokens and swizzling XCTestSuite initialization, Grab achieved better load balancing and stabilized their test time imbalance at around 0.4 minutes.
About This Article
Grab's iOS CI infrastructure had uneven test execution times because Xcode groups all tests from the same class onto a single simulator. This prevented tests from being distributed efficiently across parallel workers.
Grab's team added random tokens to test class names when passing them via the -only-testing argument. This tricked Xcode's grouping logic. They then swizzled XCTestSuite.init(forTestCaseWithName:) to strip out the tokens and restore the original test names before the tests ran.
The test time imbalance metric dropped and settled around 0.4 minutes. This let Grab add more tests to classes without slowing down the CI pipeline.