Headless Screenshot Testing in Thumbtack's Android Apps
Article Summary
Brian Terczynski from Thumbtack reveals how they ditched emulators entirely for their 1,500+ screenshot tests. The result? Tests that run at unit test speed and actually gate code merges.
Thumbtack's Android team was stuck running screenshot tests on emulators, which meant slow execution times and tests that couldn't block bad code from merging. They evaluated three headless screenshot testing frameworks to find one that could work with their legacy Dagger field injection and deliver production-quality rendering without devices.
Key Takeaways
- Roborazzi runs 1,500+ screenshot tests in CI without emulators at unit test speed
- Tests now block code merges, eliminating days-long failures on main branch
- High fidelity rendering includes shadows and elevation, matching real devices
- CI runtime only increased 20% despite adding all screenshot tests to verify jobs
Thumbtack moved all screenshot tests to headless execution with Roborazzi, enabling fast iteration and quality gates while maintaining visual fidelity comparable to physical Android devices.
About This Article
Thumbtack's screenshot tests were too slow to run during code review because they needed emulators. This meant developers could merge broken UI code into the main branch, and it would take days before anyone caught the failures.
Brian Terczynski's team switched to Roborazzi, a headless screenshot testing framework built on Robolectric. It runs UI tests in a plain JVM without emulators and works with their custom Dagger field injection patterns.
The 1,500+ screenshot tests now run at unit test speed as part of code review verification, blocking merges when tests fail. CI runtime only increased by 20% despite adding this many tests.