Automating Integration Tests To Ensure Android App Quality
Article Summary
Brian Terczynski from Thumbtack reveals how his team eliminated weekly manual QA sessions that were draining engineering time. Their solution? A fully automated integration testing pipeline that catches bugs before production.
Thumbtack's Android team was stuck in a cycle of manual testing: dogfooding new features and weekly QA sessions for regression testing. This process was time-consuming, inconsistent, and only caught bugs right before release. They needed a better approach to ensure app quality without sacrificing developer productivity.
Key Takeaways
- Combined Espresso and UI Automator with BrowserStack for automated testing on real devices
- Tests run against live pre-production backend, catching integration issues early
- Each test creates isolated user data for hermetic, repeatable execution
- Automated tests run multiple times daily plus on every release branch
- Eliminated weekly manual QA sessions while improving test consistency and coverage
Thumbtack built a complete test pyramid with automated integration tests running several times daily, replacing manual QA sessions and catching backend-breaking changes before production.
About This Article
Thumbtack's integration tests were abandoned because the build machines couldn't run emulators. Without a way to execute tests automatically and repeatedly, engineers stopped paying attention to them.
In 2019, Brian Terczynski's team switched to BrowserStack's App Automate platform. It provided an API that Jenkins could call to run Espresso tests across multiple Android devices without needing local emulators.
The team now runs tests several times daily. Reports include screenshots, videos, and logcat output that get posted to Slack. This lets them catch backend-breaking changes and Android incompatibilities before the weekly release.