How we reduced manual regression tests by 70% using AI tools

Duolingo's QA team cut manual regression testing by 70% using AI. Here's how they made it work at scale.

Brock Janikowski from Duolingo shares how their team partnered with MobileBoost's GPT Driver to automate previously difficult-to-automate regression tests. The solution needed to handle Duolingo's rapid iteration pace and complex A/B testing scenarios.

Key Takeaways

• Natural language prompts drive tests on virtual devices, no coding required
• Goal-based testing beats step-by-step instructions for handling app variability
• Team reviews recorded test runs in minutes instead of manual hours
• GPT struggles with complex timed interactions and nuanced translations

The counterintuitive lesson: making tests less specific actually made them more reliable.