Ad Metrics Quality And Testing On Mobile
Article Summary
Maya Reddy from Pinterest reveals how their team eliminated last-minute P1 bugs by automating ad metrics testing. The shift from bi-weekly manual QA to continuous integration transformed their release confidence.
Pinterest's Ad Formats team faced a critical challenge: ensuring accurate ad metrics for advertisers while shipping mobile releases every two weeks. Manual QA was catching bugs too late, risking advertiser trust. This article details their journey building an automated integration test suite that validates logging behavior across multiple ad formats and surfaces.
Key Takeaways
- Custom tool injects specific ad formats into test accounts for guaranteed coverage
- Python script auto-generates tests for new surfaces by replicating existing patterns
- Buildkite runs tests in parallel nightly plus subset per commit for early signals
- Tests verify UI actions match logged events across home feed, search, and related Pins
- Feature owners now write and maintain their own tests, not a separate QA team
Pinterest's automated ad logging tests catch bugs earlier in the release cycle, freeing manual QA for exploratory testing while maintaining advertiser trust through accurate metrics.
About This Article
Pinterest's ad metrics logging depended on accessibility labels in UI tests. When feature teams changed UI elements like buttons without telling the testing framework, tests would fail.
Pinterest split the work. Feature teams file bugs to fix accessibility label issues. The Metrics Quality & Test Tools team updates the test framework when structural problems show up.
This split in test ownership reduced how much time teams spent maintaining tests. Feature teams and the tools team could both move faster.
