Solving Native Memory Leaks in Mobile Apps

Sid Rathi from Expedia Group debugged a production nightmare: KStream apps dying in an endless OOM loop. The culprit? A single unclosed iterator eating native memory.

Expedia engineers faced a vicious cycle where Kafka Streams applications kept crashing from out-of-memory errors. New instances would spin up, rebalance, then fail again within minutes. The leak wasn't in heap memory, making it exceptionally hard to diagnose.

Key Takeaways

• Scaling horizontally made the problem worse: memory jumped 5 to 10GB in 50 minutes
• RocksDB state store iterators were never closed, leaking native memory via OS page cache
• Fix was simple: wrap iterator in try-with-resources to auto-close after use
• Docker memory usage stabilized completely after deploying the one-line fix

The article reveals why increasing app instances actually accelerated the memory leak, and shares the specific YourKit profiling technique that finally cracked the case.