Building Carousel Part II: Speeding Up the Data Model
Article Summary
Dropbox engineers faced a brutal reality: reading 5,000 photos from SQLite took a full second on a Nexus 5. For users with 100,000+ photos, the standard approach would be unusable.
This deep dive from Dropbox's Carousel team reveals how they built a data model that could handle massive photo libraries without blocking the UI. The solution required rethinking everything from SQLite cursors to in-memory data structures.
Key Takeaways
- Standard Android Cursors rerun queries with OFFSET, causing multi-second hangs on main thread
- Built C++ accumulator model shared across iOS and Android to avoid Java GC stutters
- Immutable snapshots use shallow copies: only changed events get deep copied
- Binary search over event offsets enables fast lookups by absolute photo position
- Transactional interface lets sync, disk reads, and camera roll run on separate threads
Critical Insight
By replacing full array copies with shallow event snapshots, Dropbox reduced the cost of hiding a single photo from nearly one second to negligible time, even with 50,000 photos loaded.
