Solving the Problem of One Billion Computations
Article Summary
Skyscanner needed to store 1.6 billion weights to rank hotel prices. Their in-memory approach couldn't scale, so they turned to AWS.
When Skyscanner added granular partner ranking (by market, device, and hotel), they faced a massive data challenge. Their algorithm needed to compute and store weights for every partner across every combination, updated daily and consumed in real-time.
Key Takeaways
- 1.6 billion weights stored across partner, market, device, and hotel combinations
- DynamoDB chosen for key-value storage with auto-scaling across three AWS regions
- Average search resolves 50 price parities across 15 hotels in real-time
- Auto-scaling EC2 groups handle traffic spikes without manual intervention
Critical Insight
Skyscanner solved their billion-computation problem by moving from in-memory storage to a distributed AWS architecture with DynamoDB and auto-scaling APIs.
