Structured Logging: The Best Friend You'll Want When Things Go Wrong
Article Summary
Grab's logging bills were spiraling out of control. Their solution? A complete rethink of how they capture what happens in production.
Grab's engineering team shares how they built a structured logging framework from scratch to handle billions of rides across Southeast Asia. The article walks through their journey from expensive, unstructured logs to a JSON-based system that integrates seamlessly with their Elastic stack.
Key Takeaways
- Dynamic log levels let teams switch from WARN to DEBUG at runtime without redeploying
- Common schema ensures consistent JSON structure across all services and teams
- Nanosecond timestamps enable causal ordering to see exact event sequences
- Dockerized Kibana gives developers production-like logging in local environments
- Automatic trace ID correlation links all logs for a single request
Critical Insight
Structured logging cut costs, improved debugging speed, and gave Grab's 100+ services a standardized way to understand what's happening in production.
