Tracing a 2s Latency Spike to a Single SQL Query

TL;DR AI
2 min readKey summary
Checkout p99 latency spiked from 50 ms to over 2 seconds, even though metrics and logs looked normal.
Tracing showed the PostgreSQL query itself was fast; the delay came from transferring a huge row returned by SELECT * on inventory_logs.
A 15 MB JSONB column made the database payload expensive to move over the network, with time showing up in SocketInputStream.read().
Limiting the query to only needed columns removed the bottleneck and cut end-to-end latency from about 2.1 seconds to 12 ms.
