Notes from Tail Latency Aware Caching Paper by RobinHood
The problem
When the web service latency increases, the first suggested technique is to cache. The cache is a good solution when your system is a read heavy system.
The common technique is to cache the frequently used objects. The method generally reduces the latency, but doesn’t help much for tail latency (p99). The paper “Tail Latency Aware caching - Dynamically Reallocating from cache rich to cache poor” proposes a novel solution for maintaining low request tail latency.
[Read More]