The Five-Minute Rule for the Cloud: Caching in Analytics Systems

For almost 40 years, Gray and Putzolu’s five-minute rule has helped quickly guide system architects to the break-even point between memory caching and direct local storage access. We believe similar rules of thumb are needed for object caches and storage in disaggregated cloud database system designs. However, it is not straightforward to adapt the established rules to the cloud as they presume fixed hardware, while, in the cloud, resources are dynamic and costs are determined by usage. This paper reviews requirements driving object caches, analyzes the design space, defines a cost model, and proposes new rules of thumb to help system designers determine when caches become cost-effective for analytical workloads in the cloud. While perhaps unsurprising, our analysis on AWS shows that caches are beneficial when a system makes (1) two requests per hour for latency-sensitive workloads, or (2) seven requests per second for non-latency-sensitive workloads. These results are consistent with and help explain the near ubiquity of object store caches in cloud analytics systems.