CacheServer: Disaggregated Caching for Cloud Databases

Cloud databases typically cache data fetched from cloud storage to compute nodes and conduct affinity scheduling to guarantee data locality -- scheduling queries accessing the same data segment to the same node. Although this strategy improves the cache hit rate and effectively reduces the prohibitive data transmission cost due to the low bandwidth of cloud storage, coupling cache with query execution causes major issues. First, it excludes other scheduling mechanisms from exploiting the merit of data caching. Second, it causes load imbalance under skewed workloads. To address these issues, we propose CacheServer, a system that decouples data caching from query execution, introducing dedicated cache nodes to manage data caching. Under this architecture, query execution nodes only fetch data from cache servers. Therefore, it always benefits from the high bandwidth of intra-data center network no matter how a cloud database schedules queries to query execution nodes, thus enjoying the advantage of data locality while not suffering from stragglers caused by hot data. To make this architecture effective, we propose a data prefetching mechanism that maximizes the availability of the data when a query requests it from cache servers, by overlapping fetching data from cloud storage with the execution of previous queries. Furthermore, CacheServer designs an automatic resource allocation algorithm, which given a resource budget, automatically allocates resources between cache and query execution nodes to minimize query latency. Our experimental evaluation on various workloads under different scheduling mechanisms shows that CacheServer outperforms alternatives from 2 to 10x in average execution time and tail latency, given the same resource budget.