Path to GPU-Initiated I/O for Data-Intensive Systems

The process of training and serving deep learning (DL) models is computationally expensive, mandating the use of powerful and expensive accelerators such as GPUs and TPUs. Furthermore, the prevalence of GPUs in data centers today motivate developing database systems that can leverage the available GPU resources. Both the latency of DL tasks and database queries and high utilization of these accelerators depend on how efficiently we can move the data to the accelerators. Given today’s dataset sizes, fitting everything in GPU or even CPU memory is not always feasible or can be expensive. The I/O path while fetching the data from disks, however, still dominantly relies on CPUs.
In this work, we take a step toward understanding today’s landscape for optimizing the I/O path for reading data to GPUs from disks, with a focus on SSDs. First, we review the prominent technologies that target GPU-centric storage accesses. Then, we dive deeper into BaM, as the state-of-the-art method for GPU-centric storage, and evaluate its performance in comparison to the state-of-theart CPU-centric storage interface SPDK. Our results demonstrate that while BaM is able to match the performance of SPDK without involving CPUs on the I/O path, this comes at the cost of a very high GPU use. Finally, we highlight future research directions to enable an I/O path that is both efficient and easy-to-adopt for data-intensive systems that use GPUs.