Wildfire: HTAP for Big Data

Emerging large-scale real-time analytic applications (real-time inventory/pricing /recommendations, fraud detection, risk analysis, IoT, etc.) require data management systems that can handle fast transactions (OLTP) and analytics (OLAP) simultaneously. Some of them even require analytical queries as part of a transaction. Efficient processing of transactional and analytical requests, however, leads to different design decisions in a system. This article presents the Wildfire system, which targets hybrid transactional and analytical processing (HTAP) for big data. Wildfire leverages Apache Spark to enable large-scale data processing with different types of complex analytical requests and columnar data processing to enable fast transactions and analytics concurrently.