Hybrid OLTP/OLAP Main Memory Database System Based on Virtual Memory Snapshots

• Applicant: Professor Dr. Thomas Neumann
• Subject Area: Security and Dependability, Operating-, Communication- and Distributed Systems
• Term: from 2011 to 2019
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 194871756

In the first phase of this project we developped a hybrid main memory database system that supports OLTP- and OLAP-applications. The key idea is to exploit the OS/processor-support for virtual memory management. This allows to spawn consistent database snapshots to isolate OLAP queries from the OLTP transactions -- even though they share the same database. Based on the first prototype, called HyPer, that existed already at the start of the project and proved the feasibility of such a hybrid OLTP&OLAP database, we realized in the course of the project a full-fledged database system. The numerous publications in the internationally most prestigious venues (SIGMOD, ICDE, CIDR, and VLDB) suggest that HyPer is one of the most visible academic database projects. This claim is also supported by the Best Paper Award of the ICDE 2014as well as the prestigious VLDB Early Career Award 2014 for T. Neumann's work on the HyPer database engine. In the continuation period we want to focus on the following topics:1) Multi-version concurrency control schemes are a promising method for synchronizing OLTP (online transaction processing) transactions without much overhead for read-only transactions. Furthermore, the previous work on hardware transactional memory should be extended for synchronizing the new adaptive radix-tree indexing.2) So far we have concentrated on optimizing the performance of SQL 92 queries. For decision support queries window functions constitute an important extension of the query language. So far no prior research can be found on massively parallel evalaution of queries containing window functions which is, however, essential to exploit the vast aggregated compute power of modern many-core servers.3) In the near future servers will consist of heterogeneous components (GPUs, energy-safing CPUs, high-performance CPUs, etc.) In order to utilize these components in an energy-efficient and/or performance-optimized way a scheduler is needed that allocates query evaluation tasks to the best-suited hardware component. 4) The hardware vendors (Intel, HP, etc.) are currently developing non-volatile DRAM. This new storage technology is particularly suitable for main-memory database systems like HyPer to reduce fail-over times during recovery and thus increase the availability. Each of these topics is sufficiently complex for a PhD dissertation topic. The existing HyPer implementation will be used as an experimental platform for all engineering work in order to prove its practical relevance.

• DFG Programme: Research Grants
• Co-Investigator: Professor Alfons Kemper, Ph.D.