Project Details
Projekt
MxKernel: A Bare-Metal Runtime System for Database Operations on Heterogeneous Many-Core Hardware
Subject Area
Security and Dependability, Operating-, Communication- and Distributed Systems
Term
since 2017
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 361498541
Emerging hardware platforms are characterized by large degrees of parallelism, complex memory hierarchies, and increasing hardware heterogeneity. Their theoretical peak data processing performance can only be unleashed if the different pieces of systems software collaborate much more closely and if their traditional dependencies and interfaces are redesigned.As a novel approach we have developed the key concepts and a prototype implementation of a bare-metal runtime system named MxKernel (first funding period of the project). MxKernel provides very lightweight resource management for database management system (DBMS) and operating system (OS), which both run as equal peers on top of the runtime. In MxKernel, heterogeneity and parallelism become first-class citizens and deep memory hierarchies are considered from the very beginning. Instead of a classical “thread” model, MxKernel provides a simpler control flow abstraction: MxTasks model closed units of work, for which MxKernel will guarantee the required execution semantics, such exclusive access to a specific object in memory. They can be a very elegant abstraction also for heterogeneity and resource sharing. Furthermore, MxTasks are annotated with meta data, such as code variants (to support heterogeneity), memory access behavior (to improve cache efficiency and support memory hierarchies), or dependencies between MxTasks (to improve scheduling and avoid synchronization cost). With precisely the required meta data available, MxKernel can provide a lightweight, yet highly efficient form of resource management, even across applications, operating system, and database.Based on the MxKernel prototype and our promising results, e.g. the ability to optimize memory usage by well-informed data prefetching, we are now going to explore and tap the full potential of the novel system software architecture in a more systematic way. For this purpose we will look into more realistic application scenarios and work load profiles. Besides transactional loads we will now also consider analytic and mixed profiles (hybrid transactional/analytic processing). Additionally, we will explore how multiple concurrent DBMS instances, which dynamically compete for resources, can be coordinated in an optimized way. The ability to annotate tasks and data objects shall be used for further improved resource scheduling. A query translator shall simplify the creation of tasks and annotations. Query compilation for the task-based runtime system can also be expected to improve the efficiency.Eventually, the gained insights on the better exploitation of modern manycore systems will also be used to identify migration paths to improve the concepts used in existing DBMS and OS in a less disruptive but incremental manner.
DFG Programme
Priority Programmes
Subproject of
SPP 2037:
Scalable Data Management on Future Hardware
