Optimizing Explicit and Implicit Buffer Management in Database Management Systems

• Applicant: Professor Dr. Felix Schuhknecht
• Subject Area: Data Management, Data-Intensive Systems, Computer Science Methods in Business Informatics
• Term: since 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 534271964

The buffer manager, a central DBMS component, allows disk-resident systems to handle working sets larger than the available main memory. While many DBMSs implement explicit buffer management, where page swapping is steered and performed fully by the DBMS, implicit buffer management offloads this responsibility to the page swapping mechanism of the OS, which highly simplifies the DBMS. While both approaches have their reasons to be, they unfortunately also struggle under performance and correctness problems in certain situations. Interestingly, possible solutions to these problems largely depend on whether the OS kernel can be modified (with OS-modification) or not (without OS-modification) to assist the buffer management in its task. Fortunately, we see room for improvement in both cases. In the applied-for project, we will therefore target to improve buffer management in setups with and without OS-modifications. First, we will perform an independent experimental in-depth comparison between explicit and implicit buffer management on modern systems across their design space to identify the precise current standing of both variants. To the best of our knowledge, no such independent comparison exists currently. Second, we will improve buffer management in scenarios where OS-modification is not an option. Here, we focus on explicit buffer management and propose to extend the state-of-the-art method with a manually managed page pooling mechanism that exploits (existing) features of the virtual memory subsystem of the kernel. This will remove excessive costs that currently occur under frequent page de- and reallocations. Third, we will shift the focus to improve buffer management for scenarios where OS-modification is possible. In this situation, instead of modifying the explicit variant which still requires significant maintenance effort on the DBMS-side, we aim at improving true implicit buffer management. To do so, we will extend the kernel to ensure transactional safety in the presence of in-place page eviction, removing a major downside of this variant.

• DFG Programme: Research Grants