Lightweight Compression Techniques for Optimizing Complex Database Queries

• Applicant: Professor Dr.-Ing. Wolfgang Lehner
• Subject Area: Security and Dependability, Operating-, Communication- and Distributed Systems
• Term: from 2014 to 2018
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 255187874

Large data sets, which are managed by a database system, are usually stored in compressed form on external storage media. This approach saves capacity on external storage as well as reduces the data load time to main memory. However, during query processing in database systems all relevant data including intermediate results are kept and processed uncompressed in main memory. Thereby, main memory and processor caches in particular in many-core architecture are more and more the bottleneck. To tackle this problem, we investigate lightweight compression techniques for query processing to establish a balanced ratio between reduced data volume and additional expenses for compression. Our main idea is to compress each intermediate result during query processing to increase memory bandwidth and to reduce query processing time. For that purpose, we are going to develop intra as well as inter operator compression and integrate those techniques in the whole query processing. Moreover, the compression of intermediate results will be incorporated in query optimization using an appropriate cost model. From our point of view, a balanced lightweight compression is a trend-setting technique to reduce memory bottlenecks in upcoming many-core architectures and to facilitate scalable database applications.

• DFG Programme: Research Grants