Vectorization vs. compilation in query execution

  title={Vectorization vs. compilation in query execution},
  author={Juliusz Sompolski and Marcin Zukowski and Peter A. Boncz},
  booktitle={International Workshop on Data Management on New Hardware},
Compiling database queries into executable (sub-) programs provides substantial benefits comparing to traditional interpreted execution. Many of these benefits, such as reduced interpretation overhead, better instruction code locality, and providing opportunities to use SIMD instructions, have previously been provided by redesigning query processors to use a vectorized execution model. In this paper, we try to shed light on the question of how state-of-the-art compilation strategies relate to… 

Just-in-time Compilation in Vectorized Query Execution

This master thesis asks the question whether combining these two techniques can yield yet significantly better results than using each of them alone, and implements a JIT query execution infrastructure in VectorWise DBMS.

Exploring Query Compilation Strategies for JIT, Vectorization and SIMD

There is a whole design space to be explored, in particular when considering exploiting SIMD, and a new strategy called “in-register aggregation” is proposed that reduces memory pressure but also allows to compute on more compact, SIMD-friendly data types.

A simplified Architecture for Fast, Adaptive Compilation and Execution of SQL Queries

This paper argues that code compilation and execution techniques should be fully delegated to an existing engine rather than being reinvented by database architects, and proposes this conceptual architecture using WebAssembly and V8 as an example.

Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask

This paper experimentally compare the two models of vectorization and data-centric query processing by implementing both within the same test system, and finds that both are efficient, but have different strengths and weaknesses.

Radish : Compiling Efficient Query Plans for Distributed Shared Memory

An approach for translating query plans into distributed programs by targeting the partitioned global address space (PGAS) parallel programming model as an intermediate representation, which affords a natural adaptation of pipelining techniques used in singlecore query compilers and an overall simpler design.

Compiling queries for high-performance computing

This work takes important first steps integrating query processing and distributed HPC by presenting a technique Compiled parallel pipelines (CPP) for compiling relational query plans to programs suitable for high-performance computing platforms.

Compiling Database Queries into Machine Code

This paper shows how queries can be brought into a form suitable for efficient translation, and how the underlying code generation can be orchestrated, by carefully abstracting away the necessary plumbing infrastructure to build a query compiler that is both maintainable and efficient.

Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together At Last

A query processing model called "relaxed operator fusion" is presented that allows the DBMS to introduce staging points in the query plan where intermediate results are temporarily materialized and reduces the execution time of OLAP queries by up to 2.2× and achieves up to 1.8× better performance compared to other in-memory DBMSs.

Fast Compilation and Execution of SQL Queries with WebAssembly

This work investigates query execution by compilation to WebAssembly, and provides both low latency and high throughput, is adaptive out of the box, and is straight forward to implement.

Filter Representation in Vectorized Query Execution

This work analyzes each approach's strengths and weaknesses and offers recommendations on how to implement vectorized operations and shows that bitmaps perform better for operations that can be vectorized using SIMD instructions and that selection vectors perform better on all other operations due to cheaper iteration logic.



Compiled Query Execution Engine using JVM

Both an interpreted and a compiled query execution engine are developed in a relational, Java-based, in-memory database prototype, and experimental results show that, despite both engines benefiting from JIT, the compiled engine runs on average about twice as fast as the interpreted one, and significantly faster than an in- memory database prototype.

MonetDB/X100: Hyper-Pipelining Query Execution

An in-depth investigation to the reason why database systems tend to achieve only low IPC on modern CPUs in compute-intensive application areas, and a new set of guidelines for designing a query processor for the MonetDB system that follows these guidelines.

Balancing vectorized query execution with bandwidth-optimized storage

A new database system architecture is presented, realized in the MonetDB/X100 prototype, that combines a coherent set of new architecture-conscious techniques that are designed to work well together and achieves in-memory performance often one or two orders of magnitude higher than the existing approaches.

Conjunctive selection conditions in main memory

It is demonstrated that branch misprediction has a substantial impact on the performance of an algorithm for applying selection conditions, and a cost model that takes branch prediction into account is proposed and a query optimization algorithm that chooses a plan with optimal estimated cost is developed.

Block oriented processing of relational database operations in modern computer architectures

It is argued that a block-oriented processing strategy for database operations can lead to better utilization of the processors and caches, generating significantly higher performance.

Generating code for holistic query evaluation

The results show that HIQUE satisfies its design objectives, while its efficiency surpasses that of both well-established and currently-emerging query processing techniques.

DSM vs. NSM: CPU performance tradeoffs in block-oriented query processing

This paper focuses on the CPU efficiency tradeoffs of tuple representations inside the query execution engine, while tuples flow through a processing pipeline, and analyzes the performance in the context of query engines using so-called "block-oriented" processing.

Monet; a next-Generation DBMS Kernel For Query-Intensive Applications

This thesis is a reference to the Monet system in all its detail, and outlines an SQL front-end that uses Monet as a back-end, for constructing a full-fledged SQL compliant RDBMS including ACID properties.

Volcano - An Extensible and Parallel Query Evaluation System

  • G. Graefe
  • Computer Science
    IEEE Trans. Knowl. Data Eng.
  • 1994
Volcano is the first implemented query execution engine that effectively combines extensibility and parallelism, and is extensible with new operators, algorithms, data types, and type-specific methods.

Improving hash join performance through prefetching

This work shows that the standard hash join algorithm/or disk-oriented databases (i.e. GRACE) spends over 73% of its user time stalled on CPU cache misses, and explores the use of prefetching to improve its cache performance.