Breaking the memory wall in MonetDB

@article{Boncz2008BreakingTM,
  title={Breaking the memory wall in MonetDB},
  author={Peter A. Boncz and Martin L. Kersten and Stefan Manegold},
  journal={Commun. ACM},
  year={2008},
  volume={51},
  pages={77-85}
}
In the past decades, advances in speed of commodity CPUs have far outpaced advances in RAM latency. Main-memory access has therefore become a performance bottleneck for many computer applications; a phenomenon that is widely known as the "memory wall." In this paper, we report how research around the MonetDB database system has led to a redesign of database architecture in order to take advantage of modern hardware, and in particular to avoid hitting the memory wall. This encompasses (i) a… Expand
A near-data select scan operator for database systems
TLDR
This dissertation implemented the near-data select scan in the row/column/vector-wise query engines for x86 and two HMC extensions, HMC-Scan and HIPE-Scan achieving performance improvements of up to 3.7× for HMC -Scan and 5.6× for HIPE -Scan during the execution of query filters that depends on in-memory data. Expand
RC-NVM: Enabling Symmetric Row and Column Memory Accesses for In-memory Databases
  • Peng Wang, Shuo Li, +6 authors Tao Zhang
  • Computer Science
  • 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA)
  • 2018
TLDR
A dual-addressable memory architecture based on non-volatile memory, called RC-NVM, to support both row-oriented and column-oriented accesses is proposed and a group caching technique that combines the IMDB knowledge with the memory architecture to further optimize the system is proposed. Expand
Towards efficient analytic query processing in main-memory column-stores
TLDR
This dissertation is to design efficient algorithms for scan, sort and join by judiciously exploiting every bit of RAM and all the available parallelisms in each processing unit to enable skip-scan, a new fast scan that enables both data skipping and early stopping without any space overhead. Expand
User Mode Memory Page Management: An old idea applied anew to the memory wall problem
TLDR
A feasibility study is conducted to determine whether the MMU for each application process should be virtualised such that it has direct access to its own MMU page tables and the memory allocated to a process is managed exclusively by the process and not the kernel. Expand
Hardware-Oblivious Parallelism for In-Memory Column-Stores
TLDR
This work proposes an alternative design for a parallel database engine, based on a single set of hardware-oblivious operators, which are compiled down to the actual hardware at runtime, which reduces the development overhead for parallel database engines, while achieving competitive performance to hand-tuned systems. Expand
Cache Conscious Column Organization in In-Memory Column Stores
Cost models are an essential part of database systems, as they are the basis of query performance optimization. Based on predictions made by cost models, the fastest query execution plan can beExpand
Elastic online analytical processing on RAMCloud
TLDR
A distributed in-memory database architecture that separates the query execution engine and data access enables the usage of a large-scale DRAM-based storage system such as Stanford's RAMCloud and the push-down of bandwidth-intensive database operators into the storage system. Expand
OLTP through the looking glass, and what we found there
TLDR
Overheads and optimizations are identified that explain a total difference of about a factor of 20x in raw performance in modern OLTP databases, and it is shown that there is no single "high pole in the tent" in modern database systems, but that substantial time is spent in logging, latching, locking, B-tree, and buffer management operations. Expand
Memory-mapped I/O on steroids
TLDR
This paper presents Aquila, a library OS that allows applications to reduce I/O overhead by customizing the memory-mapped I-O (mmio) path for files or storage devices, and shows the benefits of Aquila in two cases: using mmio in key-value stores and utilizing it in graph processing applications to extend the memory heap over fast storage devices. Expand
Accelerating mono and multi-column selection predicates in modern main-memory database systems
TLDR
This thesis tackles the aforementioned challenges of creating hardwaresensitive operator implementations automatically and exploiting the relation between multiple selection predicates and introduces the abstraction of code optimizations as a means to generate hardware-sensitive code variants automatically. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 35 REFERENCES
Database Architecture Optimized for the New Bottleneck: Memory Access
TLDR
A simple scan test is used to show the severe impact of main-memory access bottleneck, and radix algorithms for partitioned hash-join are introduced, using a detailed analytical model that incorporates memory access cost. Expand
Optimizing Main-Memory Join on Modern Hardware
TLDR
The partitioned hash-join is refined with a new partitioning algorithm called radix-cluster, which is specifically designed to optimize memory access, and the effect of implementation techniques that optimize CPU resource usage is investigated. Expand
Cache Conscious Algorithms for Relational Query Processing
TLDR
It is shown that there are significant benefits in redesigning traditional query processing algorithms so that they can make better use of the cache, and new algorithms run 8%-200% faster than the traditional ones. Expand
Cache-Conscious Radix-Decluster Projections
TLDR
It turns out, that the strategy of first executing the join and only afterwards dealing with the projection columns on DSM, in combination with a new finely tunable algorithm called Radix-Decluster, outperforms all previously reported projection strategies. Expand
Understanding, modeling, and improving main-memory database performance
TLDR
This thesis analyzes the impact of modern hardware on main-memory database performance and develops new techniques to better exploit the available hardware resources and designs detailed cost models to predict the performance behavior of database algorithms. Expand
DBMSs on a Modern Processor: Where Does Time Go?
TLDR
This paper examines four commercial DBMSs running on an Intel Xeon and NT 4.0 and introduces a framework for analyzing query execution time, and finds that database developers should not expect the overall execution time to decrease significantly without addressing stalls related to subtle implementation issues. Expand
MonetDB/X100: Hyper-Pipelining Query Execution
TLDR
An in-depth investigation to the reason why database systems tend to achieve only low IPC on modern CPUs in compute-intensive application areas, and a new set of guidelines for designing a query processor for the MonetDB system that follows these guidelines. Expand
STEPS towards Cache-resident Transaction Processing
TLDR
Steps is proposed, a technique that minimizes instruction cache misses in OLTP workloads by multiplexing concurrent transactions and exploiting common code paths, and yields up to 96.7% reduction in instruction caches misses for each additional concurrent transaction. Expand
Conjunctive selection conditions in main memory
TLDR
It is demonstrated that branch misprediction has a substantial impact on the performance of an algorithm for applying selection conditions, and a cost model that takes branch prediction into account is proposed and a query optimization algorithm that chooses a plan with optimal estimated cost is developed. Expand
Buffering databse operations for enhanced instruction cache performance
TLDR
This work answers the question "Why does a database system incur so many instruction cache misses" and proposes techniques to buffer database operations during query execution to avoid instruction cache thrashing. Expand
...
1
2
3
4
...