Abstract
The MxKernel project is part of the DFG Priority Program 2037 “Scalable Data Management for Future Hardware”. MxKernel is a joint project with Olaf Spinczyk's Embedded System Software Group at TU Dortmund University.
MxKernel's theme is Database/Operating System Co-Design. The project is driven by new application classes that put immense pressure on systems software to process data faster and at larger scale. “Real-time analytics” demands high transaction rates concurrent to complex analysis queries; “Industry 4.0” or “Internet of Things (IoT)” applications challenge database and operating systems to process data distributed, scalable, and in real time.
Emerging hardware platforms can provide the necessary base performance. But leveraging it has become difficult; it is still largely unclear how large degrees of parallelism, complex memory hierarchies, or increasing hardware heterogeneity can be adequately supported by systems and application software. Clearly, different pieces of systems software must collaborate much more closely than what is available today.
With MxKernel, we want to build a new bare-metal runtime system. MxKernel provides very lightweight resource management for database and operating system, which both run as equal peers on top of the runtime. In MxKernel, heterogeneity and parallelism become first-class citizens, ready also for memory hierarchies or modern storage technologies (such as non-volatile memories). Instead of a classical "thread" model, MxKernel provides MxTasks as an abstraction for work items. MxTasks describe closed units of work, for which MxKernel will guarantee atomic execution.
MxTasks resemble “events” in event-based systems and inherit their excellent scalability as well as the potential to scale out to large, distributed systems. But MxTasks can be a very elegant abstraction also for heterogeneity and resource sharing. In the MxKernel project, we will enrich MxTasks with meta data, such as code variants (to support heterogeneity); memory access behavior (to improve cache efficiency and support memory hierarchies); or dependencies between MxTasks (to improve scheduling and avoid synchronization cost). With precisely the required meta data available, MxKernel can provide a lightweight, yet highly efficient form of resource management, even across applications, operating system, and database—a characteristic that we see increasingly important, e.g., in the light of cloud-based or consolidated systems.
In the project, we will investigate the MxKernel concept from both an operating system and a database system angle. The goal is to understand the feasibility of the concept; its implications on the design of (database) software; as well as its potential for performance and scalability. We will develop methods and guidelines to realize database tasks using the MxTask concept, including concrete solutions for critical database tasks (indexing and transaction processing).
The MxKernel project will be driven by the development of demonstrators. Building on conceptual results developed in the project, we want to quickly push our ideas to working prototypes, which will be modelling relevant and practical use cases from the transaction processing and IoT domains.
The MxKernel project officially starts on October 1, 2017.
Publications
Project Information
MxKernel Students
- Michael Müller (PhD student, ESS Group)
- Jan Mühlig (PhD student, DBIS Group)
Master/Bachelor Theses
- David Vu. Prefetching in parallelen Umgebungen. Bachelor's Thesis. TU Dortmund University. DBIS Group. May 2021.
- Kristina Sax. Transactional Memory zur Synchronisation in Task-basierten Umgebungen. Master's Thesis. TU Dortmund University. DBIS Group. May 2021.
- Steffen Nießing. Transactional Memory in systemnahen Datenstrukturen. Bachelor's Thesis. TU Dortmund Universiry. DBIS Group. August 2020.
- Omar Mutlak. HTTP-Unterstützung für MxKernel. Bachelor's Thesis. TU Dortmund University, DBIS Group. Januarary 2020.
- Anna-Sophie Picker. Placement-Optimierung in Task-basierten Umgebungen. Bachelor's Thesis. TU Dortmund University, DBIS Group. October 2019.
- Sebastian Golbert. Lockless parallel skiplists on the MxKernel runtime system. Master's Thesis. TU Dortmund University, DBIS Group. August 2019.
- John Maleki. Taskbasierter Zugriff auf verteilte Datenbank Indexstrukturen. Master's Thesis. TU Dortmund University, DBIS Group. July 2019.
- Michael Ihne. MxKernel-Scheduling: Strategien für die Nutzung von Speicher- und Rechenressourcen in Many-Core Systemen. Master's Thesis. TU Dortmund University, ESS Group / DBIS Group. March 2019.
- Roland Kühn. Aggregation auf Manycore-Architekturen. Master's Thesis. TU Dortmund University, DBIS Group. March 2019.
- Philipp Menken. Parallel AES Processing Based on the MxKernel Task Model. Master's Thesis. TU Dortmund University, DBIS Group. January 2019.
- Rico Ahlbäumer. Skalierbare Hashtabellen für moderne Multi-Core-Systeme. Bachelor's Thesis. TU Dortmund University, DBIS Group. March 2018.
- Jan Mühlig. MxOS: Ein Betriebssystem mit Threads auf Task-Basis. Master's Thesis. TU Dortmund University, ESS Group / DBIS Group. September 2017.
- Karolina Hilkens. Lightweight Object Threads for In-Memory Database Indexes. Master's Thesis. TU Dortmund University, DBIS Group. June 2017.
Funding
The MxKernel project is funded by the Deutsche Forschungsgemeinschaft (DFG); grant number TE 1117 (SP 968); priority program SPP 2037.
