Execution Strategies for Compute Intensive Queries in Particle Physics
Title
Execution Strategies for Compute Intensive Queries in Particle Physics
Authors
Maximilian Berens
Published
In Proceedings of the 31th GI-Workshop Grundlagen von Datenbanken (GvDB' 19).
Download
via file.
Abstract
Data analysis in many scientific domains, such as high energy physics, pushes the available resources of even big research collaborations to their limit, because it not only requires huge amounts of data but also compute intensive calculations. Current approaches require increased storage and computing resources, lack sufficient flexibility and are still far away from interactivity. In this paper we present ideas to cope with challenges posed by compute-intensive analyses of high-volume data in a many-user- and resource-restricted environment. Extreme selectivity of user queries (that is inherent in high energy physics analyses, for instance) permits us to reduce the computational effort to a small portion, when irrelevant data can be discarded by more efficient means. Our focus lies on providing execution strategies for analysis queries, guided by the expertise of the user and
executed as a scalable, scan-based preselection (introduced in DeLorean). Future research will encompass the development of a compact data summary that facilitates fast, columnar scan queries, as well as a domain specific language that provides us with the required information to generate them.
Projects