Snowfall: Hardware Stream Analysis Made Easy
Jens Teubner and Louis Woods
Proceedings of the 14th GI-Fachtagung Datenbanksysteme für Business, Technologie und Web, Kaiserslautern, Germany, March 2011.
Field-programmable gate arrays (FPGAs) are chip devices that can be runtime-reconfigured to realize arbitrary processing tasks directly in hardware. Industrial products as well as research prototypes demonstrated how this capability can be exploited to build highly efficient processors for data warehousing, data mining, or stream analysis tasks.
On the flip side, the construction of dedicated hardware circuits requires considerable engineering efforts and skills that are often not available in application-focussed development teams. To bridge this gap, at ETH we have developed a set of tools that aid developers of high-performance stream processing solutions and enable agile hardware generation for changing application demands.
In this demonstration, we showcase Snowfall, a compiler tool for low-level stream analysis. Comparable to scanner generators for software-based systems (e.g., lex/flex), Snowfall can be used to decode incoming data streams in hardware, react to low-level patterns in a stream, and perform initial input data analysis. Snowfall plays well together with Glacier, a query-to-hardware compiler that we described and demonstrated earlier. A typical use case is to use Snowfall for input parsing and pre-processing, then perform SQL-style query processing on top with a hardware query plan obtained with the help of Glacier.
In the demo, we illustrate Snowfall based on a real-world use case with exceptionally high demands for throughput and latency. With the help of Snowfall, we perform risk checking for financial trading applications. Snowfall allows for a declarative description of the problem, yet will generate a hardware circuit that can process input streams in real time.
camera-ready for BTW 2011
submission to BTW 2011 (accepted)