Spinning Relations: High-Speed Networks for Distributed Join Processing
Philip W. Frey, Romulo Gonçalves, Martin Kersten, and Jens Teubner
Proceedings 5th Int'l Workshop on Data Management on New Hardware (DaMoN), Providence, RI, USA, June 2009.
paper (PDF), presentation slides (PDF)
By leveraging modern networking hardware (RDMA-enabled network cards), we can shift priorities in distributed database processing significantly. Complex and sophisticated mechanisms to avoid network traffic can be replaced by a scheme that takes advantage of the bandwidth and low latency offered by such interconnects.
We illustrate this phenomenon with cyclo-join, an efficient join algorithm based on continuously pumping data through a ring-structured network. Our approach is capable of exploiting the resources of all CPUs and distributed main-memory available in the network for processing queries of arbitrary shape and datasets of arbitrary size.
camera-ready for DaMoN 2009
submission to DaMoN 2009 (accepted)