The speed of modern network standards has long surpassed hard disk bandwidths and approaches the speed of machine-internal interconnect technologies (memory bus, PCIe, etc.). This suggests to put the ancient wisdom of distributed database systems–try to avoid communication at (almost) any cost–upside-down. In Data Cyclotron, a joint project with the database group at CWI in Amsterdam, the strategy is to leverage the available bandwidth, rather than spending efforts in trying to avoid it.
The crux is, however, that the traditional way of interfacing with the network is highly CPU intensive (an old rule of thumb is that about 1 GHz of CPU performance is needed for every Gb/s of network speed). This cost can be avoided with remote direct memory access (RDMA), a hardware acceleration feature available in many modern network cards.
RDMA is not a drop-in replacement for traditional network stacks. Rather, it requires a careful adaptation of algorithms to match the expectations of the RDMA stack. An example of an algorithm that includes such adaptations is cyclo-join, an RDMA-accelerated join algorithm for distributed databases.
Project collaborators: