Reviews for paper FPGAs for Dynamic (XML) Query Workloads, submitted to (P)VLDB 2011.
Overall rating: reject
(This work was later published as “Skeleton Automata for FPGAs: Reconfiguring without Reconstructing” at SIGMOD 2012.)
Reject
The paper proposes to split FPGA programs into two parts, a static compiled circuit and a dynamic configuration. This allows to reconfigure FPGAs for different queries much quicker. XML path projection is used as an example. The paper is well written, and the ideas are sound, but FPGA-based query processing has been introduced to the DB community before (by the same authors). The authors claim the separation of parameters and template circuit is their key contribution, but this ships commercially as part of the Netezza appliances already, and hence is less revolutionary than they claim.
S1 well written, illustration-based explanation suits fpga algorithms
S2 more expressive queries than very basic previous fpga approaches
S3 good introduction
W1 claimed key innvoation is commercially available
W2 no comparison to other approaches (pure software based, fully hardcoded fpga without parameterization,...), according to suitable metrics (throughput/power/$cost/...)
YES
With some new ideas
XML processing on fpgas new, parameterization not really new
Improvement over existing work
Solid work
Ok, but certain claims are not covered by the experiments
Excellent: careful, logical, elegant
No
In the intro, you state that "the language it accepts ... can be arbitrarily changed at runtime", which even with the context of automatons is a little overgeneralized. Just some state transitions are configurable, not the whole language.
The techniques are well presented, but I am not sure where the sweet sport of this technique is, and the paper does little to discuss this. Where is this technique performing well in the overall design space, what are the end-to- end applications where you need projection like this, with quickly switchable configurations, and how do alternatives compare, e.g. purely software based approaches.
Reject
This paper targets the problem of high volume XML filtering. It is based on field-programmable gate arrays (FPGSs) technologies that are designed and used for offline (static) query compilation. The paper proposed an dynamic solution which allows runtime query workload reconfiguration. This is made possible by 1) a generic hardware implementation of non-deterministic finite state automation (NFA) for XML path filtering and 2) taking query workload as configuration parameters in runtime. As a result, no need to learn query workload ahead of time and changes can become effective immediately. The experiments demonstrate the scalability of the solution; it also showed that the solution can be used as a network filter to let users benefit from reduced memory/parsing overhead.
1. The main contribution is it proposed a reconfigurable solution for XML projection using FPGAs. Query workflow changes can be applied in runtime and can take effective immediately.
2. Experiments showed the scalability of the approach as well as its great system integration opportunities (e.g. used as a network filter).
3. The paper is well presented. It is logical and clear.
Many XML projection paths are not independent and may share some portions. Since chip space is limited resource, authors may want to consider sharing partial path matching chains to save chip spaces and further improve performance.
YES
With some new ideas
Improvement over existing work
Solid work
Very well presented, supports the claims made in the paper
Excellent: careful, logical, elegant
Well, not really
The main contribution is it proposed a reconfigurable solution for XML projection using FPGAs. One XML project path can be translated into a NFA; they thus implemented a generic path-matching engine consisting of a chain of segment matchers -- each matcher implements one of the NFA segments. To achieve runtime re-configurability, the engine stores query workload specific information in on-chip storage and the changes can be applied by writing new parameters into the storage cells. Multiple-path matching can be achieved by concatenating multiple path matching chains on the chip with an additional merging segment. This layout can also efficiently utilize chip space resources.
The system is implemented and evaluated on widely available and low-cost FPGA hardware. The evaluation demonstrated the scalability of the system -- it can maintain clock frequency even with high chip utilization (as high as 70-80%). It also showed the potential of using this system as a network filter --- users can benefit from reduced XML parsing overhead and memory consumption.
The paper is well presented. It is logical and clear. The main body of the paper focused on the key principles of the system and implementation details are put into appendix to give readers a more complete view.
As stated in section 4, authors may want to explore possibilities of sharing partial path matching chains to save chip spaces and further improve performance.
A minor comment is in Figure 7 on takens: for XML element < abc id="..." / > , the token of " > " is both OPENINGTAGEND and CLOSINGTAGEND,
Reject
This paper presents a generic FPGA circuit design for XML filtering which can change at runtime without the need of reprogramming the FPGA. To achieve that XML filtering is broken down to three phases: the initial XML parsing, the final XML serialization and, in the middle, the matching which is achieved by a set of segment matchers which decide if the streamed XML matches the filter. I think that this is a well-written paper. Unfortunately, the authors do not convince why it is challenging to achieve such a design and the reader does not learn something new. From the products of companies such as Netezza and Xtreme Data we already know that it is possible to have FPGA-based systems that process workloads which change at runtime. On the other hand, the authors should quantify what is the overhead of adding such flexibility to the FPGA design (e.g. what is the difference in performance and space of static designs vs. the one presented). In addition, this design is so specific to XML filtering (at a point which is questionable why the authors decided to even put parentheses around the word XML in the title) and the authors should discuss lessons learned -- as paper is now it seems difficult to be used as a guide for the implementation of other data management operations which change are runtime on FPGAs.
S1. It shows that we can have FPGA-based XML filtering that can change at runtime without the need of reprogramming the FPGA.
S2. Well written, clear objective.
W1. Incremental improvement over existing work. The authors should present convincing arguments why it is a challenge to design such a configurable XML filtering FPGA circuit, and discuss lessons learned for achieving it, so that it can be useful to the community. In addition, products by companies such as Netezza and Xtreme Data already execute dynamic query workloads.
W2. Paper does not quantify what is the overhead of adding the configurable at runtime ability. It would be useful to have a set of experiments which show the difference in the performance and space of static designs vs. the proposed one.
W3. It is not clear how dynamic the design is. In particular, how is the number of pipelines (of xml parser, segment matchers and serializers) or the number of segment matchers in each pipeline decided? If there is need to change the resource allocation, doesn't this mean that the FPGA needs to be reprogrammed?
YES
Ideas are too simple (say how)
This paper presents a more generic FPGA circuit design which is able to execute XML projections that can change over time. Someone a bit familiar with hardware would consider this the result of good engineering, relevant to what has already been done by companies that use FPGAs, such as Xtreme Data.
Improvement over existing work
Syntacticaly complete but with little contribution
Ok, but certain claims are not covered by the experiments
Excellent: careful, logical, elegant
Well, not really
This paper presents a generic FPGA circuit design for XML filtering which can change at runtime without the need of reprogramming the FPGA. If it had been submitted to PVLDB it would deserve a second chance, but at its current form it needs some improvement to make the cut for VLDB.
D1. It is a well-written paper.
D2. I am not really convinced why it is challenging to achieve such a design and hence what is its important contribution.
D3. There is scarce discussion any lessons learned for achieving it, so that it can be useful to the community.
D4. Isn't dynamic FPGA-based query execution already implemented by companies such as Xtreme Data and Netezza (most probably and Kickfire)?
D5. The authors should quantify what is the overhead of adding such flexibility to the FPGA design. For example, it would be interesting to see the difference in the performance and space of a static FPGA design vs. the one presented for a set of different XML projections (e.g. the worst case and the best case). Also, it would be interesting to give a specific number of how much time does it take now to reprogram an FPGA (the authors mention in the intro that the pre-compilation is expensive).
D6. The design is so specific to XML filtering and the authors do not discuss any lessons learned or difficulties that they encountered that it seems difficult to be used as a guide for the implementation of other data management operations which can be reconfigured at runtime on FPGAs.
D7. Some implementation details are missing. For example, one thing that it was not clear is how it is decided the number of pipelines (parsers, segment matchers, and serializers) or the number of segment matchers per pipeline. Doesn't it need reprogramming of the FPGA if there is need to change the allocation of those resources?
D8. By the way, one piece of related work that it would be good to cite is [1] since it seems to be the base of many hardware-based XML filtering implementations.
[1] J. V. Lunteren, T. Engbersen, J. Bostian, B. Carey, and C. Larsson. XML accelerator engine. In Int. Workshop on High Performance XML Processing, 2004.