DaMoN 2020 Reviews
Reviews for paper Efficient Generation of Machine Code for Query Compilers, submitted to DaMoN 2020.
Overall Rating: accept
Reviewer 1
Is the topic of the paper relevant to and important to the DaMoN community?
Yes
Is the paper readable and well organized?
Definitely - very clear
Are the research contributions substantial and novel enough to warrant acceptance?
Yes - the contributions are above the bar
Are the paper's methodology, assumptions, models, and arguments free of serious flaws?
Yes
Overall rating
Accept
Briefly justify your Overall Rating. Papers with Reject or Strong Reject ratings should have at least one negative score on Q1-Q4.
The paper introduces Flounder IR, a lightweight intermediate representation (IR) that is designed specifically for relational operations. The main technical contribution that is described in the paper is an algorithm for faster translation algorithm that performs faster register allocation and machine instruction generation. The results support that Flounder IR offers a favorable compile time/run time trade-off compared to LLVM, an existing general-purpose solution. The technical contribution is rather incremental, but the paper complements this strength with a very compelling vision for JIT-based methods that are tailored to relational query processing. Overall, the paper is approachable, well organized, with substantial new research contributions and experimental support to merit a presentation slot at DaMoN.
Detailed Comments
The paper nicely articulates a broader need for domain-specific JIT methods, and the choice of integrating Flounder IR at the query processing level makes sense for short and complex queries. The vision described in the paper is very compelling.
In addition to the strong vision, the experimental results show that Flounder IR offers a sensible compile time/run time tradeoff where compilation time is significantly faster for a run time penalty that is a few percentage points at most. However, the evaluation uses "complexity" to only refer to the number of attributes being projected, the number of tables being joined and the number of clauses in the where clause. The concern here is whether the run time cost would accumulate in queries of higher structural complexity, such as combinations of nested queries, aggregation and ordering clauses. It would be nice to see how Flounder IR handles queries that support more complex workloads, such as TPC-H and TPC-DS, in the future.
Although these are not weaknesses per se, I would have appreciated if the paper went further in discussing the following directions:
* How are different SQL types supported in Flounder IR.
* NULLs add code complexity to the evaluation code. How are NULLs handled in Flounder IR.
* Produce/consume has been criticized as not being very data-friendly in terms of the stack operations needed and the one-tuple-at-a-time approach that may not fully utilize the memory bandwidth. Does Flounder IR support other models than produce/consume? In particular, can Flounder IR support vectorized execution, given that ASMJit claims to support the full AVX-512+ instruction set?
Finally, some minor presentation issues:
* (Figure 1) It would be nice to add an axis title to the vertical axis in the "JIT Compile Time" box. This is where readers would look first, instead of the caption of the box or the text to understand what is being shown.
* (Section 5) The evaluation uses "complexity" a lot, but it is a very generic term. Consider replacing with a specific term, such as "number of joined tables", "number of projected attributes", "number of clauses in selection", etc. This would allow clear naming of the axis title and units in Figure 8, which is fairly confusing as it is currently presented.
* (Figure 10) "total" may be a better term than "all" here.
If this is a full paper and it is rejected, woud you support its acceptance as a short (2 page) paper?
Yes
Is this paper a good candidate for invitation (of an extended version) to a special "Best of DaMoN" issue of the VLDB Journal?
Yes
Reviewer 2
Is the topic of the paper relevant to and important to the DaMoN community?
Yes
Is the paper readable and well organized?
Definitely - very clear
Are the research contributions substantial and novel enough to warrant acceptance?
Yes - the contributions are above the bar
Are the paper's methodology, assumptions, models, and arguments free of serious flaws?
Yes
Overall rating
Accept
Briefly justify your Overall Rating. Papers with Reject or Strong Reject ratings should have at least one negative score on Q1-Q4.
The main contribution of the paper is reduction of query JIT compilation time.
The results of this reduction vs. LLVM are excellent.
As a side-effect, the overall execution time is also improved in most cases.
This contribution is due to the clever IR of Flounder with its light-weight abstractions and integration of machine-level code generation
Detailed Comments
The abstractions and the translation algorithm has some novelty.
In order to prove the usefulness and practicality of these ideas, they should be developed further and cover (traditional query compilation) issues like columnar data layout, SIMD, etc.
If this is a full paper and it is rejected, woud you support its acceptance as a short (2 page) paper?
Yes
Is this paper a good candidate for invitation (of an extended version) to a special "Best of DaMoN" issue of the VLDB Journal?
Yes
Reviewer 3
Is the topic of the paper relevant to and important to the DaMoN community?
Yes
Is the paper readable and well organized?
Definitely - very clear
Are the research contributions substantial and novel enough to warrant acceptance?
Definitely - a significant advance
Are the paper's methodology, assumptions, models, and arguments free of serious flaws?
Yes
Overall rating
Strong Accept
Briefly justify your Overall Rating. Papers with Reject or Strong Reject ratings should have at least one negative score on Q1-Q4.
The paper tackles a timely and highly relevant topic. It does so in a very thorough and comprehensive manner with results that are conclusive and illustrative of the problem. I see the paper becoming a key reference in an are that is growing in importance.
Detailed Comments
Query compilation has become an important topic and, given the growing workloads, the overhead of query compilation is not something that can be further ignored. I like the paper in the way it poses the problem and explains its ramifications. I also like the way it treats the design space and the approach it takes.
Compared with other papers, this one gives the impression of having more solid results and much more work behind. The evaluation is impressive and quite convincing. It is also a perfect match for DaMON as these types of frameworks and the widespread use of intermediate representations is becoming a very important topic in industry that academic research might not be paying enough attention to.
If this is a full paper and it is rejected, woud you support its acceptance as a short (2 page) paper?
Yes
Is this paper a good candidate for invitation (of an extended version) to a special "Best of DaMoN" issue of the VLDB Journal?
Yes
Related Information