In this file are the verbatim referee reports from the ISSAC 2001 program committee chair. There are comments below. -----------verbatim reports------------ Subject: ISSAC 2001 - Submission #30 Date: Mon, 19 Mar 2001 18:00:10 -0500 From: Gilles Villard To: fateman@cs.berkeley.edu CC: Gilles.Villard@ens-lyon.fr Dear Richard Fateman, This note is to inform you that your submission "A critique of openmath and thoughts on encoding mathematics ....." to ISSAC 2001 has been rejected. We had to take a selection among very good submissions, tough choices have finally been taken by the vote of the program committee. The grades given to your paper by the referees have not been sufficiently high. I include the reports below. Sincerely yours, Gilles Villard, PC Chair. -------------------- .......................... Fill in Referee-assigned grades below grades are to be in the range 1..4 (Use: 1, Excellent; 2, Good; 3, Below Average; 4, Poor.) I. RECOMMENDATIONS [ ] Excellent quality, definitely should be accepted [ ] Good quality, acception is recommended [X] Marginal quality, acception is inappropriate [ ] Poor quality, rejection is imperative II. SCOPE [2] Relevance to symbolic computations III. EVALUATION OF CONTENTS [2] Relevance [3] Originality [3] Importance [3] Difficulty [4] Correctness [3] Completeness [4] Literature (Cites previous work correctly?) IV. EVALUATION OF STYLE AND FORM [2] Clarity [3] Introduction (Relevance clearly stated in intro/abstract ?) [2] English [2] Length (Optimal, Appropriate, too long/short, unreadable because of length) V. COMMENTS TO THE AUTHOR(S) The author says at the outset, and often repeats, that the OpenMath project has the implicit aim of solving the software re-use problem. I do not think this is true: if Openmath solves any re-use problem, it is that of DATA re-use. This may in turn translate into software re-use, but that is not the primary goal of OpenMath as a whole (though the software vendors involved may think differently). The author says (p. 2) that this could be done in any ``language capable of representing attributed trees''. This is true, and XML is such a language. The real question is ``what should the attributes be''? On page 3, the author points out, quite correctly, that much mathematical discourse is (unintentionally) ambiguous. Indeed, I would go further, and say that there is deliberate, but unfortunate, re-use, e.g. in $\sin^{-1}$. He then claims that one neds to be skilled in the discipline. The OpenMath project discovered (in a paper unfortunately not in this collection) that knowing the Mathematics Subject Classification (often quite coarsely) solved a great many ambiguities --- e.g. $\pi$ in Group Theory is much more likely to be a permutation than 3.14159. It is certainly possible, as the author does (page 4) to believe that MathML has eaten OpenMath's lunch, but the fact that the MathML standard explicitly points to the OpenMath standard as a means of extensibility is at least a counter-argument. On page 5, the author says that semantics means ``the meaning of $Y$ to the program $X$'', and therefore that OpenMath does not solve the $n^2$ problem. As far as I can tell (there is no bibliography) the author does not refer to the Corless paper, which demonstrates some of the difficulties in this area, and the techniques for overcoming them, notably by having a reference `abstract' meaning. Where the author is right is that different programs $X$ may do different things to the objects they handle --- after all, there is no point in writing a symbolic integrator in \TeX{}, and not much point in wiriting a true typesetter in anything else. The statement on page 5 about the n-ary version of the axiom of commutativity for IEEE arithmetic is true, as indeed is the stronger one about the failure of associativity. The problem is rather that all numeric techniques are approximations, or, in the language of my paragraph above, what IEEE does with OpenMath objects such as an n-ary plus may well not be mathematically perfect, but that is not the fault of OpenMath: we have expressed our idea correctly, and then sent it to an imperfect program. The reference to Kajler & Soiffer is a good one, in terms of display, and indeed might be releant to asking how computer algebra systems generate MathML-P, whether or not this is done via OpenMath. At the top of page 7, I fear that the author has fundamentally missed the point. His students on their beginning programming class were rendering mathematics on a known system, whereas the purpose of the WWW, and therefore MathML, is to render on an unknown system. Hence the conversion via XSL into MathML is done at the sending end, but the receiving end renders that, which may be into speech rather than 2-D diagrams at all. It is therefore right that the XSL should describe, not, as the author says, the program to display the object, but rather the pre-rendering (Not in the sense of sin x rather than sin (x), but issues of layout) format of the object. Conversely, I agree that his point further down the page about ``written mathematics'' (which I take really to mean ``high-resolution quality mathematical printing'') being best rendered by \TeX{} is well-taken, and indeed one unsolved MathML challenge is high-quality (i.e. via \TeX{} in most cases) printing. I find the attack on Strotmann/Kahout on page 8 hard to follow. What S/K are essentially saying is that OpenMath HAS learnt, and HAS a better system than most computer algebra systems (say), but (though they do not make this as explicit as I will) that the common calculus technique $$ \int_0\sin(x)\d x $$ as a short-hand for $$ \int_0^x\sin(x)\d x = \int_0^x\sin(y)\d y $$ (by the Openmath rules for bound variables) is revolting to computer scientists. On page 9, the author objects to the fact that OpenMath did not use mathematics ``as published''. Clearly reproducing the whole research literature (and its CDs --- a non-trivial task) is difficult, but several studies into OpenMath versus published literature have taken place. The only one in this collection is Kohlhase's paper --- again not referred to. As a very minor point, the eulogy of Lisp on page 6 has unmatched parentheses. Even more minor, a missing stop at the end of the penultimate paragraph on page 7. In sum, I find it hard to see that the author has understood the issue he is criticising, though part of this may be a valid criticism of the presentation. I suggest it be published as a rebuttal in the same journal. .......................... .......................... Fill in Referee-assigned grades below grades are to be in the range 1..4 (Use: 1, Excellent; 2, Good; 3, Below Average; 4, Poor.) I. RECOMMENDATIONS [ ] Excellent quality, definitely should be accepted [ ] Good quality, acception is recommended [ ] Marginal quality, acception is inappropriate [x] Poor quality, rejection is imperative II. SCOPE [1] Relevance to symbolic computations III. EVALUATION OF CONTENTS [1] Relevance [3] Originality [3] Importance [3] Difficulty [3] Correctness [3] Completeness [4] Literature (Cites previous work correctly?) IV. EVALUATION OF STYLE AND FORM [ ] Clarity [ ] Introduction (Relevance clearly stated in intro/abstract ?) [1] English [1] Length (Optimal, Appropriate, too long/short, unreadable because of length) V. COMMENTS TO THE AUTHOR(S) This article discusses several relatively well-known issues related to the communication of mathematical data. While this is a topic of great importance to symbolic computation, there is little in the way of original contribution in this aspect of the paper. The discussion related to OpenMath seems to be more an analysis of the author's conclusions about OpenMath rather than an analysis of OpenMath itself. There are some valuable observations in this discussion, but these need to be fished out of the rest of the article. .......................... -----------end of verbatim reports------------ Comments on referee 1. This person thought the paper was of marginal quality. We are all entitled to our opinions. It is relevant to symbolic computation (2=good) but neither original nor important (3=below average). Its "difficulty" is poor. Does that mean it is easy? or very difficult? This ambiguous rating has been in the referee forms for ISSAC for a long time. Am I the only one who can't figure it out? Apparently my paper does not cite literature, though the first line of the abstract says that it all depends on SIGSAM Bulletin vol 34 number 2. I guess the referee wanted to see this line in the bibliography. It may also be the case that this publication is inaccessible to many people and therefore not a proper reference. It seems that the effort to make the SIGSAM Bulletin on-line has faltered. Too bad. Apparently my style and form are mostly (2=good) except for the introduction. now for the comments to the author. It seems the referee misses the point of Openmath as explained by my paper, which must either mean I did not make the point clearly enough, or the referee was not willing to hear it. Although OM seems to be concerned about notation, the REASON for OM cannot be the notation. For that we already have conventional mathematical notation, for good or ill. Some people working on OpenMath may think their primary goal is data related, but the IMPLICIT AIM is to solve the SOFTWARE problem. If there is no software involved,who cares about their notation? There are many notations for mathematics. Look at Leibnitz. Newton. Godel. Russell/Whitehead. Turing. Bourbaki. Wolfram(!) Do we need another one? Reading this referee's report one would perhaps conclude that OM was a great success. Instead it has been, so far, without any impact except perhaps as a footnote to MathML. -------------------------------- Comments on Referee #2's report This person thought the topic relevance was "excellent" but declines to credit the content with anything better than below-average(3). He/she seems to think I've gotten the right style and length. The problem here is that the referee wants someone to write a paper explaining OpenMath. I can't image why that would be original. After all, there are many many words written in the OM archives. Instead, I wrote a paper trying to explain why it is wrong-headed. This too appeared in the OM archives, and attracted substantial comments, (some of the comments were private so as to not adversely affect funding or inter-personal relationships!) If the only people at conferences to discuss OM are the advocates of OM, then it must remain a mystery to the community -- perhaps handed from person to person-- as to why it adoption seems to be so slow. Of course I could be just impatient and will eventually be shown to be wrong, and OpenMath is just what is needed. I note that in the program for ISSAC 2001 there are no accepted papers on OpenMath. I have no idea if any other papers on OpenMath were submitted. But then computer algebra systems papers may no longer have much relevance to ISSAC conferences, since of the 46 papers perhaps 2 concern systems issues. April, 2001. Richard Fateman