Suppose one has written a math/computer science paper that is more focused in the math part of it. I had a very complicated function and needed to find its maximum, so I used Mathematica (Wolfram) to do it. How do I explain that? "Using wolfram we find the maximum of $f$ to be $1.0328...$ therefore...". It looks very sloppy.
Asked
Active
Viewed 3,071 times
8 Answers
21
Welcome to MO! I believe the answer to your question depends on what Mathematica command(s) you used to find the maximum.
If you used the command Maximize[], then its output is exact and, in my view, can in general be trusted no less than the work of about any human. A reason for this belief is that Maximize[] has probably been tested at least hundreds or thousands as many times as an average proof by an average mathematician.
Still, others may disagree with this comparison between human work and the outputs of commands such as Maximize[], and so, then you may have to try to obtain the maximum in a way that can be verified by hand. Also, of course, Maximize[] can only solve comparatively easy maximization problems.
On the other hand, if you just used the command NMaximize[] -- which tries to find the maximum numerically, then its output can only be considered a suggestion -- certainly not a proof.
Iosif Pinelis
- 116,648
-
Out of interest, is ball arithmetic implemented in Mathematica? If so, does Maximize[] use this? – François Brunault May 16 '19 at 22:38
-
1@FrançoisBrunault : There is a command Interval[{a1,b1},{a2,b2},...] (giving the union of the intervals), and these interval objects can be the values of arguments of a function. This way, interval arithmetic is implemented. – Iosif Pinelis May 16 '19 at 23:35
-
21I’m one of those who would disagree that Mathematica can be “trusted no less than the work of about any human”. Mathematica absolutely has bugs, sometimes with embarrassingly simple inputs. I would at minimum cross-check with Sage/Maple/etc., if not actually verify mathematically. – wchargin May 17 '19 at 02:42
-
1According to the documentation https://reference.wolfram.com/language/guide/IntervalArithmetic.html Mathematica has interval arithmetic only for the basic functions. – François Brunault May 17 '19 at 06:34
-
@wchargin : Of course, you are free to disagree with anything. However, here are a few points to consider: (i) I did not say that Mathematica in general can be trusted no less than the work of about any human. I said this only about Mathematica's command Maximize[]. In fact, I also said that the output of Mathematica's command NMaximize[] is not a proof and hence cannot be trusted, at least in this sense. – Iosif Pinelis May 17 '19 at 13:33
-
11Previous command continued: (ii) To make a proper comparison between the work of humans and the output of Maximize[], you may want to recall that practically all mathematicians have made mistakes as well, sometimes in embarassingly simple situations, too. – Iosif Pinelis May 17 '19 at 13:33
-
@FrançoisBrunault : I think it makes some sense to have built-in interval arithmetic only for the basic functions, because I think this is how interval arithmetic will be usually used. However, one can of course write their own code to extend this to some-nonbasic functions by incorporating commands Maximize[] and Minimize[] and perhaps other "bounding" commands of their own. Do you know of Mathematica packages, or maybe even commands, that provide such an extension? – Iosif Pinelis May 17 '19 at 14:11
-
2In the beginning of my "Previous command continued:" comment, I made an obvious and embarrassing mistake. Of course, I wanted to say "Previous comment continued:". :-) I guess I was tired talking about Mathematica commands. – Iosif Pinelis May 17 '19 at 15:25
-
@IosifPinelis I think it is useful to have interval arithmetic for all kinds of functions, not just the basic ones, but I agree it's nontrivial so I understand if Mathematica doesn't have it. Also I don't like the idea of each person writing their own code (but ok functions must be heavily tested before becoming part of the software). Regarding Mathematica commands and packages, I don't find any convenient place to browse the entire documentation or to find packages. – François Brunault May 17 '19 at 15:37
-
To know what's possible today, you can look at the C library arb, developed by Fredrik Johansson, e.g. there is an impressive certified numerical integration. – François Brunault May 17 '19 at 15:40
-
Finally, I'm not sure Mathematica's interval arithmetic is certified, I couldn't find it mentioned in the doc. – François Brunault May 17 '19 at 15:43
-
-
@IosifPinelis I mean, given an interval $I$ of floating point numbers, and a continuous function $f$, output an interval $J$ such that we can ascertain that $f(I) \subset J$. I don't know but maybe Mathematica only wants to compute an interval $J'$ such that $f(I)$ is approximately equal to $J'$, which would be a different question. – François Brunault May 17 '19 at 16:51
-
@FrançoisBrunault : I think to get an exact interval in the output in Mathematica, one will need to convert the input interval of floating point numbers to an interval with rational endpoints. – Iosif Pinelis May 17 '19 at 17:08
-
@FrançoisBrunault : Concerning "certified numerical integration" you mentioned, what class(es) of functions will it work for? – Iosif Pinelis May 17 '19 at 17:10
-
1@IosifPinelis Sure, both humans and computers make mistakes. Human mistakes arise from insufficient (but still deep) understanding, and can be checked then corrected by improved understanding. But if a result is obtained solely by computers, it often means that humans hardly have any understanding. If there is any mistake, it can only be discovered and corrected once humans begin to have some understanding. It is either a mathematical proof, or a human certifiable program. We do mathematics for humans to understand, not to generate numbers for humans to believe. – Hao Chen May 17 '19 at 17:29
-
@HaoChen : I agree with your comment. Surely, human mistakes differ in character from those of a computer program -- even though such programs are written by humans, oftentimes based on rather deep results also obtained by humans (as is the case with commands like Maximize[], based in part on work by Tarski and his followers). However, my answer was about a much narrower subject, namely, the reliability of the output of such commands as Maximize[] vs. the corresponding work by a human. – Iosif Pinelis May 17 '19 at 18:20
-
@IosifPinelis In PO's case (very complicated function and a numerical output), I strongly doubt that Maximize[] has called NMaximize[], but PO could only become aware of this possibility if he is an expert, or has read the manual very carefully (does MMA throw a warning?). That aside, I support "computer aided proof", where humans make the plan and computers do the work. But I oppose "computer calculation" as proof, where humans give up understanding and ask software written by others to output a number. In PO's case, I would only accept if he completely understands Maximize[]'s algorithm. – Hao Chen May 17 '19 at 19:30
-
-
Thank you Dima. However, I don't see there what functions arb can integrate numerically with controlled/certified precision/accuracy. – Iosif Pinelis May 17 '19 at 21:53
-
@IosifPinelis The endpoints of the intervals are usually decimals. Guaranteed computation should be possible for pretty much any "explicit" continuous function, although one may need to write new code for each new function. Rigorous numerical integration is explained here: http://fredrikj.net/blog/2017/11/new-rigorous-numerical-integration-in-arb/ As you will see, singularities are also allowed as long as rigorous bounds are provided to the computer. – François Brunault May 18 '19 at 18:12
-
Note however that computing a funtion in the above sense ($f(I) \subset J$) is stronger than just having an algorithm to compute any $f(x)$ to arbitrary precision. – François Brunault May 18 '19 at 18:16
-
@FrançoisBrunault : From that blog post, it appears that the integrands have to be (piecewise) meromorphic. For some reasons, the post uses "holomorphic with poles" instead of "meromorphic". – Iosif Pinelis May 19 '19 at 02:33
-
@IosifPinelis It is not necessary that the integrand is (the restriction of) a holomorphic function, it only improves speed. For more details, see here http://arblib.org/acb_calc.html#integration The allowed integrands are described here http://arblib.org/acb_calc.html#c.acb_calc_func_t – François Brunault May 19 '19 at 06:39
-
@FrançoisBrunault : Alas, I am already lost at the beginning of the description at http://arblib.org/acb_calc.html#integration: "Typedef for a pointer to a function with signature:
int func(acb_ptr out, const acb_t inp, void * param, slong order, slong prec)"
and I don't see any explanations for Typedef, pointer, signature, acb_ptr out, const acb_t inp, void * param, slong order, slong prec anywhere on that page.
– Iosif Pinelis May 19 '19 at 13:56 -
I think it's not enough to trust that Maximize[] is implemented correctly. I'd additionally require trust that the components used to write that complicated function from OP are behaving correctly. – sdcvvc May 19 '19 at 14:32
-
@IosifPinelis I am sorry, in this document I only wanted to point out the explanations which are in plain English, namely "When called with order = 0 (...)" in the section you mention, which says for example the function can have discontinuities. The subsequent section "Integration" also has brief explanation on the method used. I don't know C very much so I'm afraid I couldn't explain in more details. You may want to look at Fredrik Johansson's website and blog http://fredrikj.net/ which I hope can be useful to you. Of course it takes time to learn C language, like any mathematical theory... – François Brunault May 21 '19 at 11:09
18
As is evident from the responses, different mathematicians have different attitudes towards computer calculations. If the computer calculation is not actually logically necessary for your proof, then it should be fine to say that you used Mathematica. Unfortunately (or fortunately, depending on your point of view), the world is rapidly progressing to the point where computers are actually needed to prove certain things. If a particular step in your proof requires a computation that consumes 1000 CPU hours then there is no way anyone is going to be able to "do it by hand." It doesn't sound like your computation was that large, but the point is that one is not always going to be able to sidestep the computer, and so the mathematical community needs to come up with a satisfactory approach to this sort of thing.
One approach, addressed in another MO question, is to archive or publish the computer software along with a paper. You might find some of the answers to that MO question useful. If you go this route, then I would recommend trying to perform the computation using some open-source software so that there is no problem with making the entire code publicly available.
Making the software available only addresses the issue of making your computation reproducible, and does not address the question of whether the computation is reliable. Again, assuming that the computer calculation is ineliminable, then formalizing the computation using a proof assistant (Coq, HOL Light, Lean, etc.) is the gold standard nowadays. Unfortunately, proof assistants are still not very user-friendly by the standards of the average mathematician. If you don't want to bite that bullet then probably the best you can do is to perform the calculations two or three times independently, using independent software and (preferably) a different person doing the programming. For a numerical calculation, as others have mentioned, it is important to use interval arithmetic or some other algorithm that guarantees correctness (assuming of course that the algorithm is implemented and executed correctly!). Floating-point computations are usually insufficient, according to commonly accepted standards of mathematical rigor.
Timothy Chow
- 78,129
-
13I’d add that using open-source software is akin to publishing in an open access venue, while using e.g. Mathematica is akin to publishing something behind a permanent paywall. – Dima Pasechnik May 17 '19 at 18:40
-
@DimaPasechnik : It is great that such open-source software (OSS) exists, at least because Mathematica has to compete with it. On the other hand, Mathematica is more universal and in most cases more powerful than the corresponding OSS. So, assuming that money is an expression of the expended time, what will be the balance between (i) paying for Mathematica and (ii) spending time to learn and use a variety of specialized and oftentimes less powerful OSS? – Iosif Pinelis May 17 '19 at 19:03
-
5what is the balance of publishing in Elsevier journal, compared to publishing in a refereed arxiv.org-based journal? well, these without funds wont’t read you. Oh, but the Elsevier shareholders obviously would be happy... – Dima Pasechnik May 17 '19 at 19:20
-
3also, imagine for a moment Wolfram Inc stopping with Mathematica for some reason. then all this code would quickly become next to useless... just like texts written in ChiWriter... – Dima Pasechnik May 17 '19 at 19:24
-
@DimaPasechnik I was a heavy user of Sage until recently, when I begin to work a lot with elliptic functions. I tried many OSS (pari/gp, mpmath, scipy etc.) and finally obtain a Mathematica license from my university. Although Mathematica also fails me, I must say that industrial softwares are very powerful in some aspects that OSS cannot compete with. This is very different from Elsevier vs OA. But on the other hand, I never publish anything that the reader would need Mathematica to verify, i.e. I never put my readers behind any paywall. – Hao Chen May 17 '19 at 19:51
-
well, I have 0 objection to private use of Mathematica (or other Mmas) as long as you can get a verifiable without it result. – Dima Pasechnik May 17 '19 at 19:55
-
4@DimaPasechnik : I don't see much relevance of the Elsevier/Springer/etc matter here. Also, in the event that Wolfram Inc stops Mathematica, there still will be a solution: to check the claims by using other software. However, I have rather seen cases when OSS gets abandoned. (As a practical matter, except for the developers, I think hardly anyone would ever check any part of the computer code itself, be it Mathematica or OSS). – Iosif Pinelis May 17 '19 at 22:11
-
@IosifPinelis if code is a part of a publication, it ought to run on a system that is available to everyone, otherwise it is basically a paywall situation. – Dima Pasechnik May 17 '19 at 23:13
-
1@IosifPinelis : If you read through the comments here https://mathoverflow.net/a/316177/3106 you will see that docker is at least a partial solution to the problem of the operating system being abandoned. – Timothy Chow May 17 '19 at 23:28
-
@IosifPinelis I don't think that your point (ii) "spending time to learn and use a variety of specialized and oftentimes less powerful OSS" is accurate. For example, it is possible to use within Sage a variety of specialized softwares. Also, can you substantiate the claim that OSS are oftentimes less powerful than Mathematica? Do you mean that for a given problem, all the OSS are less efficient, or that most of them are? – François Brunault May 18 '19 at 20:25
-
@TimothyChow : Thank you for your comment. This is good to know. – Iosif Pinelis May 19 '19 at 02:40
-
@FrançoisBrunault : My experience with OSS alternatives to Mathematica has been very limited, but not happy. I tried to use a number of such alternatives, but was more or less able to use only Reduce and QEPCAD B, with much difficulty, and their performance was much inferior to that of Mathematica; I did that because referees and editors of the paper (https://projecteuclid.org/euclid.ejp/1457706456) requested that. – Iosif Pinelis May 19 '19 at 03:16
-
Previous comment continued: Eventually I also found a long solution that can be verified by hand. As for your last question: No, I do not think are all the OSS are less efficient (than Mathematica) in what they can do. I think they tend to be more specialized and, based on my limited experience, seem to be less or much less efficient than Mathematica even in what they can do. Also, they seem to be hard to learn and use. I had much less difficulty learning and using Mathematica to more or less satisfy my needs. – Iosif Pinelis May 19 '19 at 03:17
-
@DimaPasechnik : "it ought to run on a system that is available to everyone, otherwise it is basically a paywall situation." Again, I think one should try to compare two options: (i) to pay with money (which is, in the final analysis, time) for a Mathematica license (if one does not already have it, through their institution or otherwise) vs. (ii) to pay with time to try to learn and use in most cases more specialized, not very user-friendly, and possibly/probably less capable software. I think it is a mistake to assume that option (ii) involves no costs. – Iosif Pinelis May 19 '19 at 04:56
-
@IosifPinelis I do number theory and my experience is opposite: OSS have far more possibilities than Mathematica e.g. elliptic curves, modular forms, $L$-functions, but there are plenty of other examples. I don't know about speed aspects however. The analogue of Mathematica in number theory is Magma, which is proprietary but the source code can be read and the documentation is very detailed and very convenient to browse (as opposed to my limited Mathematica experience), so at least we know what the software is doing. – François Brunault May 19 '19 at 06:11
-
@IosifPinelis The equivalence between time and money is true only for those people or institutions who can afford it. More fundamentally, we must consider how a given software helps spreading knowledge. – François Brunault May 19 '19 at 07:58
-
1@FrançoisBrunault - Magma has a lot of C code which is not open source. Also, Magma isn't as readily available as Mathematica (apparently due to being the weapon of choice of its largest customer, NSA...) – Dima Pasechnik May 19 '19 at 09:50
-
@IosifPinelis - I am not saying that (ii) involves no "costs" - but these costs are akin to "costs" of learning/improving a body of knowledge (as opposed to (i), which at most produces a circle of closeted know-how) – Dima Pasechnik May 19 '19 at 10:04
-
@FrançoisBrunault : I think we can agree that different people will make different choices, depending on their needs and preferences. – Iosif Pinelis May 19 '19 at 14:03
-
@DimaPasechnik : As a mere user of software with limited time and money resources, I have to choose what software systems are worth acquiring, learning, and using -- based on some kind of "Usefulness-to-CostsOf(Acquiring+Learning+Using)" ratio. In my limited experience, this ratio was much higher for Mathematica concerning symbolic and numerical calculations. For other people, this may be different. – Iosif Pinelis May 19 '19 at 14:13
-
@IosifPinelis Sure, except that some people have hardly any choice. – François Brunault May 19 '19 at 15:01
-
@IosifPinelis: I think what people are trying to get you to see is the tradeoff between your time/money/effort and the time/money/effort of the consumer of your work (e.g., a mathematician in an underprivileged country who wants to understand your paper). Even if it's more work for you to use open source software, there's an argument that as a courtesy to consumers who do not have the resources that you have, you should invest that extra work to make life easier for them. – Timothy Chow May 19 '19 at 18:17
-
@TimothyChow : I understand that. However, a few points here are worth considering, I think: (i) To use any kind of sorfware, a person will have to have good enough access to a computer (and also to a journal), which may cost much more than a Mathematica license. Posting papers on arXiv may alleviate the concern about the journal, and I do try to post my papers on arXiv. (ii) The readers of my paper will, in most cases, have to go through the same painful process of learning and using the software. – Iosif Pinelis May 19 '19 at 18:42
-
Previous comment continued: (iii) The calculations done by such commands as Mathematica's Maximize[] are entirely algorithmic and routine, and it would probably take at least hundreds of pages to transcribe them into human language. Who would ever want to go through all such pages? On the other hand, the usage of Maximize[] is very straighforward and not at all instructive. So, readers who desire to check the Maximize[] output, can always do that using any other software that they have and like. – Iosif Pinelis May 19 '19 at 18:51
-
@IosifPinelis assuming one needs a full-blown, non-crippled version of Mathematica to run your code, we talk about £2300 license fee per year (for non-academics). This money buys 5-6 quite OK computers nowadays... – Dima Pasechnik May 19 '19 at 20:51
-
@DimaPasechnik : (i) I have checked Mathematica prices for Viet Nam and Venezuela; see http://store.wolfram.com/view/app/mathematica/home-annual. They seem to be 335 USD for life or 170 USD annually (home edition, across the board, for all kinds of customers), hardly enough to buy 5-6 (or even one) decent computers. (ii) There is almost no such thing as code for Maximize[]; it's just a few straightforward lines (https://reference.wolfram.com/language/ref/Maximize.html), which can be (I hope) easily implemented with any relevant software one has and likes. – Iosif Pinelis May 19 '19 at 22:11
-
I have no idea how crippled the home edition is. I quoted a UK price for non-academics. – Dima Pasechnik May 20 '19 at 06:36
-
@IosifPinelis the link to Maximize does not include any code, it's just a manual entry on how to call the function. What it does is a black box; and indeed Maximizing a non-concave function with many local maxima on the domain is in principle messy thing involving precision considerations etc. – Dima Pasechnik May 20 '19 at 06:40
-
@DimaPasechnik : I don't know why you are saying that a home edition is crippled. I have used both workplace and home-use versions of Mathematica and seen no discernible difference. Similarly, between the Home and Professional/Enterprise versions of Windows. – Iosif Pinelis May 20 '19 at 12:27
-
@DimaPasechnik : In your latest comment, you wrote: "the link to Maximize does not include any code, it's just a manual entry on how to call the function." -- In a previous comment (now apparently deleted), you mentioned something like "your code [that is, my code] for Maximize". My response was that such code is so short and straightforward that one can hardly call it code, and I included a link to a manual entry for Maximize[] showing how simple that "code" is and that it cannot possibly be of any instructive value. – Iosif Pinelis May 20 '19 at 12:37
-
@IosifPinelis there are many crucial differences between Home and Pro Windows versions that you might not notice if you only use your machine as a glorified typewriter. E.g. one cannot use WSL. With that "hobby/home" Mma licence you cannot legally do academic work, namely "Work or research in which the product benefits your work" is forbidden. – Dima Pasechnik May 20 '19 at 12:37
-
@DimaPasechnik : Those "crucial" differences have not been of any value to me, and I don't even know (or want to know) what WSL is. – Iosif Pinelis May 20 '19 at 12:41
-
1@IosifPinelis Once again, there is no publicly available Mma code for Maximize[], what are you talking about? And surely it runs into hundreds of lines, calling more blackboxes... – Dima Pasechnik May 20 '19 at 12:41
-
@IosifPinelis - no, thanks, before following your advice on Mma versions one must talk to a lawyer :-) – Dima Pasechnik May 20 '19 at 12:43
-
@IosifPinelis if you don't know a difference between Windows Home and Pro, and willingly admit to being a total non-expert on this matter, why do you even make such comments? "I don't see a difference cause I am actually blind" :-P – Dima Pasechnik May 20 '19 at 12:46
-
@DimaPasechnik : You are free to use all kinds of epithets like "crippled", "actually blind", "total non-expert", "willingly admit", etc. The matter is simply that those "crucial" differences between versions of Windows (say) make no difference for my needs in mathematics and elsewhere. If they matter to you, enjoy! – Iosif Pinelis May 20 '19 at 12:55
-
@DimaPasechnik : The Mathematica home-use license was given to me by my (academic) institution for work on my personal computers, and I don't think I need a lawyer to use it for research. – Iosif Pinelis May 20 '19 at 12:59
-
@DimaPasechnik : You originally mentioned "your code" (that is, my code), and now you are talking about Mathematica's internal code, which is certainly not mine. Also, as I said earlier, I don't know if anyone would like to read all that internal code, even if transcribed into (quite possibly at least hundreds of pages of) human language. – Iosif Pinelis May 20 '19 at 13:05
-
It looks like this discussion boils down to this choice: Should I (i) try to use my main (I think) comparative advantage to produce mathematics hopefully of interest to other people or should I (ii) instead spend time trying to learn and use software that other people can use to check the output of commands like Maximize[]? – Iosif Pinelis May 20 '19 at 13:07
-
@IosifPinelis thus you have a Mma license paid by your employer, no problem for you. I mentioned that an unaffiliated individual may need a £2K+ p.a. license to be able to do academic work, and you disagreed, saying that a "hobby" license is much less. I pointed out that the latter won't do. Do you agree that a "hobby" license you pointed at specifically excludes the kind of use we are talking about? Or you will continue to pretend you have no clue? – Dima Pasechnik May 20 '19 at 13:52
-
@DimaPasechnik : I did not notice those licensing restrictions. Now I see that the apparently most relevant restricted use of that home edition of Mathematica is this: "Work or research in which the product benefits your work" (emphasis mine). So, if a person works, say, for a company that is not interested in mathematics or Mathematica, then I don't see why he/she cannot use the home edition of Mathematica to do mathematics. However, all this licensing discussion is almost entirely beside the point, the point being about the choice delineated in my previous "boils down to" comment. – Iosif Pinelis May 20 '19 at 14:09
-
@IosifPinelis work is also stuff one publishes; the restriction basically says that one cannot legally publish work done with help of so licensed Mma. – Dima Pasechnik May 20 '19 at 16:36
-
@DimaPasechnik : I think Wolfram Inc's concern is just with commericial use of Mathematica, and the publication of a paper is not such use. Anyway, once again, these licensing matters are almost entirely beside the point, because the output of the Mathematica commands like Maximize[] can be verified by any other available relevant software, and so, Mathematica is not really needed for such verification. – Iosif Pinelis May 20 '19 at 17:03
-
Well, in some countries you get bonuses for publishing (good) papers. I was getting these while working in Singapore :-) – Dima Pasechnik May 20 '19 at 17:20
-
and almost everywhere publishing is a pre-req. to get grants. So you see, it's directly related to one's income, one cannot say it's non-commercial. – Dima Pasechnik May 20 '19 at 17:22
13
The answer to this will differ from one field of math to another. In my opinion (I work primarily in combinatorics and algebraic geometry) a paper should contain enough data that the reader could verify its claims without relying on a particular software package, or should link to a repository of such data.
The main paper where this came up for me is Version 1 of my paper with Robert Kleinberg and Will Sawin. In Section 7 of this paper, we need to verify that a certain linear program, whose constant terms are polynomials in an algebraic number $\rho$, is feasable. What we do in the paper is to find a solution point whose exact coordinates are polynomials in $\rho$ and provide the reader with both those exact polynomials and with their numeric values to enough accuracy to confirm that it is a solution. The skeptical reader may use the exact formulas to verify that the equality conditions hold and may then compute the numerical values on their own to enough accuracy to check that the inequalities hold. We didn't provide the (quite hackish) Mathematica code which computed this witness data.
We printed the first few solutions in the paper and then uploaded a text file to the arXiv (see "Download Source" here) with more.
This material is missing from the final version of the paper, because a more general result was proved by Pebody and Norin. Otherwise we would have done something similar in the final paper; I don't know whether the journal would have wanted all the data in print, would have wanted to host the extra data themself, or just to keep it on the arXiv, but I would want to do one of these.
David E Speyer
- 150,821
13
25 years ago I published a paper where there were computer-based enumerations of certain combinatorial objects. Unfortunately I did not publish the code (in GAP, an OSS system) and it vanished after a place where I used to be a postdoc some years later shut down an ftp server with my files). Fortunately some results there were then verified by hand. Just a month ago I had to sit down and re-do these computations for the remaining cases, as someone asked for it. Fortunately it did not took too much time, and I even learned something new (and publishable).
Anyhow, publish your code, and make it sufficiently clean to be understood years later...
Dima Pasechnik
- 13,634
-
I think we need to distinguish "computer aided proof" (your case) and "computer calculation" (OP's case). I support the former and regard it as a solid math proof, but I strongly oppose the latter as proof in any sense. – Hao Chen May 17 '19 at 19:58
-
well, it is a computer calculation, what I did, using a relatively straightforward recursive search sped up by symmetry considerations (computing orbits). – Dima Pasechnik May 17 '19 at 20:01
-
1I also have papers where result depends on running numerical optimization - the answer however verifiable by cheap checking of optimality conditions (which basically only needs matrix arithmetic) – Dima Pasechnik May 17 '19 at 20:04
-
3I use "computer aided proof" for the case where a human set up a plan and computers do the work. This can then be verified by checking the plan and the implementation. I use "computer calculation" for the case where a human use a software, whose algorithm he does not read / not completely understand, just for an output. – Hao Chen May 17 '19 at 20:06
-
-
@HaoChen : I sympathize with your point, but would you extend the ideal of "reading and completely understanding" to the compiler, operating system, and hardware of the computer being used? I suspect not. Even "understanding an algorithm" can be a gray area. Suppose I use Mathematica to multiply out a bunch of polynomials with integer coefficients. I understand the naive algorithm for doing this, but what if Mathematica is using some fancy fast multiplication algorithm? Do I have to completely understand the fancy algorithm before I can use the result in a proof? – Timothy Chow May 17 '19 at 21:03
-
Mathematica is basically a black box, you can't possibly understand what it does, unless you have a privilege to see its source code (or do prohibitively hard reverse engineering). So it's a moot point, as opposed to Hao Chen's point, which is that understanding may be always made deeper in the setting of open source. – Dima Pasechnik May 17 '19 at 21:29
-
@TimothyChow Actually, I think you do ... In the scenario of a "computer added proof", where the plan involves billions of multiplications (so that we have to use computer), the best case would be that each multiplication is verifiable by author, the second best case would be that the implementation and algorithm of multiplication is accessible and verifiable by author, although the author does not need to really do these. This is my threshold. So the author should at least write their own code, or use and cite openly accessible code. – Hao Chen May 18 '19 at 04:03
-
1@TimothyChow my requirements do not extend to compiler, OS or hardware, but let me explain. Fot a proof to be acceptable, there should be a purely logic possibility that any human being, given infinitely much time and enough knowledge, could work out every step of the algorithm. This is possible once the algorithm and the necessary knowledge is accessible. OS etc. doesn't matter here. But if aliens come with a machine that is said to do multiplication with a more advanced method that humans do not have access/understand, then mathematicians should not use this machine for any proof. – Hao Chen May 18 '19 at 04:22
-
@HaoChen : The compiler, OS, and hardware are, in effect, a machine that "does multiplication with an advanced method" that you don't understand (unless you make the effort to understand them). You're telling it, "multiply these two integers together" and then it goes off and does some incomprehensibly complicated manipulations and comes back with what it claims is the product. – Timothy Chow May 18 '19 at 15:53
-
@TimothyChow The "unless you make the effort" part is way weaker than my "purely logic possibility with infinite time". As in "science is fausifiable", physics is science because there is a hypothetical scenario that makes it false, which does not mean it's actually false. Moreover, I'm making the statement with an ideal machine (human with infinite time and always enough knowledge). All algorithm should run with this machine, so compiler, os, hardware are not in the equation. – Hao Chen May 18 '19 at 17:24
-
@HaoChen : I still do not understand your idealization. If I am a human with infinite time then I can examine the Mathematica binary file, figure out what it's doing, and verify everything that it is computing. I don't actually need to do this verification; it just has to be possible in principle, right? – Timothy Chow May 19 '19 at 18:13
-
@TimothyChow So you involve a reverse engineering activity here. Thought about this. I would like to forbid this by requiring my machine (this human) not to "cheat" by learning from other machines. He has access only to existing books and papers, as well as existing algorithms described in human language, but not existing implementations of the algorithms. By the way, I don't think my idea is perfect, not even well-defined. But I enjoy any discussion / opposition that could make it better, ultimately towards a well-defined standard for computer involvement in maths. – Hao Chen May 19 '19 at 18:31
-
@TimothyChow - reverse engineering like you describe might be illegal. in some jurisdictions at least. – Dima Pasechnik May 19 '19 at 20:58
8
Other answers and comments have made passing suggestions that you could perhaps consider making your code available. I think that that is far too weak: it is imperative that you should make your code available, and I would not accept your paper as an editor or referee if you did not do that. You should upload the code to the arxiv as a supplementary file, because that is a fairly reliable way to make sure that the original version remains available. You could also put it on github or a personal web page.
Neil Strickland
- 55,003
-
2Personal web pages change frequently. Something more stable such as the arXiv seems much more desirable. – Andrés E. Caicedo May 18 '19 at 18:08
-
1@AndrésE.Caicedo Are we reading the same thing? It sounds like Neil Strickland is suggesting putting the code on the arXiv, and "also" possibly ("could") on a personal webpage/github. Presumably for ease of access – getting the source on arXiv is a bit annoying. – Najib Idrissi May 20 '19 at 12:32
-
To clarify, I am suggesting that code could go on a personal web page as well as the arxiv, it would certainly not be satisfactory to have it only on a personal web page. – Neil Strickland May 20 '19 at 14:54
7
How do I explain that? "Using wolfram we find the maximum of $f$ to be $1.0328...$ therefore...". It looks very sloppy.
Well, it looks sloppy because it kind of is sloppy by your description. Maybe the right framework for thinking about your question is to imagine that the maximization you are claiming is a component in a claimed proof of, say, the Riemann hypothesis. Would you pay serious attention to a paper containing such a claimed proof that purported to rely on an opaque software-based maximization without providing any code, let alone any guarantees that the code and the underlying software platform it runs on do what they say they do? I wouldn’t, nor do I expect any serious person to allow such a paper to be published.
Now, I‘m guessing the problem your paper claims to solve isn’t as important as the Riemann hypothesis. The question of what authors of math papers can get away with and still get their paper published is distinct from the question of what authors should do as an ideal to aspire to that would make their paper’s claims truly convincing and watertight. People do get away with all kinds of minor (or even major) sloppiness, of the kinds you mention and of other kinds having nothing to do with the use of computers, all the time. I know I have. Nonetheless, since you seem to be asking about what is the “right” way to explain your result, the answer (assuming you are publishing what is meant to be a rigorous theorem with a proof that’s up to the standards of a good pure math journal) is: provide as many details about your claimed maximization as are needed to convince almost everyone in the research community of the validity of your claims.
For myself, seeing your Mathematica code would be an absolute minimum to satisfy this condition. Depending on the precise nature of your calculations, I may also want to see that they satisfy some combination of the following conditions:
- They are based on algorithms that have been around for a long time and everyone is sure are correct.
- They are based on algorithms that have themselves been published in peer-reviewed journals.
- They can be replicated with relative ease in software packages other than Mathematica.
Hope this helps.
Dan Romik
- 2,480
4
What you did is a numerical investigation, so make a numerical claim, nothing more. It should not be claimed as a proposition, or be used as a step in a proof/argument. In particular, I don't like the "therefore" in your example. No matter how many times a numerical method has been tested, it is never a proof and should not be considered solid in the mathematical sense (except for, maybe, those symbolic or analytic result that can be verified by hand).
In many cases of mathematics, the concrete number does not matter. You might be able to prove the existence of a maximum within a range (could be very hard). This is usually better (trustworthy and verifiable) than an actual number. In fact, we often need to prove the existence of a solution whose value has been known since long (recently been there ...)
Hao Chen
- 2,541
3
Did you try softwares doing rigorous numerical computation? For example arb is a C library freely available in Sage. Such software gives guaranteed enclosures $f(I) \subset J$ where $I,J$ are intervals. So an obvious (but not necessarily optimal) algorithm for finding the maximum of your function $f:I \to \mathbb{R}$ is to subdivide your interval $I$ and apply $f$ to each subinterval, and keep subdividing until you get the maximal value to the precision you want. Personally I would judge this as rigorous and acceptable in a proof. This should work as long as the functions you need are present in arb, at worst the algorithm will be too slow.
François Brunault
- 20,484
-
-
@NajibIdrissi The OP did a computation using Mathematica, which is not rigorous in the usual mathematical sense (I'm not saying this function may have bugs, just that he's trying to maximize a function, so Mathematica uses floating-point numbers, which may cause some problems, see e.g. the first integral here: http://fredrikj.net/blog/2017/11/new-rigorous-numerical-integration-in-arb/ ). Thus a natural thing to do is to use a software implementing floating point computations with rigorous bounds (this is usually called interval arithmetic). – François Brunault May 20 '19 at 12:40
-
1Mathematica doc: "If Maximize is given an expression containing approximate numbers, it automatically calls NMaximize". – François Brunault May 20 '19 at 12:45
-
Sure, but that's not the question that was asked. The question was how to present results obtained (no matter the method) in a paper. I'm not sure why everyone jumped straight on the NMaximize bandwagon. – Najib Idrissi May 20 '19 at 13:40
-
1@NajibIdrissi If there is no way to present it in a paper because it is not rigorous enough (no matter the presentation), then I think this is a solution. At least this is my interpretation of the question that it was about presenting but also making things rigorous. – François Brunault May 20 '19 at 13:42
-
@NajibIdrissi Maybe the OP should clarify the question then. – François Brunault May 20 '19 at 13:47
1.0328from Maximize[]? If the latter, I suspect somewhere in the function you have an approximation. The docs for Maximize[] say "If Maximize is given an expression containing approximate numbers, it automatically calls NMaximize.", and I wouldn't trust NMaximize in a rigorous setting. – sdcvvc May 19 '19 at 14:25