Computer algebra errors

Question

In the course of doing mathematics, I make extensive use of computer-based calculations. There's one CAS that I use mostly, even though I occasionally come across out-and-out wrong answers.

After googling around a bit, I am unable to find a list of such bugs. Having such a list would help us remain skeptical and help our students become skeptical. So here's the question:

What are some mathematical bugs in computer algebra systems?

Please include a specific version of the software that has the bug. Please note that I'm not asking for bad design decisions, and I'm not asking for a discussion of the relative merits of different CAS's.

Answer 1

I don't know any interesting bugs in symbolic algebra packages but I know a true, enlightening and entertaining story about something that looked like a bug but wasn't.$\def\sinc{\operatorname{sinc}}$

Define $\sinc x = (\sin x)/x$.

Someone found the following result in an algebra package: $\int_0^\infty dx \sinc x = \pi/2$

They then found the following results:

$\int_0^\infty dx \sinc x \; \sinc (x/3)= \pi/2$

$\int_0^\infty dx \sinc x \; \sinc (x/3) \; \sinc (x/5)= \pi/2$

and so on up to

$\int_0^\infty dx \sinc x \; \sinc (x/3) \; \sinc (x/5) \; \cdots \; \sinc (x/13)= \pi/2$

So of course when they got:

$\int_0^\infty dx \sinc x \; \sinc (x/3) \sinc (x/5) \; \cdots \; \sinc (x/15)$$= \frac{467807924713440738696537864469}{935615849440640907310521750000}\pi$

they knew they had to report the bug. The poor vendor struggled for a long time trying to fix it but eventually came to the stunning realisation that this result is correct.

These are now known as Borwein Integrals.

A video on this topic, titled "Researchers thought this was a bug," is on the 3Blue1Brown YouTube channel here.

Answer 2

In 1999, when I first bought an HP49G, whose major selling point was a CAS, I thought I'd try summing the harmonic series $\sum_{n=1}^\infty \frac{1}{n}$. I was a bit surprised to see the result 1151.8697216.

Now, I wouldn't have been too surprised if it had handled an infinite sum by just adding up "a lot" of terms until the partial sums seemed to be converging, but knowing how slowly the harmonic series grows, it wasn't plausible that it could have actually summed enough terms to get to 1151.8697216.

It turned out that it knew how to numerically compute the discrete antiderivative $\Psi(m) := \sum_{n=1}^m \frac{1}{n} \approx \ln m + \gamma$, and in the particular mode that it happened to be in, it would replace $\infty$ with the largest floating-point number it could represent, which was just under $10^{500}$. Indeed, $\Psi(10^{500}) \approx 500\ln 10 + \gamma \approx 1151.8697216$.

The story has a happy ending: after changing some flags, it returned $+\infty$.

Answer 3

Because the most popular systems are all commercial, they tend to guard their bug database rather closely -- making them public would seriously cut their sales. For example, for the open source project Sage (which is quite young), you can get a list of all the known bugs from this page. 1582 known issues on Feb.16th 2010 (which includes feature requests, problems with documentation, etc).

That is an order of magnitude less than the commercial systems. And it's not because it is better, it is because it is younger and smaller. It might be better, but until SAGE does a lot of analysis (about 40% of CAS bugs are there) and a fancy user interface (another 40%), it is too hard to compare.

I once ran a graduate course whose core topic was studying the fundamental disconnect between the algebraic nature of CAS and the analytic nature of the what it is mostly used for. There are issues of logic -- CASes work more or less in an intensional logic, while most of analysis is stated in a purely extensional fashion. There is no well-defined 'denotational semantics' for expressions-as-functions, which strongly contributes to the deeper bugs in CASes.

Answer 4

A quite serious error in Mathematica 7 in my opinion is that it thinks $ \sqrt{x^2} =x$, not $|x|$, leading for example to 2 solutions to the following differential equation: $$ y'(x) = 2 y(x) (x \sqrt{y(x)} - 1) \quad y(0) =1$$ Mathematica happily gives the following solutions: $$ y(x) \rightarrow \frac{1}{(1-2 e^x +x)^2}, \quad y(x) \rightarrow \frac{1}{(1+x)^2} $$ Of course, it is a theorem that there is a unique solution to a differential equation of this type, but that doesn't mean my students hand in the wrong answer in droves...

Mathematica code: FullSimplify[DSolve[{y'[x] == 2 y[x] (x Sqrt[y[x]] - 1), y[0] == 1}, y[x], x]]

Answer 5

$2^{4^{4^4}} < 4^{4^{4^4}}$

WA: False

Update: it seems to be fixed now

Answer 6

A friend of mine told me about his experience with Maple (version 5 or 6, I think) when dealing with matrices over $\mathbb{Q}(\sqrt{2},\sqrt{3})$. When he computed the rank and the determinant for one particular $3\times3$-matrix, he was told that the rank was 3, and the determinant was equal to zero. The answer to this paradox is, that by default, for determinants the symbolic computation methods were used for radicals, and for ranks, the floating point representations of matrix elements!

This can be thought of as either a bug or his naiveness (for not checking out how to represent elements of number fields so that floating point representations never appear), but in any case is a serious argument for treating the computer algebra software with care...

Answer 7

Sometimes a CAS cannot get the right branch of inverse trig functions when calculating integrals symbolically. See for instance: https://web.archive.org/web/20160817112947/https://pantherfile.uwm.edu/sorbello/www/classes/mathematica_badintegral.pdf

Apparently this is an unsolved problem in computer algebra.

Answer 8

Wolfram alpha is saying that the series of $\sum_k\sin(2 k \arctan(k^2))$ does not converge:

https://www.wolframalpha.com/input/?i=sum+sin%282+k+atan%28k%5E2%29%29

instead it converges! Seems that mathematica is only dealing with limits of functions not with limit of sequences.

Another simpler example is $\sum_k \sin(2k \pi + 1/k^2)$:

https://www.wolframalpha.com/input/?i=sum+sin%282k+pi+%2B+1%2Fk%5E2%29

E.

Answer 9

This story heard from Enrico Bombieri. I do not know if it qualifies, since it is not a CAS bug, and in addition it is second-hand. However it might be quite effective in casting doubt in the mind of your students, if that's your purpose :)

E.B. was doing some Riemann zeta zero crunching on his PC some years ago, the software he wrote seemed ok, and the next step was to run it on a mainframe to get some serious data. He was given the privilege to try it on the first Cray supercomputer. Most of the time results were nice, but every now and then he got really weird results. He and his coworkers spent some awful weeks trying to catch the bug. In the end, they cornered the problem: when the Cray divided 1 by 12 the result was a negative number...

EDIT: I double checked, it was not a Cray supercomputer but a computer based on an early iteration of the Pentium chip (I guess an IBM one), and the Pentium bug was also encountered by others of course. Sorry for the inaccuracy.

Answer 10

Here is an example in Wolfram Alpha. A student had been given the assignment of finding the limit as $n$ tends to infinity of $\frac{1}{1+\frac{(-1)^n}{log(n)}}$. He had correctly arrived at the answer 1. Now he used WA to check if he was correct. WA returned 0 (the command lim n-> inf 1/(1-(-1)^n/log(n)) ). On examining the steps, it turned out that WA had manipulated a bit and used L'Hopital on the expression $\frac{log(n)}{(-1)^n+log(n)}$.

Note that if one instead asks for the limit of $\frac{1}{1-\frac{(-1)^n}{log(n)}}$ WA correctly returns 1, using the same method one usually would.

Answer 11

From the sage-support mailing list.

Sage 5.10 claims $$\forall a,b \in \mathbb{R}, \; \sqrt{(a+b)^2}=\sqrt{a^2}+\sqrt{b^2} $$

though it contradicts it numerically for $a=1,b= -1$.

Session:

sage: var('a,b');assume(a,'real');assume(b,'real');ex=sqrt( (a+b)^2 ) - (sqrt(a^2)+sqrt(b^2));ex
(a, b)
sqrt((a + b)^2) - sqrt(a^2) - sqrt(b^2)
sage: ex.full_simplify()
0
sage: ex.subs(a=1,b=-1)
-2

Answer 12

My advice is never to trust a single CAS. I only wrote one computer aided paper: I did the programming on Mathematica / Linux, and my collaborator did it on Magma / Solaris. We also made a point of not communicating while writing the programs.

Answer 13

This error affects all versions of Mathematica from 6 to 8. The result of a function depends on what letter is chosen for argument when calling it. In the simplest case it can be illustrated as follows:

in:

$A[\text{x_}]\text{:=}\sum _{k=0}^{x-1} x $

$A[k]$

$A[z]$

out:

$1/2 (-1 + k) k$

$z^2$

The correct answer is evidently, the later. This behavior affects not only sums but also integrals, so one have to check so that the letter user for the argument not to coincide with the index variable used for definition. In the case of recursion this becomes very difficult. The following example shows that moving a factor not dependent on the index variable out of the sum sign changes the result:

in:

A1[0,x_]:=1
A2[0,x_]:=1

A1[n_,x_]:=Sum[A1[-1 - j + n, x]*Sum[A1[j, k], {k, 0, -1 + x}], {j, 0, -1 + n}]
A2[n_,x_]:=Sum[Sum[A2[j, k]*A2[-1 - j + n, x], {k, 0, -1 + x}], {j, 0, -1 + n}]

A1[1,x]/.x->2
A1[2,x]/.x->2
A1[3,x]/.x->2

A2[1,x]/.x->2
A2[2,x]/.x->2
A2[3,x]/.x->2

A2[1,2]
A2[2,2]
A2[3,2]

out:

2
5
13

2
5
12

2
5
13

Answer 14

In Mathematica 7, the command

Table[DirichletCharacter[4, 2, n], {n, 0, 8}]

should return a list of values of the character with modulus 4 and index 2, evaluated at 0, 1, 2, ..., 8. Instead, it returns the decidedly non-multiplicative:

{0,1,0,-1,0,-1,0,-1,0}

Answer 15

Analytical Number Theoretic Functions in Mathematica are (sometimes) unreliable.

• http://fredrik-j.blogspot.com/2010/01/zeta-evaluation-with-riemann-siegel.html
• http://fredrik-j.blogspot.com/2009/08/torture-testing-special-functions.html
• http://fredrik-j.blogspot.com/2009/07/another-mathematica-bug.html
• http://fredrik-j.blogspot.com/2009/06/meijer-g-more-hypergeometric-functions.html

https://code.google.com/archive/p/mpmath/ is part of Sage --- https://www.sagemath.org/ --- hence you can double check Mathematica values there.

(sorry, I'm not allowed to post hyperlinks...)

Answer 16

A bug, this time from MATLAB. While trying to obtain:

$$\int_0^a x^2\sqrt{-x^2+ax}\,\mathrm{d}x=5a^4\pi/128$$

through:

syms x a
assume(a>=0)
int(x^2*sqrt(-x^2+a*x),x)

MATLAB get unfocused between the imaginary algebra and gets the negative value (!): $$-5a^4\pi/128$$

Original question in here. Tested in MATLAB R2014b at 2017-05-16. Edit: Present in R2018a, it was already corrected in R2018b.

Answer 17

This was mentioned to me by the user Umesh Shankar on Mathematics SE.

WolframAlpha incorrectly solves the following congruence equation: $4 + 6 \times 10^n \equiv 0 \pmod{7}$. It says that the answer is $n \equiv 4 \pmod{7}$. The correct answer is clearly $n \equiv 4 \pmod{6}$.

Answer 18

I found a mistake made by WolframAlpha, which I also posted on a related thread on Mathematics Educators SE.

Ask WolframAlpha to Solve 1/floor(x) < 2/x for x > 1, and it says that the solution over the reals is $x = 99/5 \approx 19.8000$". Rearrange the terms and ask it to Solve x < 2 floor(x) for x > 1 and it gives a different solution "$x = 759/10 \approx 75.9000$". Increase the lower bound on $x$ from $x > 1$ to $x > 76$, say, to get WolframAlpha to commit to different specific answers.

Of course, the inequality is actually true for all $x > 1$.

Answer 19

This might get fixed in the future, but at the time of this writing, Wolfram Alpha gets apparently sometimes confused by logarithms of complex numbers:

Wolfram Alpha -- $\log(1+ \frac{1}{2}i) - \log(1 - \frac{1}{2} i)$

For reference, should the problem get fixed: it claims that $2i = 2i\cot^{-1}(2) \approx 0.9272$.

Curiously, the numerical approximation is correct, but the symbolic form seems to be wrong.

Answer 20

There are too many to be listed on the margins of MO.

Look at the archives of the newsgroups comp.soft-sys.math.maple, comp.soft-sys.matlab, sci.math.symbolic, comp.soft-sys.math.mathematica. There you can find hundreds of bugs reported.

There is a notorious CAS bug hunter who once maintained a bug list for Maple and shows more than 5000 disturbing observations. (Press the Go! button.) Or go to MapleSoft and search Maple Primes.

Please don't shoot the messenger.

Answer 21

If you are performing numerical computations, then a more likely source of error is in roundoff or over/underflow. In these cases, I wouldn't say that the CAS is necessarily in the wrong, just that you need to know the properties of the underlying algorithm and either recast your input or reimplement it in a more numerically robust way. In such cases, decent introductions to numerical analysis should give you a feel for the types of issues you need to worry about.

Of course, on the matter of symbolics, then there are no excuses for errors.

Answer 22

David Bailey and Jonathan Borwein said in a talk yesterday that the most recent editions of both Maple and Mathematica give the nonsensical result $$\int_0^1\int_0^1|e^{2\pi ix}+e^{2\pi iy}|\,dx\,dy=0$$ I later verified this for Maple 17, entering int(int(abs(exp(2*Pi*I*x)+exp(2*Pi*I*y)),x=0..1),y=0..1).

Answer 23

http://oeis.org/A110375

A110375   Numbers n such that Maple 9.5, Maple 10, Maple 11 and Maple 12 give the wrong answers for the number of partitions of n.

Answer 24

Here are some results where different CAS give conflicting results:

1. $\int_{y}^{\infty} \frac{e^{-x}}{x}{d x}$ for $y \in \mathbb{R}$ and $y>0$. Wolfram Alpha gives $$\log{y}+\Gamma(0,y)$$ and sage 4.7.1 gives $$ -{\rm Ei}\left(-y\right) $$

2. For all integers $n$, Coq proves $$n \mod 0 \equiv 0$$ and Isabelle (Wayback Machine) proves $$n \mod 0 \equiv n$$ (The proofs are just stated in theorems, I can give the exact theorems if needed). Interesting, both proofs doesn't seem to lead to inconsistency though AFAICT they depict the usual mod.

[Added] I am a fan of sage, but this bug annoyed me.

sage 4.7.2 incorrectly reports the girth of a 7 vertex graph:

H=Graph([(0, 1), (0, 3), (0, 4), (0, 5), (1, 2), (1, 3), (1, 4), (1, 6), (2, 5), (3, 4), (5, 6)]) 
H.girth()
4
H.is_triangle_free()
False

sage 4.3 and 4.6.2 return correct value.

sage session in the notebook and a plot of the graph

Answer 25

(I haven't sufficient points to post a comment to Leonid Kovalev's reply.)

The problem in the numerical integration example is that numerical integration in Maple is done using Int, not int. The correct command should be

evalf(Int(sin(x)^44,x=0..sqrt(44)));

which should produce consistent results (and much more quickly).

Answer 26

Here's one I came across just now, in Maple 12. The code

with(combinat):
F := fibonacci:
phi := (1+sqrt(5))/2:
G := k -> F(k+1)/phi^k;
limit(G(n), n=infinity);

returns 0. But from the usual explicit formula for the Fibonacci numbers, which gives $F(n) \sim \phi^n/\sqrt{5}$, the output should be $\phi/\sqrt{5}$, or $(5+\sqrt{5})/10$. Replacing the built-in Fibonacci function with one that gives the explicit formula, and running the code

F := n -> 1/sqrt(5)*(((1+sqrt(5))/2)^n-((1-sqrt(5))/2)^n);
phi := (1+sqrt(5))/2:
G := k -> F(k+1)/phi^k;
limit(G(n), n=infinity);

gives the correct answer. I've encountered things like this fairly frequently when using the built-in routine for Fibonacci numbers; presumably this routine doesn't "know" the asymptotics.

Answer 27

In the paper The Misfortunes of a Trio of Mathematicians Using Computer Algebra Systems. Can We Trust in Them?, the authors report a bug in Mathematica (which is still present in the version 10) that happens when computing determinants of matrices with large integers as entries.

The strangest thing of this bug is that for some matrices, the determinant function can give different values. The Mathematica notebook which reproduces the bug is available here.

Answer 28

Just found this in Mathematica 7.0 for Mac OS X x86 (64-bit) (November 11, 2008):

x=Exp[Pi Sqrt[163] ];
N[x-Round[x] ]
N[x-Floor[x] ]
N[x-Ceiling[x] ]
N[x - Round[x], 2]
N[x - Floor[x], 2]
N[x - Ceiling[x], 2]

The functions Round, Floor, and Ceiling are the obvious functions, while "N" converts the infinite-precision expression to a floating point number (the last three lines are aimed at 2-digit precision, while the first three should be 16-digit).

The first three calculations turn up as "-480." The last three give more correct values of -$7.5*10^{-13}, 1.0, -7.5*10^{-13}$.

Answer 29

The PARI/GP Thue equations solver gives wrong results when they are conditional on GRH.

Affected are at least versions 2.5.1 (latest) and 2.4.3.

? p=x^3 - 18*x^2 + 81*x + 1;a=3^3
%1 = 27
? t=thue(thueinit(p,0),a);[#t,t] \\ conditional on GRH
%2 = [3, [[0, 3], [3, 0], [19, 2]]]
? t=thue(thueinit(p,1),a);[#t,t] \\ uncoditional
%3 = [4, [[0, 3], [3, 0], [27, 3], [19, 2]]]

Found on the pari-dev mailing list http://permalink.gmane.org/gmane.comp.mathematics.pari.devel/3629.

Answer 30

We found some interesting bugs in Mathematica's integration software on this thread.

To wit, set

integral[m_,n_] = Integrate[Log[2+Cos[2Pi x]+Cos[2Pi y]] Cos[2Pi m x] Cos[2Pi n y],
                      {x, 0, 1}, {y, 0, 1}];

Then integral[1,1] should be $1/2-2/\pi$, but Mathematica 8.0.1 returns $1/2+2/\pi$. Values for other $m$ and $n$ are also wrong (see the question linked above), as can be quickly verified by replacing the "Integrate" command with "NIntegrate".

Curiously, if one changes the limits of integration to {x,-1/2,1/2} and {y,-1/2,1/2}, then the correct answers appear.

Answer 31

Over the summer I came across an elementary bug in Magma when working with congruence subgroups of SL_2(Z). The isEquivalent function, which is supposed to tell whether two points are identified by a congruence subgroup, would miss a lot of identifications. For example:

G := CongruenceSubgroup(2); % \Gamma(2)

H := UpperHalfPlaneWithCusps();

(G! [-11,4,8,-3]) in G; % Cast this matrix into \Gamma(2)

true % It's really in \Gamma(2)!

(G! [-11,4,8,-3]) * (H! 3/8); % Have the matrix act on the point 3/8

oo % Magma correctly computes that it gets sent to infinity

IsEquivalent(G, H! 3/8, H! Infinity()); % Are 3/8 and infinity equivalent under the action of \Gamma(2), and specifically, can you given me a matrix representing an element of \Gamma(2) sending the former to the latter?

false [1 0] [0 1] % Doh!

It's a pretty simple computation, and it was pretty clear what loop it was leaving out. We may have been running an old version of Magma, but anyway we reported the error to them, and they fixed it quickly, but I've never trusted computer algebra systems since!

Answer 32

According to Wolfram Alpha

$$ (\log{(5+i)}+\log{(5-i)})^4= 10\,000$$

When one clicks on "10 000" WA spells it as integer.

Answer 33

As was noted for Sage, for any open source CAS you can just look up the issue tracker. For example, here's the list if all the issues in SymPy tracker with the WrongResult label: http://code.google.com/p/sympy/issues/list?q=label:WrongResult. Most of them are pretty rare. You're much more likely to hit a bug that just gives an error when it shouldn't, or that gives an unexpected, but not technically wrong (mathematically), result.

My advice is to double check your answer in some other way. The chances of the same bug manifesting itself in two different ways are almost zero. For example, you can check a result numerically, which will use a completely different algorithm from the symbolic version. Many CASs even have built in functions that do this for you.

Answer 34

Mathematica 7.0.1 says that Sum[1/(k*Length[Divisors[k]]), {k, 1, n}] is the harmonic number $H_n$, which is clearly wrong. The correct answer is at An elementary number theoretic infinite series

---

Edit:

This is less a bug and more a misunderstanding of how to use Mathematica. The culprit is that Length[Divisors[k]] for k without a value evaluates to 1 (which is consistent with how Mathematica structurally treats expressions). The correct way to express the sum is

Sum[1/(k DivisorSigma[0, k]), {k, 1, n}]

which, as expected, now remains unevaluated.

Answer 35

Wolfram Mathematica 7 routinely confuses sums with integrals.

Example 1:

DSolve[(-Log[Log[a]] f'[x] + f''[x])/(Log[a] f'[x]) == D[Sum[f[x], x], x], f[x], x]

g[x_] := f[x] /. s
g[x]

Checking the result by inserting it into the equation shows the result is incorrect:

(-Log[Log[a]] g'[x] + g''[x])/(Log[a] g'[x]) - D[Sum[g[x], x], x]

Example 2:

s=NDSolve[{0.9159460564995328*Derivative[1][f][x] == f[x]*Product[f[x], x], f[0] == 1}, f, {x, -1.9, 15}]

Plot[Evaluate[f[x] /. s], {x, -0.4, 1.5}, AspectRatio -> Automatic, AxesOrigin -> {0, 0}]

In Mathematica 8.0 this has been fixed (i.e. it will report inability to solve the equations.

Answer 36

Not a bug but a difficulty for users:

I do often not really understand how assignements work for CAS:

Given a variable $a$ with value, say, $\pi$, set $b:=a$ and set now $a$ to, say, $e$. What is the value of $b$?

As I understand the answer depends sometimes on the context (working with symbolic variables, vectors, floating numbers etc.) and the exact behaviour is sometimes difficult to guess for me.

Answer 37

Sagemath and pari/gp disagree about the degree of the zero polynomial over arbitrary rings. Since pari/gp is proper subset of sagemath, this gives us inconsistency in sagemath.

Session:

sage: K.<x>=QQ[];K(0).degree()
-1
sage: gp.poldegree(K(0))
-oo

Answer 38

Here is a very simple common bug: ask your cas to solve in x the equation a x-´b = 0. Probably, the answer would bex = - b/a, that is wrong if a = 0. Most cas assume implicitly that a is not zero (Mathematica is an exception).