# What are some simple problems that calculators get wrong?

A frequent issue with students is their uncritical over-reliance on computation devices. While these can of course be quite helpful, they also need to be handled with care, because they make systematic mistakes (such as rounding mistakes). Thus, a good intuition of the underlying domain is still required to make good use of them.

I am trying to come up with calculation problems that can easily be solved by hand if one makes some obvious manipulations, but will lead to wrong or no results when given to a calculator. This may, e.g. be problems where rounding errors accumulate, while some operations clearly cancel out.

• I'm not sure what you mean by systematic mistakes, nor by rounding mistakes. There is 'rounding error', of course. I've seen graphing calculators do weird things, but wrong answers to a calculation are rare. Something like .0001^5000 gets you the wrong answer of 0. You might be able to similarly get a 1 in error, but I couldn't just now. I don't know of other types of errors. Commented Apr 15 at 15:39
• Not a case of a calculator being ‘wrong’ per-se, but I will always remember how amazed my peers were in high school math class that I could multiply small matrices in my head faster than they could on their calculators. It took them 20+ seconds just to enter a simple 2 by 2 matrix of single digit positive integers, which was more than long enough for me to do the required multiplications in my head. Commented Apr 16 at 1:34
• This used to be a relatively frequent topic in sci.math in the 1990s. For example, see this 29 August 1995 post by Dave Rusin. See also the google search results for "lies my calculator told me". Commented Apr 16 at 9:31
• I downvoted because I don't think the premise is accurate. I'd like to know which calculators cause issues. It does not seem like much of a problem to me. Student over-reliance on calculators is more of a problem because they make a typo and don't think about whether the result is reasonable. Commented Apr 16 at 16:06
• @SueVanHattum I think the greater problem with student over-reliance on calculators, rather than typos, is that they have no idea which operations are actually necessary to solve a problem and they try just punching numbers and operators until an answer pops out. Commented Apr 16 at 17:05

The first sentence of the question is "A frequent issue with students is their uncritical over-reliance on computation devices," hence my answer is in the spirit of presenting something where the result from the calculator is useless, even if it is correct in some technical sense.

Calculators / graphing utilities are terrible at graphing anything which has a lot of variation in scale. For example, in a precalculus lecture last week, I had students graph the rational function

$$x \mapsto -\frac{(x^2-1)(x^2+4x+4)^3(x+2)}{(x^2-4)(x-3)^4(x+5)^3}.$$

Naively plugging this into GeoGebra gives something similar to

From this picture, many of the important features of the graph are missing. For example, it is hard to see that there are two zeros in $$(-5,2)$$, the "hole" in the graph at $$x=-2$$ is not apparent (and if you click on the graph, GeoGebra will note that there is a removable discontinuity, but there is numerical error in the reported location of the hole), the asymptotic behaviour of the function as as $$x \to \pm \infty$$ isn't very clear, and it is impossible to see what is happening on $$(2,3)$$. A lot of useful information is lost.

On the other hand, my hand-drawn graph is

The vertical and horizontal scales in my picture are total nonsense, but the asymptotes are shown, the zeros are located, the removable discontinuity is marked, and the general behaviour of the function is a bit easier to see. This picture incorporates information about singularities, zeros, and limiting behaviour—in a calculus class, we might also locate local extreme values, and incorporate information about monotonicity and concavity into the picture.

A simple question about this function is "What is the natural domain?" A student who plugs the function into a graphing utility and tries to read off the graph will get it wrong. They will almost certainly miss the removable discontinuity entirely, and are very likely to exclude an interval like $$[2,6]$$ from the domain. Indeed, I often catch students cheating because they make exactly this kind of mistake.

• One can be both correct and wrong at the same time. The computer generated graph is technically correct, but it is not useful for any real purpose, hence it is the wrong tool for the job. Personally, I find this a lot more compelling that the many examples of numerical error which others have pointed out. Commented Apr 15 at 19:53
• Agreed. But that answers an entirely different question. No calculator got anything wrong here. Commented Apr 15 at 20:07
• @pedro I don't think that any of the other examples show calculators getting things wrong, either. They show people misusing the tool and misunderstanding the results. But if you need anything to be wrong, GeoGebra does not correctly identify the removable discontinuity, which I pointed out above. Commented Apr 15 at 21:56
• In any event, the first sentence of the question is "A frequent issue with students is their uncritical over-reliance on computation devices." This gives the context that the examples sought should be ones where relying on a calculator gives a result that, if used by a student, is wrong. The GeoGebra plot is wrong, in that it captures very few of the important features of the graph. A student relying on that picture is going to make mistakes. E.g., if I ask students what the domain of that function is, dollars to donuts, several of them will say exclude $[2,6]$ and include $-2$. Commented Apr 15 at 22:01
• Yes I have to agree with @XanderHenderson here. The issue is not that computers get things wrong exactly, which is fairly rare; more typically the issue is the computer does exactly what you ask it, without understanding why you want it, and the student uncritically copies that output from the device into their final submission, because it is "correct" (in a sense). Commented Apr 16 at 14:46

The last time I taught introductory numerical analysis, my first slide was this print from excel:

If you want another example:

These calculations are clearly easy and wrong, as required.

Remark 1: Honestly, I don't know if these errors are a concern given that even NASA only needs 15 decimal places.

Remark 2: The presented errors result from the representation of repeating binary numbers.

• Wild! And yet, this doesn't seem to happen on calculators. Commented Apr 16 at 1:14
• @SueVanHattum Because almost all calculators use either fixed-point or arbitrary precision representations, not floating-point like most general purpose computers do. Commented Apr 16 at 1:30
• '=SUM(A1-B1)' is just a sum of one quantity. A simpler way is to use '=A1-B1'.
– JRN
Commented Apr 16 at 3:42
• @AustinHemmelgarn Nope, that's not the issue. All calculators I'm familiar with use floating point. The issue is use of binary floating point which cannot represent decimal fractions like 0.1 accurately. This is solved by using decimal floating point. HP calculators used the last BCD processors ever built, and systems like IBM mainframes and POWER CPUs also have DFP hardware. I'd argue here the core issue is Excel sucks, and if you used a proper environment like Python, you'd just write Decimal(0.6) and it would compute exactly. Commented Apr 16 at 21:40
• @JRN I just tried this, since I have rarely encountered this type of error on 64-bit Excel. =A1-B1 gives the correct result, =SUM(A1_B1) has the error! Perhaps the scale is taken into account with a simple expression, but not with the SUM() function. Commented Apr 17 at 6:50

Evaluate $$L=\lim\limits_{x\to 0^+}(\sin x)(\ln x)^{100}$$.

Letting $$f(x)=(\sin x)(\ln x)^{100}$$, students might use their calculators to find that:

$$f(0.1)\approx 1.66\times 10^{35}$$
$$f(0.01)\approx 2.11\times 10^{64}$$
$$f(0.00001)\approx 1.31\times 10^{101}$$

and then conclude that $$L=\infty$$ (does not exist).

But actually $$L=0$$.

\begin{align} L&=\lim\limits_{x\to 0^+}(\sin x)(\ln x)^{100}\\ &=\lim\limits_{x\to 0^+}\frac{(\ln x)^{100}}{(\sin x)^{-1}}\\ &=\lim\limits_{x\to 0^+}\frac{100(\ln x)^{99}\left(\frac{1}{x}\right)}{-(\sin x)^{-2}\cos x}\space\text{L'Hopital's rule}\\ &=-100\left(\lim\limits_{x\to 0^+}\frac{\tan x}{x}\right)\left(\lim\limits_{x\to 0^+}(\sin x)(\ln x)^{99}\right)\\ &=\dots\\ &=100!\lim\limits_{x\to 0^+}(\sin x)(\ln x)^0\\ &=0\\ \end{align}

• @Carl In this case, I would say that the only thing wrong here is the hypothetical calculator user who thinks that correct numerical estimates can be used to calculate a limit. The calculator did its job correctly, but they were incorrectly interpreted by the user. For example, the function $f(x)=x+\sin(\pi/x)$ leads to the same type of error: calculating $f(0.1)=0.1+\sin(10\pi)=0.1$, $f(0.01)=0.01+\sin(100\pi)=0.01$,... the student could conclude that the limit is zero. However, the limit does not exist. No calculator is needed here. It's not a calculator error; it's a conceptual mistake. Commented Apr 15 at 20:18
• @Pedro I agree with what you're saying, but the OP is asking for problems that "lead to wrong or no results when given to a calculator". In my example, the student who looks at the (correct) calculator results, is led to thinking that the limit is infinity.
– Dan
Commented Apr 16 at 9:28
• The maximum value is near $x=\exp(-100)$ giving a function value $\approx(100/e)^{100}\approx3.72×10^{156}$. Commented Apr 18 at 16:03

The easiest example off the top of my head is evaluating $$(9^n)^{1/n}$$ for a ridiculously large value of $$n,$$ large enough to have the calculator throw an overflow error. Something like $$(9^{9^9})^{(1/9^9)}$$ usually does it.

But honestly, if the heart of this question is just trying to come up with a good reason why students shouldn't over-rely on calculators, I don't think this is the best direction to look.

A better reason would be that if you over-rely on a calculator, you won't develop automaticity on the skills that you're offloading to the calculator, and that will prevent you from building on top of those skills.

For instance, if someone always uses a calculator and doesn't build automaticity on their arithmetic skills, then they're going to struggle in algebra. Even if they manage get through algebra, they won't have developed automaticity on their algebra skills, and consequently they're going to struggle even more in the next math course they take.

• See also: Every math competition always has one or two questions that use the current year somewhere, often in an exponent or another place that creates inconveniently large numbers. The point is to force the student to reason about the problem and find some elegant shortcut, rather than trying to solve it by brute force number crunching. Commented Apr 16 at 0:59
• Nice example, thanks! I agree with your evaluation, but the point is to convince students who believe that they do not need algebra skills, automaticity etc. because there are calculators to reconsider.
– Carl
Commented Apr 16 at 14:20
• @Carl: For many such students, they *don't need" these skills. Thus, trying to convince them otherwise will be difficult. The thing is, having these skills is part of your course, so they have to learn them anyways. Similarly, a student of a history class can't argue against learning history, even if they won't need it. Commented Apr 17 at 13:34
• @Brian That's the thing, though. Unlike algebra or trig, ordinary people need history on a regular basis. Just look at how many stupid things, that past generations knew were stupid, we're repeating today because current generations could not be bothered to study history and learn from it! Commented Apr 17 at 19:38
• @Brian I disagree. We need education, both in math, and in history; if "you need it because you are in my course" was the only reason in favor of learning something, it would be an argument against the course. Mathematical skills are required for forming informed opinions in a world full of quantified facts.
– Carl
Commented Apr 18 at 14:51

Ask a TI-89 to evaluate $$\sin(n \pi)$$ for different integers $$n$$, and you'll start out fine (when $$n = 0, 1, 2, 3$$), but starting with $$n = 4$$, you'll get values like $$-2 E -13$$.

Or there's $$8^{8^8}$$ which has the output $$\infty$$ on that calculator.

• If you type in $9^{9^9}$, will the output be a sideways $9$?
– Dan
Commented Apr 15 at 23:02
• @Dan Try it out and report back! Commented Apr 15 at 23:34
• Don't own any TI, but does it really return Infinity rather than some overflow warning? That seems very silly. Commented Apr 17 at 0:25
• It returns $\infty$, and also displays a warning like "overflow value converted to infinity". Commented Apr 17 at 0:34

The easiest of all:$$\frac{1}{3} \cdot 3$$ :-)

You can also go for large squares, something like:

$$\frac{123456789^2 - 123456788^2}{123456789 + 123456788}$$

=> for a human, this is just $$\frac{a^2-b^2}{a+b} = a - b$$

• 1/3*3 works fine on my calculator. Commented Apr 15 at 15:34
• On lots of calculators, you get 0.9999999. Commented Apr 15 at 17:02
• @Sue In general, it happens on really cheap calculators in which the internal calculations have the same precision as the displayed digits. In other words, if you divide 1÷3, it says 0.3333333 on the display, and it has exactly 0.3333333 in memory. As such, when you multiply 0.3333333×3, you get 0.9999999. But all you need is a single digit more precision than displayed to get the right answer. 1÷3=0.33333333, which is rounded to 0.3333333 for display. Multiply by 3, and you get 0.33333333×3=0.99999999, which is rounded to 1 for display. Commented Apr 15 at 23:27
• @trikly, Have you seen it on a particular calculator? I'm curious which one(s). Commented Apr 16 at 1:15
• @SueVanHattum: as a child, my mother had a shop where she used a simple pocket calculator, showing the exact behaviour, explained by trlkly. Maybe nowadays such calculators are not very much in use anymore (seen the fact that there are even better calculators, installed as an app on a smartphone) but if you get the opportunity to lay your hands on a simple pocket calculator (without scientific functions or so), you might be surprised of the amount of digits of $\frac{1}{3} \times 3$ :-) Commented Apr 17 at 12:54

In a rather different vein, calculators can be correct in a numerical sense but miss an optimal solution method. In this answer two such cases are described, and I refer the reader to the above link for details:

• The quartic equation $$x^4+x^3+x^2-x+1=0$$ can be solved by setting up a factorization $$(x^2+ax-1)(x^2+\overline{a}x-1)$$ and solving quadratic equations. Wolfram Alpha recognizes such a solution, but a TI-89 calculator gives only numerical answers.

• The quintic equation $$x^5+10x^3-5=0$$ is provably soluble by radicals and gives a relatively simple form for the roots, but even Wolfram Alpha misses this and gives only numerical answers.

Occasionally I ask my students true-false questions of the following form:

• T/F? $$1/3 = 0.333333333$$
• T/F? $$\pi = 3.141592654$$

They usually get these right as a group, but I think it's good to check in on it.

• It would be very instructive to include something like $\frac{7}{512}=0.013671875$ Commented Apr 16 at 0:05
• They're both false, right? Seeing as both are actually equal to an unending decimal string? Commented Apr 17 at 2:02
• Given the number $13671875$: (1) add the number to itself; (2) if the last digit is nonzero stop, else drop the final zero and return to (1); (3) if steps (1) and (2) can be iterated nine times with $7$ as the endpoint, then $13671875×10^{-9}=7×2^{-9}$ using only addition and zero removal as operations. Commented Apr 18 at 16:20

Combinatorics might be a good place to look.

A very basic example: given 200 items, how many ways are there to pick 1 item?

Using the permutation formula involves calculating $$200!/199!$$, which not even Google's calculator can manage.

But it's very easy to do by hand:

$$200!/199! = (200)(199!/199!) = 200$$

• I think "using the permutation formula" is an unwanted learning outcome for "how many ways are there to pick 1 out of 200 items". Anyone who does not answer "200", without calculating anything, has not understood. Commented Apr 17 at 14:28
• A lot of pocket calculators include a permutation function, e.g. the nPr function on a modern Casio which displays as 200P1 and gives the answer 200. Commented Apr 17 at 20:30
• @TorstenSchoeneberg fine, make it 2 out of 200. Commented Apr 18 at 10:08
• @infinitezero Then it's $\frac{200\cdot 199}{2}$ (if order does not matter) or $200\cdot 199$ (if it does) which a calculator has no trouble with either, and although I would brag to my students that I can do that in my head, I would not scold them for using a calculator. And I would do the same as soon as it's 3 or more out of 200. In fact, one learning outcome in this lesson would be to find a formula which can be put in the calculator, and as user Silverfish says, for both options, a calculator to be used in such a class has buttons nPr and nCr. Commented Apr 18 at 16:27

Many cheap calculators ignore operator precedence. Type 1+2*3 on a good one and you'll get 7, do it on a cheap one and you get 9.

And then there's the anecdote a math teacher friend told me where it's unclear if the square of -5 is -25 or 25.

• The value of "one plus two, times three" is different from the value of "one plus twotimesthree". Prior to the advent of computer languages like FORTRAN, operator precedence would generally be indicated by placement. If one had a column of numbers with a plus in front of the last one and a line underneath, that would indicate that the next value should be the sum. If underneath that was an "x" and another value, and there was a line under that, that would indicate that the product should be placed somewhere below that. Commented Apr 16 at 16:58
• If one wanted to notate an expression meaning "one plus twotimesthree" on a single line, 1+2(3)" would have been considered a clear and unambiguous way of doing so. If one wanted to notate "one plus two, times three", writing symbols in the same order as they would be spoken, "1 + 2 x 3" would be viewed as informal notation for use in situations where such meaning could be inferred from context. Even in speaking, the proper way to express "one plus twotimesthree" would be "one plus quantity two times three". Commented Apr 16 at 17:08
• It is not cheap vs good, it is simple vs algebraic. Simple one have just two registers, so cannot hold the complete expression. Well, I guess they are cheap :) Commented Apr 17 at 1:59
• @supercat the column you describe uses notational convention (implicit as it may be) to override default precedence, just as one does with parentheses. Commented Apr 18 at 18:26

Computers are quite bad at drawing curves with self-intersections. Examples below are made with Maple 13 (a quite dated version).

The graph of $$x^2+y^2=3xy$$ looks particularly strange.

We can get a better figure by forcing the software to compute more values.

Even with $$1000$$ points, the graph may be confused with a hyperbola. It is actually two lines that intersect at the origin.

Finally, here is an example of a polar curve $$r=\sqrt{\cos(2\theta)}$$.

• It should be noted that Desmos (which is more popular among students these days) does excellently with both of these graphs, but struggles with others. I don't have other examples off the top of my head, though. Commented Apr 18 at 15:13
• @OpalE: good to know. I should also have mentioned that my version of Maple is quite dated. Commented Apr 18 at 23:25

Some scientific calculators will round certain rational numbers to rational multiples of pi. For example, $$\frac{11^6}{13} \approx \frac{156158413}{3600} \cdot \pi$$. They may be assuming that the user is converting a rational multiple of degrees or gradians to radians.

Here is a simple example. $$\def\lfrac#1#2{{\large\frac{#1}{#2}}}$$

$$\color{green}{\Big( 99 · \big( 100·\sum_{k=0}^{10} (-\lfrac1{99})^k-99 \big)^\lfrac1{10} - 1 \Big)^\lfrac1{9} = 0}$$.

Windows Calc ( 99*( 100*(1-1/99+1/99y2-1/99y3+1/99y4-1/99y5+1/99y6-1/99y7+1/99y8-1/99y9+1/99y10)-99 )y(1/10) - 1 )y(1/9) = gives 0.0117619348321432859010038254248.

My Casio scientific calculator does not have enough input space for the whole expression so I used it to first compute $$\big( 100·\sum_{k=0}^{10} (-\lfrac1{99})^k-99 \big)$$, which gives 0, and then used it to next compute $$(99·0^\lfrac1{10}-1)^\lfrac19$$, which gives -1.

Note that both of these results are as expected, because the expression is intentionally constructed to create and amplify rounding errors.

Continuing in the vein of "The calculator is right, but the interpretation by the user is wrong" I add a class of problems of the form

$$\lim_{x \to \infty} \frac{p(\log(x))}{q(\log(x))}$$

where $$p$$ and $$q$$ are polynomials. While another user has given a specific answer in this vein that uses L'Hopital's rule, I think adding a general class of functions might be of some use as well. This class of limits are also readily evaluated using elementary techniques (e.g. without funny business like L'Hopital's rule), and so they can be introduced early in the calculus course (around when you need to get buy-in from students about this very issue).

One such example is evaluating the limit, $$\lim_{x\to \infty}\frac{\log(x)}{10+\log(x)}$$

The answer is, of course, $$1$$. However, students may be inclined to estimate the limit numerically or visually. A graph of the function will not readily yield this limit. In fact, one must enter values of at least $$10^{40}$$ to reach a function output of at least 0.8, values of at least $$10^{90}$$ to reach an output of at least .9, at which point the graph looks visually entirely flat; reaching what students might consider the "gold standard" of 0.99 requires inputs of at least $$10^{990}$$, whereas Desmos won't graph past the $$10^{100}$$ scale. Both entering values and looking naively at a graph of the function will lead to difficulty obtaining the correct result.

Problem: Evaluate $$\text{log}_{10}(10^{\frac{1}{10^{17}}})^{-1}.$$

By hand, this easily evaluated to be $$10^{17}$$, but, for example, Python 3.10 complains about a "division by zero" error:

Input:

import math
math.log10(10 ** (1 / 10**17)) ** (-1)


Output:

ZeroDivisionError: 0.0 cannot be raised to a negative power


Wolfram Alpha still gets it right (and so will Python, when you change the datatype), but in that case, it suffices to replace 17 with something larger. (For 99999, Wolfram Alpha does not return an answer, and plugging in 999999999 makes Wolfram Alpha return "indeterminate".)

Another impressive example, found here in a worksheet called "rounding errors lab" https://cse.unl.edu/~scooper/securecoding/RoundingErrorsLab.pdf is that Google will compute 1-0.999998=0.00000199999.

• Why is anyone else's LaTeX code compiled, but not mine?
– Carl
Commented Apr 18 at 15:52
• Mis-matched { } characters, so MathJax cannot understand it. Adding one extra } in there, I get $$\text{log}_{10}(10^{\frac{1}{10^{17}}})^{-1}.$$ Commented Apr 18 at 16:28

An example discussed in math.se in the past is this: $$-4^2$$ is interpreted by all mathematicians as $$-(4^2) = -16$$. But some calculators (and even some elementary-school teachers) will interpret it as $$(-4)^2 = 16$$.

Of course I am talking about algebraic calculaors, where you can enter something like -4^2.

example
In Tiger Algebra, I typed -4^2

When I was looking for this example, I was pleased to find that many on-line algebraic calculators do this the other way. Are things improving?

• Which calculators? Commented Apr 16 at 1:12
• Standard spreadsheet software also interprets it incorrectly. Commented Apr 16 at 13:26
• This is ambiguous enough that the programming language JavaScript will just throw an error if you try it! Commented Apr 16 at 15:19
• This isn't an error by the calculator though. It's an error by the calculator user if they expect to enter symbols into a simple calculator in the exact order they're written, when the calculator is limited to parsing button presses as soon as they're entered. The calculator itself does exactly what the user asked for. Commented Apr 16 at 17:02
• As @Graham is getting at, most calculators don't allow you to enter LaTex. So when you say "calculators interpret $-4^2$ as -16", what you mean is "people press a sequence of buttons that they think signifies $-4^2$, but they're actually pressing a sequence of buttons that represent $(-4)^2$. The $^2$ button on a calculator doesn't mean "square the last number I entered", it means "square the last number that was calculated". Commented Apr 17 at 1:34

Type the following into Wolfram Alpha:

integrate sin(pi x)/(x(1-x)) from x = 0 to x = 1

Half the time Wolfram Alpha gives a correct estimate and half the time it says the integral is $$0$$, even when it displays a valid graph showing the function is positive on $$[0,1]$$. If you try that and get a positive answer the first time, then input the request a second time and, in my experience, Wolfram Alpha will say the integral is $$0$$.

This glitch was brought to my attention several years ago and it's still there (I just checked).

• That is intriguing! Any idea what happens there?
– Carl
Commented Apr 22 at 11:06
• I have no idea.
– KCd
Commented Apr 22 at 11:37
• Have you reported it as a bug to Wolfram? If not, please do so. Commented Apr 23 at 6:04
• @KCd, I found the reason. Apparently Wolfram Alpha parses the query in plain English differently each time. If the result is 3.7..., the input (under "Plain Text" button) is N[Integrate[Sin[Pi x]/((1 - x) x), {x, 0, 1}]] (i.e. calculated numerically), but if the result is 0, this is for input Integrate[Sin[Pi x]/((1 - x) x), {x, 0, 1}]. Commented Apr 23 at 6:13

# Division by 0

This is pushing the definition of "easily be solved by hand" since the "solution" is to see that it's indeterminate.

# Which calculators have this issue?

When you try 1/0 on the system calculator that comes with Windows XP you'll get "Infinity". This is wrong, however Windows 8 no longer has this problem and I doubt that your students will try to rely on Windows XP.

For a modern example https://www.wolframalpha.com/input?i=1%2F0 returns "complex infinity". This is intentional behavior, however if you are working strictly with real numbers then this isn't the right answer.

This answer is pretty lame but it has lead to confusion before. I met some students in high school and college who thought "Infinity is the correct answer to 1/0 but that's only for advanced math" however this isn't true. Even for surreal numbers (AFAIK) the result of 1/0 is undefined. I'm guessing this misconception came about because of the Windows XP calculator (and maybe some other old calculators with the same problem) so this is likely not an issue anymore.

Where this answer does still matter is for programming. The FPU and many programming languages intentionally have 1/0 = Infinity because that's the intended way to obtain an Infinity value from the FPU. Programmers who aren't aware of this may not realize that 1/0 doesn't result in NaN. Failure to account for division by 0 will lead to calculation bugs such as that in Windows XP.

# Other programming issues

Everyone else has already commented on rounding errors and precision limitations. These are called underflow and overflow errors. There's also truncation and other issues. When programming math it's important that you understand the math!

A simple example, due to Velvel Kahan (UCB), is $$(((4/3)-1)*3)-1$$ is usually not zero (and should not be zero on IEEE-754 compliant floating-point systems; the HP 35s yields $$-1{\rm e}{-}11$$). Note parentheses. It normally yields the "machine epsilon", the smallest number when added to $$1$$ yields a sum different than $$1$$. This is the smallest difference between (binary64) floating-point numbers lying in the interval $$[1,2)$$.

This example does not work on the Apple iPhone (added: or Google and some others — see comment below), which has some strange numerics (I suspect "cosmetic rounding" but I cannot confirm). On the iPhone, compute $$(((258/257)-1)*257)-1$$ or $$((4/3)-1)-(1/3)$$ These do not result in machine epsilon, however.

For a TI-84, this variation returns the (negative of) the epsilon: $$(((100/3)-33)*3)-1 \mapsto -1{\rm e}{-}12$$

On the HP Prime Lite app (iOS), $$(((10/3)-3)*3)-1 \mapsto -1{\rm e}{-1}11$$

On the Google calculator, we have $$(((1000/3)-333)*3)-1 \mapsto 2^{-44} \approx -5.6843419{\rm e}{-}14$$ which is, in Matlab code, eps(1000/3), the epsilon relative to 1000/3 in binary64 (double precision floats).

Similarly $$0.3 - (3*0.1)$$ is not zero on binary IEEE-754 compliant calculator/computer systems (again, it's zero on iPhones). It's not hard to come up with similar examples (e.g., $$(23*0.1 - 20*0.1) - 3*0.1$$).

On decimal calculators, the answer should be zero (no rounding error).

• While this is true for calculators that use floats, it seems most calculators nowadays either round the result or use arbitrary precision calculation. None of the calculators I tried (TI 36X II, Android 13, Qalculate, Google) fell for these examples.
– jpa
Commented Apr 18 at 7:43
• @jpa Thanks. I appreciate it. I mainly use computer systems and don't have any recent calculators. I hadn't realized that "correcting" round-off had become a thing. On Google, $((4/3)-1)-(1/3)$ gives $-5.55\dots\times10*{-17}$, which is the result expected in double floats. Google also returns an integer for $\exp(\ln(2)+2^(-49))-2$, so it seems to do what I called "cosmetic rounding" like the iPhone (except the basic engine in the iPhone seems to use quad precision). That is, results close to an integer are rounded to an integer. Commented Apr 18 at 11:03
• Interesting that you get -5.5e-17 on Google, because I get 0! But I guess this does point out kind-of fallibility on calculators, though it can be annoyingly difficult to demonstrate.
– jpa
Commented Apr 18 at 12:20

Why not problems like $$(-1)^z=2$$, or $$i^z=3$$? Even better if you ask for all solutions in their exact form. ($$i$$ in this case being defined as $$\sqrt{-1}$$)

For both of these examples:$$(-1)^z=2\\\text{We can use }(-1)=e^{2i\pi n}\text{ where }n\in\mathbb Z\\\text{So it becomes }e^{2i\pi nz}=2\\\implies2i\pi nz=\ln2\\\implies z=\frac{\ln2}{2i\pi n}=\frac{-i\ln2}{2\pi n}\quad n\in\mathbb Z$$And then$$i^z=3\\\text{We can use }i=e^{i\pi/2+2i\pi n}\text{ where }n\in\mathbb Z\\\text{So it becomes }e^{(i\pi/2+2i\pi n)z}=3\\\implies(i\pi/2+2i\pi n)z=\ln3\\\implies z=\frac{\ln3}{i\pi/2+2i\pi n}=\frac{2\ln3}{i(\pi+4\pi n)}\\=\frac{-2i\ln3}{\pi+4\pi n}\quad n\in\mathbb Z$$The reason I'm saying it's even better if you ask for all solutions in exact form is because even if their calculators can solve this in exact form, most likely it's just going to be one possible solution, rather than all possible solutions.

• Sure, these are examples of calculation problems that (simple) calculators cannot solve, but I am more after examples where they make numerical mistakes. They exist, of course (which is why numerical math exists), but I am looking for examples simple enough so that students can get an immediate and strong experience from it.
– Carl
Commented Apr 15 at 17:36
• "$i$ in this case being defined as $\sqrt{-1}$" says nothing, because I can also define it like that, and yet my $i$ could be your $-i$. Commented Apr 19 at 3:34

Here's an expression that evaluates to something very clearly inaccurate on most calculators:

$$\frac{1}{\left(\left(\frac{4}{3}\right)-1\right)\cdot 3-1 + 10^{-20}}$$

Most floating point calculators should give something like   -4503802460601329.0, but the correct answer is of course $$10^{20}$$, a factor of $$10^5$$ larger. The relatively small floating point errors add up.

• My Sharp EL-510, which cost me 12 bucks a few years ago, gives the correct answer $10^{20}$. Commented Apr 19 at 3:29

Most calculators completely ignore the concept of a multivalued function.

A very basic example:

>>> [4] [√]
2


What we got is just the principal square root. No calculator, save for a computer algebra system, reports the other root $$-2$$.

Notice how basic this example is: all the roots roots are real, and there are only two (i.e. a finite number) of them. There are more complex cases, such an $$n$$ roots of $$\root n\of {x}$$ (for $$x \neq 0$$; if we allowed the results to be complex), or infinitely many roots of functions with multiple branches, such as inverse trigonometric functions or a (complex) logarithm.

To demonstrate that to students, think of a math problem where using the principal value blindly (given by a calculator) leads to contradictory results.

• I think that there is some clarification that can happen here. It is mathematical convention that the square root and the inverse trigonometric functions are indeed proper functions (and not multivalued). This means that the calculator is correct. However, what it does not do is provide all solutions to x^2=4 by the square root, nor does it provide all solutions to sin(x)=1/2 by using the arcsin function. This is a good answer, but should be careful to not call the square root function a multivalued function -- it is supposed to always return the principal square root of a number. Commented Apr 16 at 14:46
• In this case, the calculator is doing the correct thing. The notation $\sqrt{x}$ means "the principal square root of $x$", unless specifically noted otherwise. To suggest otherwise adds to the confusion which students already seem to face with respect to the difference between $\sqrt{2}$ and "solutions to the equation $x^2 = 2$". Commented Apr 16 at 20:31
• Agree that customarily today the radical sign is defined to mean the principal square root. You may find interesting a few historical articles by Bernardo Gomez, "The ambiguity of the sign √" (2009), and "Historical conflicts and subtleties with the √ sign in textbooks" (2010). Commented Apr 20 at 1:28