Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Inaccuracy when using modulo in Lua

I used modulo (%) in Lua just a moment ago and I noticed that it was for some reason very inaccurate. I used it in World of Warcraft, but I would assume that it's at least somewhat up to date.

Anyhow, using the following example, the output would be 1;

print((0.6/(0.1^(2-1)))%1)

But when I use the following, it would return 0;

print((0.6*(10^(2-1)))%1)

And as far as I have been educated, 0.6/0.1 should be equivalent to 0.6*10.

It's not really a question or a problem, but I am merely curious about the cause. Inaccuracies in maths would be quite devastating in my opinion.

like image 576
Diederik Hu Avatar asked May 26 '13 01:05

Diederik Hu


People also ask

How to use modulo in lua?

The Lua modulo operator is used to get the remainder of a division. The modulo operator is a built-in operator in the lua programming. Modulooperatoris an athematic operator and it is denoted by the % symbol. The modulo operation return the whole number remainder, instead of the division result.

Does Lua have modulus?

According to the Lua Documentation Lua supports the usual arithmetic operators: the binary + (addition), - (subtraction), * (multiplication), / (division), % (modulo), and ^ (exponentiation); and unary - (negation).

Is the modulus operator?

The modulo operator, denoted by %, is an arithmetic operator. The modulo division operator produces the remainder of an integer division. produces the remainder when x is divided by y.


1 Answers

If you look in the Lua manual, the % operation is defined as

 a % b == a - math.floor(a/b)*b

And so the expression

 6 - math.floor(6/1) * 1

will equal zero.

However, due to floating point inaccuracies (0.1 does not exist as a floating point number, the closest should be 0.1000000014...), these should be your intermediate results:

> print(math.floor(0.6*10))
6
> print(math.floor(0.6/.1))
5

So you get

6 - 6 * 1 = 0
6 - 5 * 1 = 1

Further Reading

  • Lua Manual
  • Wikipedia: Floating-Point accuracy
like image 126
dualed Avatar answered Oct 03 '22 02:10

dualed