Lua's Decimal Calculation
Posted 08 September 2006 - 05:07 AM
Posted 09 September 2006 - 05:12 PM
print(789.000006) Output: 789.00000599999998
Posted 09 September 2006 - 06:05 PM
Posted 09 September 2006 - 08:26 PM
You are right, explaining the same thing again and again is annoying, but I will answer for the last time: This is not a bug. This is normal computer arithmetics which cannot be avoided, even on super computers. Using a computer means that you accept the fact that it has limited floating point precision.
floating point precision... Do we really need to explain that one more time?
In theory, it is possible to use arbitrary floating point precision, but this is not a good idea for the ClassPad, because it costs computation time... a lot of computation time.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users