Why the result of an exp got wrong (nealy true) when use lua.
Lua's Decimal Calculation
Started by
vanhoa
, Sep 08 2006 04:07 AM
4 replies to this topic
#1
vanhoa
-
- Members
-
- 854 posts
Casio Overlord
- Gender:Male
- Location:Vietnam
-
Calculators:
AFX 2.0, CP 300, CP 330, nSpire, TI 89, FX 5800
Posted 08 September 2006 - 04:07 AM
#2
Orwell
-
- Members
-
- 777 posts
Casio Overlord
- Gender:Male
- Location:Paris - France
-
Calculators:
Casio AFX 1.02 / Casio ClassPad 300
Posted 08 September 2006 - 07:50 AM
What makes you think this? Can you give an example?
#3
Kilburn
-
- Members
-
- 491 posts
Casio Technician
- Gender:Male
- Location:France
- Interests:Blah
-
Calculators:
FX-7500 G
ClassPad 300
Posted 09 September 2006 - 04:12 PM
Maybe this:
print(789.000006) Output: 789.00000599999998
#4
Orwell
-
- Members
-
- 777 posts
Casio Overlord
- Gender:Male
- Location:Paris - France
-
Calculators:
Casio AFX 1.02 / Casio ClassPad 300
Posted 09 September 2006 - 05:05 PM
floating point precision... Do we really need to explain that one more time? 
#5
PAP
-
- Members
-
- 681 posts
Casio Overlord
- Gender:Male
- Location:Somewhere in Europe.
- Interests:Computer Algebra, Numerical Analysis.
-
Calculators:
ClassPad 300 (plus an old Casio model, with only a few Kb ram).
Posted 09 September 2006 - 07:26 PM
You are right, explaining the same thing again and again is annoying, but I will answer for the last time: This is not a bug. This is normal computer arithmetics which cannot be avoided, even on super computers. Using a computer means that you accept the fact that it has limited floating point precision.floating point precision... Do we really need to explain that one more time?
In theory, it is possible to use arbitrary floating point precision, but this is not a good idea for the ClassPad, because it costs computation time... a lot of computation time.
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users
