PROGRAMMING 15. CUSTOM MACRO
B–63004EN/02
295
Errors may occur when operations are performed.
Table 15.3 (b) Errors involved in operations
Operation Average
error
Maximum
error
Type of error
a = b*c 1.55×10
–10
4.66×10
–10
a = b / c 4.66×10
–10
1.88×10
–9
1.24×10
–9
3.73×10
–9
a = b + c
a = b – c
2.33×10
–10
5.32×10
–10
a = SIN [ b ]
a = COS [ b ]
5.0×10
–9
1.0×10
–8
a = ATAN [ b ] / [ c ] (*4) 1.8×10
–6
3.6×10
–6
NOTE
1 The relative error depends on the result of the operation.
2 Smaller of the two types of errors is used.
3 The absolute error is constant, regardless of the result of the
operation.
4 Function TAN performs SIN/COS.
S The precision of variable values is about 8 decimal digits. When very
large numbers are handled in an addition or subtraction, the expected
results may not be obtained.
Example:
When an attempt is made to assign the following values to
variables #1 and #2:
#1=9876543210123.456
#2=9876543277777.777
the values of the variables become:
#1=9876543200000.000
#2=9876543300000.000
In this case, when #3=#2–#1; is calculated, #3=100000.000 results.
(The actual result of this calculation is slightly different because
it is performed in binary.)
S Also be aware of errors that can result from conditional expressions
using EQ, NE, GE, GT, LE, and LT.
Example:
IF [#1 EQ #2] is effected by errors in both #1 and #2, possibly
resulting in an incorrect decision.
Therefore, instead find the difference between the two variables
with IF[ABS[#1–#2]LT0.001].
Then, assume that the values of the two variables are equal when
the difference does not exceed an allowable limit (0.001 in this
case).
D Operation error
a + b
Ǹ
Relative error(*1)
ε
b
Min
(*2)
ε
b
ε
c
ε
Absolute error(*3)
degrees