
B-63944EN/02 PROGRAMMING 16.CUSTOM MACRO
- 417 -
This is because an error may occur in operation N20 and the result
may not be
#2=2.0000000000000000
but a value a little smaller than 2 such as the following:
#2=1.9999999999999997
To prevent this, specify N30 as follows:
N30 #3=FIX[#2+0.001];
Generally, specify the FIX function as follows:
FIX[expression] → FIX[expression ±ε]
(Specify +ε when the value of the expression is positive or -ε when
it is negative, and 0.1, 0.01, 0.001, ... for ε as required.)
NOTE
The operation result of exponential function
#i=EXP[#j]; overflows when #j exceeds about 790.
When bit 0 (F16) of parameter No. 6008 is set to 1
Errors may occur when operations are performed.
Table 16.3 (b) Errors involved in operations
Operation Average
error
Maximum
error
Type of error
a = b*c 1.55×10
-10
4.66×10
-10
a = b / c 4.66×10
-10
1.88×10
-9
a = b 1.24×10
-9
3.73×10
-9
Relative error(*1)
a = b + c
a = b – c
2.33×10
-10
5.32×10
-10
(*2)
a = SIN [ b ]
a = COS [ b ]
5.0×10
-9
1.0×10
-8
a = ATAN [ b ] / [ c ] 1.8×10
-6
3.6×10
-6
Absolute error(*3)
degrees
NOTE
1 The relative error depends on the result of the
operation.
2 Smaller of the two types of errors is used.
3 The absolute error is constant, regardless of the
result of the operation.
4 Function TAN performs SIN/COS.
5 Note that, in the case of natural logarithm #i=LN[#j];
and exponential function #i=EXP[#j];, the relative
error may become 10
-8
or greater.
6 The operation result of exponential function
#i=EXP[#j]; overflows when #j exceeds about 110.
• The precision of variable values is about 8 decimal digits. When
very large numbers are handled in an addition or subtraction, the
expected results may not be obtained.
ε
a
ε ε
MIN
b c
ε