
F. FLOATING-POINT OPERATION LIBRARY
B–61863E–1/06
APPENDIX
504
When using the floating point library functions in a program, always be
sure to exercise caution as to whether the degree of precision is adequate.
The effective number of digits in data for input and output is eight decimal
digits. Therefore, the size of the decimal part of a number is determined
depending on the size of the integer part.
For input, a total of eight digits are accepted for the integer and decimal
part.
For output, the ninth digit is rounded off as shown below.
1234.00001 → 1234.0000
12345677.7 → 12345678
However, when the integer part has nine digits, a data error occurs.
123456789 → 123456790
Example 1:
123.45678 + 0.000004 = 123.456784
The internal result of the above operation is 123.45678, meaning that
adding 0.000004 had no effect.
Example 2:
123.45678 + 0.000005 = 123.456785
However, in the above case, since the last five is rounded up, the internal
result is 123.45679.
The effective number of digits for these library functions is also eight
decimal digits.
The number of digits after the decimal point is specified with an
argument. Only the specified number of digits after the decimal are kept.
Data 3 digits specified after decimal point No specification
1 1.000 1.0000000
12345 12345.000 12345.000
123456 Data error 123456.00
As shown in the table, for library functions that do not specify the number
of digits below the decimal point, the number of these digits is determined
by the size of the integer part. When three digits are specified below the
decimal point, the integer portion can only be up to five digits with a
maximum value of 99999.
Up to seven digits can be specified below the decimal point. When eight
or more digits are specified, seven digits are used.
F.3
PRECISION
F.3.1
Library Functions that
do not Specify the
Decimal Place
F.3.2
Library Functions that
Specify the Decimal
Place