HD6417751RBP240V Datasheet, PDF (288/1226 Pages) Renesas Technology Corp – Old Company Name in Catalogs and Other Documents

6. Floating-Point Unit
Example: If the double-precision FSUB instruction (FSUB DR0, DR2) is executed with input
data DR0 = H'C1F00000 80000000, DR2 = H'C4B250D2 0CC1FB74, and FPSCR = H'000C0001,
the correct operation result is DR2 = H'C4B250D2 0CC1F973, and FPSCR.Flag.I and
FPSCR.Cause.I should be set to 1. However, the result actually produced by the FPU is DR2 =
H'C4B250D2 0CC1F974, and FPSCR.Flag.I and FPSCR.Cause.I are not set to 1.
Effects: In addition to the problem described above, the numerical size of the result of the
operation may contain a minute operation error equivalent to 1/256 of the LSB of the mantissa of
the unrounded value. This is can be described as within the scope of the subsequent rounding
mechanism. Strictly speaking, it consists of the following.
a: The infinite-precision operation result
b: The closest expressible value less than a
c: The closest expressible value greater than a
d: The operation result when a is rounded correctly
e: The operation result when a is rounded by the FPU
• The rounding error when rounding is performed correctly in Round to Nearest mode is:
0 ≤ | d − a | ≤ (1/2) × (c − b)
And the rounding error when rounding is performed by the FPU is:
0 ≤ | e − a | < (129/256) × (c − b)
If c – b is considered the LSB of the mantissa, the range of rounding error is equivalent to
1/256 of the LSB of the mantissa of the correctly rounded value.
• The rounding error when rounding is performed correctly in Round to Zero mode is:
(−1) × (c − b) < | d |−| a | ≤ 0
And the rounding error when rounding is performed by the FPU is:
(−1) × (c − b) < | e |−| a | < (1/256) × (c − b)
If c – b is considered the LSB of the mantissa, the range of rounding error is equivalent to
1/256 of the LSB of the mantissa of the correctly rounded value.
Rev.4.00 Oct. 10, 2008 Page 188 of 1122
REJ09B0370-0400