HD6417750RF240V Datasheet, PDF (482/606 Pages) Renesas Technology Corp – Renesas 16-Bit Single-Chip Microcomputer H8S Family/H8S/2600 Series

Section 16 A/D Converter
16.6 A/D Conversion Precision Definitions
This LSI's A/D conversion precision definitions are given below.
• Resolution
The number of A/D converter digital output codes
• Quantization error
The deviation inherent in the A/D converter, given by 1/2 LSB (see figure 16.4).
• Offset error
The deviation of the analog input voltage value from the ideal A/D conversion characteristic
when the digital output changes from the minimum voltage value B'0000000000 (H'00) to
B'0000000001 (H'01) (see figure 16.5).
• Full-scale error
The deviation of the analog input voltage value from the ideal A/D conversion characteristic
when the digital output changes from B'1111111110 (H'3E) to B'1111111111 (H'3F) (see
figure 16.5).
• Nonlinearity error
The error with respect to the ideal A/D conversion characteristic between zero voltage and full-
scale voltage. Does not include offset error, full-scale error, or quantization error (see figure
16.5).
• Absolute precision
The deviation between the digital value and the analog input value. Includes offset error, full-
scale error, quantization error, and nonlinearity error.
Rev. 7.00 Sep. 11, 2009 Page 446 of 566
REJ09B0211-0700