I wrote a post @ Code Compiler about Floating Point errors. Actually, i read about that in a book, and it works exactly as said. Can Any Explain me, on what is happening internally…? and Why actually there is a round-off error…?
The second question is the easiest to answer: round-off errors exist because there is an infinite number of values that need to be represented by a finite number of values (the float or double in your program). Every number is therefore represented by the nearest number that can be represented by the finite representation. This involves rounding of the values.
The first question is a bit harder:
Normally, we write numbers in a decimal representation, e.g. 3.14, 2.718, etc. When numbers get larger or smaller (near to zero) whe write them in exponential notatation, e.g. 1e100 means 1 * 10^100, 1.66e-27 means 1.66 * 10^-27 (or 1.66/(10^27)).
Internally, the computer uses a binary version of this exponential notation. This means that part of the representation is reserved to represent the significant digits (or bits, in binary), and another part (sometimes called mantissa or exponent) is reserved for the power of two by which the siginificant digits need to be multiplied.
Moving back to the decimal notation: There is only room for a fixed number of significant digits. Let’s say that 5 significant decimal digits are available. This means that 12345 can be represented as 12345e0, and 123450 can be represented by 12345e1.
123456, however, has 6 significant digits and cannot be represented directly. This will become 12345e1 (here we have rounded off the value).
When adding two numbers, the exponents of the two numbers need to be taken into account. For example: when adding 12345e0 and 11111e-3, the actual numbers that should be added are 12345 and 11.111. The result would be 12356.11, which would be represented by 12356e0.
When adding 12345e0 and 11111e-5, we find that the result as it can be stored is 12345e0. The smallest number does not have any effect, no matter how many times we add it to 12345e0.
If, however, we start with zero and add 11111e-5 twenty times, we end up with 22222e-4. If we add 12345e0 to 22222e-4, we end up with 12347e0. This is because we have reduced the rounding inaccuracy by adding numbers in the same order of magnitude as much as possible.
I hope this helps a bit.