kicad-developers team mailing list archive

["Alex G." <mr.nuke.me@xxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/12/2010 12:36 AM, Dick Hollenbeck wrote:
>
> One poster said something that I still wonder about, and that C++
> cannot reliably multiple and divide by 2.54 and do a round trip
> conversion using the current internal unit granularity.
>
> It is mind boggling that 2.54 and using a C++ double does not allow
> this to happen. I don't have time to test it, but I am wondering
> about it. Is this a problem with 64 IEEE float, or that we simply
> do not have enough resolution in our internal units?
>
It boils down to how floating-point values are stored. It's the very
nature of floating point that does not allow an _exact_ representation
of a number to be stored. I remember debugging code where I
specifically set a double to 5, but it showed up as 4.999999...
It's not a matter of resolution, as much as it is of nature, and it is
definitely not a fault of C++. You have less 2^64 numbers that a
double can represent exactly, but it's range is far greater than that.
It's very unlikely that the number you wish to represent is one of
those numbers.

Alex
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJMs4rqAAoJEL0kPHNPXJJKM10QAJ/Sex3oH5fCda9vISz21mEU
eJIfds8pDFVZ74kN1Bc75HxSMaYxyWWBjUq1VbFypRboNrnpaZCMRXEGy9CmJDtT
o24lsv1Dfii+Qy+NxvG37BMsC06E50uuV1Az8ukiV8BU480vhsCLbnb1//RiAKMm
8+5wiN806DQ3VHzpBPVgDOGXOLBRvmzijMmQudyR55q2eqIRj9Xi0bt+sxznrQ6v
DEf4jkhjr0rh5RQJHuqitEM5v/heiN7DXoI8qRm8pJlFYV1tjav6A+8+dGQn8QPk
4rw8dXj7cwYuHzlwtyXOjHCmcwKwJ+OA7s1uD8Su7pE8PEhhkGOfyZDWee/hqfTJ
xthPvY6IbUKzAtPuyY7STAtzV0ZVWgU4fYkOcOOpnYA1Dsw9dzuvmcWCkf0U1psH
XbIDPIGpwgbygVuYbuFSp1lpO261GbM6cyVZwFSouTc1ZJDefpZwFl0Ai0v8kujk
7vOyh9wkWuaOD1smq0r/SauwYBpo3RdUo2xUnoiNm6ceUBq84b1FS/efj2bT/4bx
vLYtpD0nisj/Agwx5JLWlwANrEm6udJpp5Ea8H+aI9rvQI+yH489eBQVVYxw3NOT
Gca2wUKis8vMz/M7aTkFQOKw6ykLpoBnxXCiCEnWDhEJA/0x0u7iV+T3IglX0FWJ
I5zwIsq7e7gXYlPAmKkm
=ALyu
-----END PGP SIGNATURE-----