← Back to team overview

kicad-developers team mailing list archive

Re: Question about gerber job file numeric format


On 28/12/2019 13:37, Mark Roszko wrote:
But that 1.600 is not a double/floating number, it  truncated from the
double of 1.600000000000000088817841970012523233890533447265625

The value you quote here is illusory: the ieee-754 double format is incapable of storing more than 17 decimal (53 bimary) digits, and that value has over 30 digits.

I wrote a program (below) to demonstrate this, showing that printing using the generic float format "%g" you do indeed get 1.6, and you can only get the 'additional' precision by forcing the formatter to 'find' it using the precision options of printf. As well as this gcc/Linux version, I checked with both python 3.8 on windows 10 and php 7.3 on Linux and they all have the same behaviour (unsurprisingly).

The "Hex" printout shows that assigning a simple '1.6' double has the same encoded representation as the longer value, backing up my 'illusory' comment. Those other digits are not there:


double = 8 bytes, long long = 8 bytes
For: v = 1.600000000000000088817841970012523233890533447265625
  Natural:  %g    v=  1.6
  30 digit: %f    v=  1.600000000000000088817841970013
  Hex:            v=  3ff999999999999a
For: w = 1.6
  Natural:  %g    w=  1.6
  30 digit: %f    w=  1.600000000000000088817841970013
  Hex:            w=  3ff999999999999a


#include <stdio.h>

void main() {
  double v = 1.600000000000000088817841970012523233890533447265625;
  double w = 1.6;

  printf("double = %lu bytes, long long = %lu bytes\n", sizeof(double), sizeof(long long));

  long long *vx = (long long *)&v;
  printf("For: v = 1.600000000000000088817841970012523233890533447265625\n");
  printf("  Natural:  %%g    v=  %g\n", v);
  printf("  30 digit: %%f    v=  %30.30f\n", v);
  printf("  Hex:            v=  %llx\n", *vx);

  long long *wx = (long long *)&w;
  printf("For: w = 1.6\n");
  printf("  Natural:  %%g    w=  %g\n", v);
  printf("  30 digit: %%f    w=  %30.30f\n", v);
  printf("  Hex:            w=  %llx\n", *wx);

Software Manager & Engineer
Tel: 01223 414180
Blog: http://www.ivimey.org/blog
LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/