openerp-expert-framework team mailing list archive

[Xavier Morel <xmo@xxxxxxxxxxx>]

On 2010-08-18, at 16:53 , Raphaël Valyi wrote:
> BTW, Ruby get the result right always using Ruby Floats, not sure how they
> differ.

Different output formatting. Floats are floats are floats (unless they're doubles) and can't store `0.1005` exactly, but they way they're printed can change.

Take the expression above in a Python console and compare its repr and its str.

As for Ruby, check the value it yields if you print it with a precision of 17 or more:

>> sprintf( "%.17f", v)
=> "0.10049999999999995"
>> sprintf( "%.20f", v)
=> "0.10049999999999995048"

With a precision of 16 or less, the rounding will yield `0.1005[a bunch of zeroes]`. Python exhibits the exact same behavior:

>>> "{:.16}".format(v)
'0.1005000000000000'
>>> "{:.17}".format(v)
'0.10049999999999995'
>>> "{:.20}".format(v)
'0.10049999999999995048'

Python's repr just happens to have a default precision of 17.