openerp-india team mailing list archive

[Raphaël Valyi - http://www.akretion.com <882036@xxxxxxxxxxxxxxxxxx>]

Hello guys,

just a word that I find it very interesting to try to recover from the
float lost precision at the rounding stage by looking for the shortest
decimal fraction that rounds correctly back to the true binary value. I
always spotted and insisted a few years ago that even with good coding (not
to be assumed too lightly) rounding would not behave like in a decimal
system and i would make OpenERP numbers awkward at best, but possibly
illegal if some law forces you to adopt some specific rounding technique.

Going to decimal might be the solution. But If that can not be done
(probably not in 6.1), I think that re-defining a rounding function that
will round like a decimal system (assuming a limited precision) is
something interesting.

I've been surprise o see that Python 2.7 would use the same technique now.
For instance if you write 0.1 in the interpreter it will write 0.1 while it
would have written 0.10000000000000001 in previous Python versions as it
was closer to the approximated binary value for 0.1.

I've played a bit with the r2 method provided in
https://bugs.launchpad.net/openobject-server/+bug/882036/comments/19
def r2(f, p=0):
    return round(f + cmp(f, 0.0) * (.1 / pow(10, p+2)), p)
Overall it's interesting but I could spot a bug: it has a very limited
precision.
If you try r2(1.00499, 2) you get 1.01 instead of 1.00

After my very little test, you can increase the precision of it by putting
more then 2 in p+2, like:
def r2(f, p=0):
    return round(f + cmp(f, 0.0) * (.1 / pow(10, p+5)), p)

This increases the precision apparently. I didn't bother computing what
would be the maximal precision one could reach.

Now some questions:
Do one of you know if that approach is totally flawed? If yes, could you
provide some counter example that given some limited entry precision would
fail to round a 2 digits like a decimal system?

@Cloves, do you have some law text that says we should use ROUND_HALF_EVEN ?
I've always looked for that and never found anything. Even in France I
couldn't find such a text law (except for the old Franc / Euro conversion I
think; but hey it may become actual again ;-) )


Regards.


On Wed, Nov 23, 2011 at 2:52 PM, Ferdinand @ Camptocamp <
882036@xxxxxxxxxxxxxxxxxx> wrote:

> may be this highlights the problem
> http://www.bytereef.org/mpdecimal/index.html
>
> --
> You received this bug notification because you are a member of OpenERP
> Committers, which is subscribed to OpenERP Server.
> https://bugs.launchpad.net/bugs/882036
>
> Title:
>  rounding error
>
> Status in OpenERP Server:
>  New
>
> Bug description:
>  Concerns 6.0 and trunk.
>  If you define a precision of 0.01, the rounding of 0.125 must be 0.13 and
> not 0.12. The error is in the call of the format string "%.2f"%val which
> introduces a mathematical error. The round function must be called to apply
> the correct rounding before formatting the string. It should be:
>  "%.2f"%round(val*100)/100
>  Fix class digits_change of class float in the server. BUT fix also the
> gtk client AND web client as they all have that error (I let you find the
> right line)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openobject-server/+bug/882036/+subscriptions
>

-- 
You received this bug notification because you are a member of OpenERP
Indian Team, which is subscribed to OpenERP Server.
https://bugs.launchpad.net/bugs/882036

Title:
  rounding error

Status in OpenERP Server:
  New

Bug description:
  Concerns 6.0 and trunk.
  If you define a precision of 0.01, the rounding of 0.125 must be 0.13 and not 0.12. The error is in the call of the format string "%.2f"%val which introduces a mathematical error. The round function must be called to apply the correct rounding before formatting the string. It should be:
  "%.2f"%round(val*100)/100
  Fix class digits_change of class float in the server. BUT fix also the gtk client AND web client as they all have that error (I let you find the right line)

To manage notifications about this bug go to:
https://bugs.launchpad.net/openobject-server/+bug/882036/+subscriptions