DEV Community

David Cantrell
David Cantrell

Posted on

3 1

I Hate Floating Point!

In one of my projects, I need to know what is the minimum number of bytes needed to store a given integer. The Obviously Correct answer, ignoring negatives and the special case of 0, is something like ...

1 + int(log($data) / log(256))
Enter fullscreen mode Exit fullscreen mode

I'm lucky that my test suite included a value of $data that is an exact multiple of 256 (which is likely to expose floating point errors in that calculation), and doubly lucky that the particular value in the test suite had a floating point error in the right direction so that the test actually failed, because ...

$ perl-5.30.2/bin/perl -E 'say 1 + int(log(0x1000000) / log(256))'
4
$ perl-5.30.2-quadmath/bin/perl -E 'say 1 + int(log(0x1000000) / log(256))'
3
Enter fullscreen mode Exit fullscreen mode

Ouch! The first does the calculation using a 64 bit IEEE754 float, the second uses gcc's libquadmath.

My thanks to the CPAN-testers for making my tests fail!

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs