DEV Community

David Cantrell
David Cantrell

Posted on

3 1

I Hate Floating Point!

In one of my projects, I need to know what is the minimum number of bytes needed to store a given integer. The Obviously Correct answer, ignoring negatives and the special case of 0, is something like ...

1 + int(log($data) / log(256))
Enter fullscreen mode Exit fullscreen mode

I'm lucky that my test suite included a value of $data that is an exact multiple of 256 (which is likely to expose floating point errors in that calculation), and doubly lucky that the particular value in the test suite had a floating point error in the right direction so that the test actually failed, because ...

$ perl-5.30.2/bin/perl -E 'say 1 + int(log(0x1000000) / log(256))'
4
$ perl-5.30.2-quadmath/bin/perl -E 'say 1 + int(log(0x1000000) / log(256))'
3
Enter fullscreen mode Exit fullscreen mode

Ouch! The first does the calculation using a 64 bit IEEE754 float, the second uses gcc's libquadmath.

My thanks to the CPAN-testers for making my tests fail!

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read full post →

Top comments (0)

Image of Docusign

🛠️ Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more