Let's say you write some code to read an image from a file. Better still you find a library that does that for you. You write some minimal wrapper code to call it. Now you have code that reads and writes that image format from a file. All seems well for now.
Later you need to transfer the image to another service that may not be on the same machine. Now you need to either send the image over the network, or write it out to a file, with your already in-use library, and then send the file.
The choice is obvious because you already have the library and know how to use it. You're going to write the modified image out to the slow, maybe also remote, file system and then read it all back again to transfer it over the network, wrapped in a protocol like FTP, to be unwrapped at the other side and written back to the file system, because that's what FTP does, read in again on the remote machine and voila, you've now got a product 100x slower than it needs to be.
So where did this go wrong? The decision to use the library was rational. The decision to reuse the library was probably even more so. You didn't make the first error, the library author did. They provided you a library to deal with a particular image format and in order to test it themselves they obviously had to write code to read and write their input and output to files. So they bundled that code into the library and made it their API. Now it's easy to use. So that's a good thing, right?
Wrong. The library author may be the worlds greatest expert on compressed image formats but files are just files, except when they aren't. They probably used the standard library API. Maybe shunted everything into a string stream and wrote it in one hit. Maybe they used OS specific file APIs and tied you to a particular OS for no reason. It works for demos but where they provided you with every nuance and configuration of image compression settings and fine grained control over your encoding and decoding, did they do the same with the File I/O?
Is it portable? Maybe.
Can you opt to use I/O uring on Linux? Unlikely.
Can you opt to read and write asynchronously? Almost certainly not.
Can you write to a network socket? Not a chance.
Now you may be able to unpick their file IO from their serializer, if it's really well written, and perhaps bodge in your own to send the images directly over a socket or as the body of an HTTP response.
In reality that's likely going to mean understanding almost all the library code, maybe forking it at source and by then you may as well have written it yourself. There may be compromise solutions with memory mapped files or domain sockets that with exactly the right incantation, give you enough performance.
However none of this is actually necessary. If the library author had left their file IO code in their test harness, where it belongs, and provided serialization as an interface you'd easily be able to adapt that interface to your networking needs or your existing way of interfacing with files. The library also would be smaller, more focussed and real world performance would be under your control.
The key to genuinely reusable code is real separation of concerns. If a library is in fact a codec to take data from format A to format B, then that's exactly what it should be. The storage medium for that data shouldn't matter. In fact where the data lives before it's encoded and where it lives afterwards are entirely none of the business of such a codec. It shouldn't even care about or need to do memory allocation. That's a separate concern. It should expose an interface where you provide the input data and one where you can get the encoded/decoded data back. Such a codec gains portability, reusability and simplicity while loosing nothing of value. It may not seem initially as easy to use because you'll have provide more external infrastructure but you'll be glad you chose it as your project grows.
With single purpose code libraries, less really is more. Much more. Library authors can work smarter by not coupling their implementations to particular systems or storage.
As consumers we need to be savvy about which libraries we choose. The easiest to get up and running isn't always the best choice. In fact it's often a fish with the hook left in.
Top comments (0)