I've been posting articles here on dev.to recently, documenting my experiences while testing small and large language models. I'm an engineering te...
For further actions, you may consider blocking this person and/or reporting abuse
Thank you, thank you, THANK YOU! Such a key distinction.
I've been wrestling with this, and your article has provided the answer I was seeking. I literally have a Lucidchart diagram with a database symbol representing a model next to a big question mark, i.e. what the heck goes here?! Something must, but I didn't have the vocabulary.
"Inference Engine" made my freaking week. THANK YOU!!!
I'd certainly welcome more like this if you have more insights to share.
" . . . a Lucidchart diagram with a database symbol representing a model next to a big question mark, i.e. what the heck goes here?!" Haha - thanks, Doug! I was hoping the article wasn't just about language - ie - definitions and semantics and would actually reveal something useful about what's going on low-level. Researching it taught me a lot as well - it's a new technology, after all.
It certainly is. Your explanation really helped.
Good article. I could argue that a more accurate classification is that SLM/LLMs are not standalone software but perhaps represent a parameterized algorithm.
As you mentioned, models are indeed versioned, distributed, cached, deployed and rolled back - much like database records, documents, and other assets. Further, models are also written (whether the writing is performed by generation from another program), and are tested. You can also argue that as the model or matrix represents weights, these can also be represented or translated to decisions in a tree or in a graph, even a flowchart.
In that sense, you can argue that models are possibly a representation of an algorithm...and therein lies the difference. An algorithm becomes standalone software when implemented as a complete and executable set of instructions and/or declarations in a particular language.
Finally, in the context of value as part of a software system or architecture, it could be argued that models as components of said system can be as valuable, or perhaps more valuable, than the workers/runners/executors of the model.
Thanks. I agree that an LLM can be described as a a trained parameter set for a fixed algorithm, though not an implementation of one. The executable algorithm lives entirely in the runtime; the model merely supplies coefficients. The inference engine multiplies inputs by weights, adds biases, and applies simple functions like squashing or clipping values. No decisions are made by the model itself; all decisions about order, loops, and execution come from the engine. At higher levels of abstraction, both can feel like “logic” because changing weights changes behavior, much like changing code does. Still, from a systems perspective, behavior encoded as data is still data.
This seems quite similar to the relatively new concept of intents in cryptocurrencies. They specify the result that a user wants, not how to achieve it. They're not software, but submitting one produces a resulting set of actions that is often not strictly deterministic.
I see what you mean - both describe a desired or learned outcome space, but neither contains the steps to make it happen. The variability and non-determinism come from the executors, not from the artifact itself.
I’m kinda with you that the weights file is basically data, and the runtime is the actual software. But in practice it feels like firmware vibes, because swapping the weights changes behavior like changing code would. Might be worth calling it a software artifact in a broader sense, but yeah, at the low level it’s not executable on its own.
I'm an engineering tech, not a software dev myself. I agree with your insight - at an operational level, swapping weights does seem to compare to swapping code or firmware, because the system’s behavior changes dramatically without touching the surrounding stack.
These new ai models are hard to define - they aren’t software in the classic executable sense, but they also aren’t passive data in the way config files or datasets are. They're in a kind of middle space - their behavior encoded numerically rather than as control flow and instructions.
"Great perspective! It’s easy to confuse AI models with software, especially since they’re used so much in coding and development. I really like how you broke down the difference between data and software. This distinction is super important for discussions around security, scalability and deployment.
Thanks, Marry!
Great Article!
branches, loops, calls, and returns
Logic programs don't have these (tough a predicate assertion is sometimes interpreted as a call). Pure SQL doesn't have these. XSLT doesn't have these. Haskell doesn't have these.
So none of the multitude of programs in these languages are software, right?
I'm an engineering tech, not a software developer so I can't really debate at this level. I do think that my article stops generating any insight as the discussion shifts from systems behavior to definitions and edge cases in language theory.
SQL is a language (Structured Query Language), not software. SQL, e.g.
SELECT * FROM ORDERS WHERE Total > 100, is interpreted and executed by a database management system (DBMS), which is software.Similarly, XSLT is a transformation language that requires a processor (software) to transform XML documents into other formats, like HTML, ASCII text, etc.
Languages are not software; they are the syntax for expressing instructions for how to create new software or execute existing software.
Any application consists of a program and data (parts frozen into the code, parts loaded). An LLM is an abbreviation for the data part, the required execution engine is implied. Sloppy terminology if you like.
I obviously meant the programs expressed with the languages I mentioned and their execution engines. Sloppy terminology if you like.
So, is this discussion about the distinction between the initial tape of a Turing Machine (not software?) or the TM executing it?
Peter - I think you're still making it about language categories and definition. My article wasn’t about whether AI models can be forced into the 'category' of software, but rather whether treating them as if they were ordinary software leads to incorrect assumptions when trying to understand and interact with them.
Conversely, a file that merely stores data, even if it is bundled with an application, does not truly meet this definition.
This is where we disagree. As I noted, part of the data is embedded in the program, other parts are in files, DBs, read as input. Going back to fundamentals... just like TM and its tape.
I agree that treating LLMs as just software is a mistake. But so is doing that with operating systems, etc. The issue is that they have such a level of complexity that while we understand the principles of how they operate, we no longer understand it in detail.
So we should leave it at that.
BTW, your "counterexamle" of the LLM file not being executable is false. On Windows any file can be run as a program by a suitable one time association with the program that runs it. On Unix-like systems properly implemented execution engines will skip a first line that contains "#! " and after chmod +x the shell will run it. Complain to your LLM engine developer that their implementation is not complete. But on Linux, even lacking that it can be done via the /proc/sys/fs/binfmt_misc/register mechanism... that's how java jar files can be run as commands by default on most Linux distributions.
Correction: The above about jar files is no longer true on modern Linux, but similar is true about .python files (without hash bang).