A quick note before you read: I wrote this in 2023, and life got so busy that I didn't open this blog again until today. I came back to publish something about AI and coding teams, and the first thing I saw was this unpublished draft.
I'm sharing it because of how extraordinarily relevant it still is. It's something my team at WhaleHawk is actively combating. Yes, we need to be responsible. No, most of us aren't -- and it's mind-blowing that this problem is still with us. We're leaving it up to our tools to be safe, yet we're still the ones operating the controls. It's an incredibly beautiful dance we find ourselves in, and I wanted to share it.
We are in a tech turning-point in history: from one day to the next, AI has changed the game in most every software market. OpenAI's Chat software reached 100 million users within 2 months of deployment. This same audience base took Twitter (now X) 2 years to achieve. AI is everywhere and there is a race to integrate it into every aspect of software on the planet, and I can see why. Its potential to help humanity and the planet is tremendous: consider Aza Raskin's work to use AI to understand the language of animals, or using it to develop a way to eliminate micro-plastics from the ocean (Ocean Cleanup is doing a great job with Macro-Plastics, anyone know of micro-plastic initiatives?). While we are enthusiastic, it's crucial to remember this tech is new to public access. We are in uncharted territory and it is wildly exciting. However, this also means we must proceed with caution.
Don't ask me, I just work here.
I am a software developer, and have used APIs for diffusion models and LLMs to integrate AI into my apps. The process is relatively easy. I do not have formal training in math or data analysis, yet just the other day I built out a computer vision model. It took less than 10 minutes.
I am telling you this to illustrate the ease of application for this mind-blowing technology.
This blog post is mostly opinionated, and addresses one big issue I have encountered with AI deployment: confidently wrong answers. There is an ocean of safety concerns surrounding AI, but to get the conversation rolling I chose this as an entry point.
This post is for anyone in the industry to contribute to, and I want your comments. I see industry professionals and academic researchers delving into the safety and ethics of AI. Simultaneously, developers are being asked to integrate AI without a clear roadmap. I believe there should be a stronger partnership between these groups to improve AI deployment in product development.
AI isn't new.
According to Tristan Harris and Aza Raskin of the Center for Humane Technology, our first contact with AI at massive scale was social media feed algorithms optimized for engagement. LLMs have been in the works for years, and have finally reached the point of mass deployment. Hallucinations are a known issue. Although there have been improvements to decrease their occurrence, it is in the nature of the tech to happen. An LLM doesn't look up answers; it predicts the most likely next word, one token at a time, based on patterns learned in training. Most likely isn't always correct, and because each word builds on the ones before it, one wrong turn can carry through the rest of the answer.
The AI Race
Hallucinations are a fact, but for much of the industry, the excitement and potential of the tech seem to outweigh this downside. We know of the Big Dogs who have put a ton of time, money and compute power into developing LLMs: OpenAI and Google, to name two closed-source for-profit companies. You can use their API to interact with their models. Open-source platforms like Hugging Face host models (think Falcon, from the Technology Innovation Institute), and their aim is to democratize the tech so everyone has access.
The point is, the race of the Tech Giants continues. And the race of start-ups that rely on their LLMs is on, too.
What does that mean for us as software developers?
Based on how easy it is to use this tech, I can imagine developers will be asked to start using LLMs in their development process. I just spoke with a friend who is a software engineer, who is working on a project within his fintech company using external APIs to incorporate AI into payroll. I anticipate this becoming a standard request for developers. Upon hearing such news, I find myself astounded. As software engineers, we know that our job requires learning new tech quickly and often, but this is next level. To paraphrase Giada Pistilli, Principal Ethicist at Hugging Face, we are in the Wild Wild West of deployment and are being asked to do something that has never been done before--ever.
SOOO....what IS the problem?
My concern is not with the technical abilities of software engineers and developers: we pride ourselves on quick and efficient technical mastery. My concern is with the ethics: example, say we have successfully integrated AI into our fintech company to handle all things payroll and the product is deployed: how do we account for the hallucinations that will inevitably happen? Wrong answers, wrong tax reporting, wrong employee payout. These presumably will be anomalies, but one bad hiccup is enough to put a company under and cause an individual an unnecessary amount of stress.
That example was for a very black-and-white type of situation. What about the startups looking to use AI in the form of a chatbot therapist (at Boulder Startup Week 2023 I observed a few startups interested in this)? Now we are getting into the land of unclear distinctions. As we know, people are not numbers and are very complicated. What if a chatbot has a hallucination in the face of a human with a very real need for dependable information? This could cause devastating effects.
Accounting for error: a three-tier step
Level One: Big Tech Accountability
This one is giant. We want to trust the products they make and deploy them to stay competitive, and give the customers our company serves the best and latest tech. But first and foremost -- always, always do your research. For example: how does XYZ company who holds the rights to the LLM handle accountability? Are they projecting a big change in the future which might affect your software? Read how they are handling hallucinations and user unpredictability.
Level Two: Startup Accountability
So your startup has decided to use AI? Great! What are the safeguards in place to protect your company when a hallucination goes unnoticed and is used in exactly the wrong way?
Level Three: Personal Accountability
As consumers of generative AI content, it is up to us to decide if we will trust the tech or do our own research. This is tough, because we are buying the software to make a process easier, and we do not expect to need to babysit something that is guaranteed to work. But for self-preservation's sake, perhaps we should. Checking our own records is always important, and reaching out to a human for verification when we come across a fishy piece of data is important too. Informing your customers of what to watch out for and promoting awareness is practicing good transparency.
Solutions for Developers
While I may not have definitive answers for us, I have questions that I hope can get the ball rolling in this conversation. In this spirit, let's share ideas for how to safely deploy AI tech, to keep consumers and companies alike in the safe zone.
First, it is important to recognize that developers are not the only players in the vast field of AI deployment. Aside from the coding and direct tech development, there are other critical stakeholders involved: project managers, the CTOs, the business stakeholders, the quality assurance testers, devOps team, and users providing feedback as a last line of defense. Some companies have more, and some have less. The point is, these roles together create a holistic framework in which responsibility for the success and ethical use of AI is distributed. As a developer, you have friends in the business who all have the same goal as you.
In the case where all these filters fail, and as software developer you are faced with some particularly fishy implementation of AI: is it above your pay grade to question? As developers, what is our level of responsibility to human ethics in this mad dash to AI? Creating software is a giant job, and already we juggle a lot. Practically speaking, do we have the time and energy and knowledge to be aware of the potential pitfalls of the AI software we create?
Thank you for your interest in this topic. As a developer deploying AI software or an individual in the field of AI Ethics, I would love to hear from you. How should a startup deploy AI Software in the context of hallucination accountability?
What I'd say now (2026)
Three years later, hallucination looks different. Models still invent a line of code or a detail in a summary, but the labs have grown up, the companies building frontier models have grown up, and the harnesses around them have gotten incredibly good.
The safety issue hasn't gone away -- it's moved to where this post was pointing all along: how we use the tool. That still puts the developer, and the person using the tool, in the seat.
Think of a chop saw. You don't put your hand under the blade, push it down, and expect to be okay. An LLM agent is a coding tool, a developer on your team, and a thought partner in design, and even three years in, we're still figuring out how to use it well. How do we protect our codebase? How do we protect our product? I live in the applied world, and from where I sit, guardrails and knowing how to use this tool matter more than ever. (Here's how my team handles it: Engineers Decide, Agents Execute.)
Top comments (0)