Can we use LLMs to drum up some instant nostalgia? And can cloud deployments help us get there?
I recently had an itch to experiment with some emerging AI tools and thought it would be fun to write my own client to chat with one of the ubiquitous large language models. For nostalgia’s sake, I could make it look like a throwback, Windows 95-style messaging app. The original concept was really nothing more than a retro-styled skin for ChatGPT. I’d get my hands dirty with some API integration, and I also viewed the project as a cost-effective way to experiment with GPT-4 without shelling out for ChatGPT Plus.
I couldn’t just leave it at that, though. As I noodled on the implementation and thought about the sheer entertainment potential of today’s LLMs, I wanted to give the app some pizazz. Instead of a vanilla “helpful assistant,” perhaps it would be more engaging if I gave the bot a randomized persona, someone relatable that a lot of users would recognize.
LOL Instant Messenger, a nostalgic chat experience from the early days of the interwebs.
I started to tinker with a bit of prompt engineering to see if I could convince the “helpful assistant” to stay in character.
In one of my very first interactions with an early prototype of my brand new ’90s-flavored bot, I chatted with a virtual Steve Urkel, complete with his nerdy “Did I do that?” charm.
I found myself laughing a little too loudly at my desk late at night, and with that, LOL Instant Messenger was born.
Hosting Plan
My design called for a static Angular-based client that could sit on any old hosting provider. The API layer makes sense as a serverless collection of functions, and I also wanted a secrets manager to house my API key. Amazon Web Services (AWS) is a logical choice, but then again, so is Microsoft Azure or Google Cloud.
Choosing a provider can be a daunting task in itself, and the repercussions of the decision can propagate for years. Predicting growth, planning for shifting resource needs and anticipating changing costs is a high-stakes exercise that evokes a sense of gazing into a crystal ball.
One way to mitigate the Cone of Uncertainty is to postpone making a decision as long as is feasible. What if we could defer the cloud vendor choice until late in the product life cycle? Better still, what if we had a low-cost, low-risk method for switching vendors even after launch? An abstracted design pattern up front can provide flexibility later in the product life cycle.
I had recently read about the Infrastructure as Code engine Pulumi, as well as an Infrastructure from Code framework called Nitric. I liked the idea of focusing on the domain logic for my application and delegating the provisioning and management of cloud resources to someone else.
Helpfully, Nitric also ships with a local development server that emulates cloud services like serverless functions and secret management, so I was able to noodle around on my laptop with its API and iterate rapidly. In relatively short order, I had an MVP (minimum viable product) version of LOL IM ready to push to AWS and share with the world.
After a bit of paint-by-numbers config, Nitric deploys to the cloud with a single command: nitric up. This kicks off containerization of a provider-specific implementation of the business logic. The Nitric SDK handles provisioning of cloud resources based on app requirements; in the case of LOL Instant Messenger, this includes a set of serverless functions fronted by an API gateway and a secrets manager to handle the API key for LLM connectivity:
It’s alive! It’s ALIVE!
Switching It Up
There are a handful of reasons I’d like the option to easily migrate from AWS to Azure or GCP. At minimum, I want to compare performance and overall customer experience among the various platforms. Also, as the big-name providers jockey for position and market share, AI capabilities are emerging within each ecosystem. As new, more capable models become available, it might make sense to pivot away from OpenAI in favor of a provider’s native capability. And if I am making the move to a different LLM, I may as well migrate the service itself to the same platform.
Traditionally, switching providers would require some amount of redesign and/or redevelopment to support the new deployment, which of course comes at a cost. With Nitric, this is a straightforward task that only involves a bit of configuration.
Moving Day
The process of setting up deployment to another provider is literally two commands: nitric stack new followed by nitric up. Well, that and proper onboarding to the provider itself. Someone who is familiar with Azure probably already has the CLI installed and at least created a subscription, thus saving the frustration of my first hurdle:
Why is my deployment failing? Ohhhh, right, I haven’t offered any payment method to Microsoft yet. As my preschooler would say, “That’s a whoopsie-doozy.”
In my case, after completing the Azure setup, I experienced another unexpected bump in the road. Deployment failed a few times due to “MissingApiVersionParameter” on Azure deploy. I triple-checked to ensure I wasn’t making another humiliating rookie mistake, and then posted the error message on Nitric’s Discord chat. Eleven minutes later, I had a working solution: I needed a specific version of the Azure plugin for Pulumi, and once this was adjusted, the deployment went off without a hitch. 🎉
So … Did It Work?
Mission accomplished? Almost. The final sticking point was CORS (cross-origin resource sharing) configuration. I had made the required changes in AWS after the original deployment and needed to achieve the same result in Azure. This required a bit of rooting around in Azure Portal and asking ChatGPT and Dr. Google for help. There’s a Nitric roadmap item for simplifying CORS, which will truly be a luxury once it rolls out.
A simple update to the client and a peek at the network traffic confirms what we already knew. The API is now being hosted on Azure, and is behaving identically to the AWS version. I’m pleasantly surprised that I really was able to keep my codebase completely unchanged and pivot from one provider to the next. I don’t have to become an expert in the idiosyncrasies of Microsoft’s offering, just as I’m not an AWS guru. And, I can use the nitric down command to clean up Azure resources if I decide to permanently discontinue the experiment.
Checking out our new digs by chatting with a familiar figure.
Wrapping Up
Thinking ahead to what’s next for LOL IM, I’d like to try some other models. Mistral, Gemini and others provide APIs very similar to OpenAI’s offering, and it should be fairly trivial to swap in a different LLM while keeping the same prompting techniques intact. I’d like to explore some of the multimodal capabilities of these models as well; maybe when starting a conversation with a “buddy,” we could generate a profile photo or avatar that matches the vibe.
I also plan to add GCP to the cloud provider mix, as I see tremendous value in being able to seamlessly migrate from one cloud provider to another. Using an Infrastructure from Code framework avoids tight coupling to a specific vendor and keeps the details of cloud compatibility abstracted away from business logic. This separation of concerns is extremely powerful, letting developers be truly cloud-agnostic when delivering features. Consider giving Nitric a try in your next project, and be sure to share your experience!
Note: This article originally appeared on TheNewStack.io
Feeling Generous?
Thanks for reading! If you found something useful or entertaining here, I'd appreciate your support to keep writing. How about a coffee to fuel the process? https://www.buymeacoffee.com/joshknopp
Top comments (0)