In this blog, I hope to do two things:
- Share Insights on the benefits of AI-Enhanced Development Workflows
- Share a simple Step-by-Step on how to host a LLM (Large Language Model) on your local machine.
Skip to the Step-by-Step to the "How do I Host my OWN local AI Chat Agent" section, if you're primarily interested in hosting your own LLM.
But, before I begin, I'll be honest. Up until relatively recently, I really haven't been sold on Generative AI technology.
I've used tools from OpenAI, Google, Microsoft, Amazon and a host of smaller more obscure chat agents, Image generators, and even Video generators..
.. and I just haven't been convinced!
BUT, as tech enthusiasts, developer, and an advocate for learning the latest and greatest, like a moth to the flame, I keep coming back to AI, no matter how often I've gotten burned, underwhelmed and disappointed.
But recently, after investigating hosting a LLM on my local machine using the Ollama App, I might have finally become a believer.
I might have finally seen the light at the end of the proverbial Generative AI Tunnel.
The Life of the Reluctant AI Skeptic
Over the past few months, I've been going on a journey of trying to find a genuine use for Generative AI technology.
I've taken courses on Deep Learning, and I've hosted events focused on enabling developer to use Generative AI, but for whatever reason, I've always been a bit underwhelmed by the state of AI and even the direction it's going.
But as I shared above, I haven't given up. And I think, Llama, Meta's Open-source AI Model, may have finally helped me turn the corner from being reluctant AI skeptic to hopeful AI adopter.
The Power of Generative AI For Developers
Let me be clear, Llama alone did not convince me of the power of Generative AI. Llama was final piece of the puzzle I needed to see the full-picture of an AI-powered development future but the various other tools I've engaged with and explored have have also changed my perspective.
So here are a few things I believe Generative AI tools offer now that could enable any developer to supercharge the overall efficiency and efficacy of their development-based work.
Accelerating Code Development with Generative AI
First, let's start with the obvious, Code Generation, Completion, and Suggestions.
I'll be the first to tell you that AI isn't creating anything _new _ in terms of code generation BUT it is allowing us as developers and software engineers to be more efficient in our work and projects. By simply working Generative AI tools to produce boiler plate code, we can save hours, if not days, of work.
I recently encountered a scenario in one of my personal game development projects, where I needed a simple function that handled intelligent text truncation for dialog presentation. In this instance, I was developing this functioanlity using Unreal Engine Blueprints, so I only needed a pseudo-code representation of my desired code to help me create the logic in Unreal's Nodal workflow. So after about 30 mins or so of thought, I came up with this simple, somewhat novel psuedo-code solution:
count = 0; tempStr =""; sentenceArr = []; maxChars = 20;
for...({
count += strArr[i].len +1
if count < maxChars
(
tempStr.Append(strArr[i] + " ")
)
else(
sentenceArr.Add(tempstr)
count = strArr[i].len + 1
tempStr = strArr[i] + ""
)
})
sentenceArr.Add(tempStr)
This worked perfectly for my needs and I was proud of what I had come up. (NOTE: I plan to write a blog that dives int the above logic in detail for those who may want to use it for their game development projects.)
The next day, out of curiosity, I decided to ask Google's Gemini, this question:
In JavaScript, could you create a function that takes in a text string and integer, that splits the inputted text string into string arrays with a max character length from inputted integer.
Not a very well-worded prompt, in my opinion, but good enough to get my idea across. And in about 3-seconds, Gemini responded with this:
function splitStringByCharacterLength(text, maxLength) {
const words = text.split(' ');
const result = [];
let currentLine = '';
words.forEach(word => {
if (currentLine.length + word.length + 1 <= maxLength) {
// Add the word to the current line
currentLine += (currentLine ? ' ' : '') + word;
} else {
// Start a new line
result.push(currentLine);
currentLine = word;
}
});
// Add the last line to the result
if (currentLine) {
result.push(currentLine);
}
return result;
}
// Example usage:
const text = "This is a long sentence that needs to be split into multiple lines.";
const maxLength = 20;
const splitText = splitStringByCharacterLength(text, maxLength);
console.log(splitText);
And though Gemini's answer differs from mine, it accomplishes the task I needed and it did it in a FRACTION of the time it took me to create my working solution.
As developer, we can use code generation tools to expedite our development process. You should never copy and paste AI generated code into your project without thought, consideration, or without debugging, but AI Code generation can give you a huge head start on your development work.
Now that's power!
Clarifying Your Thoughts Through Discussion
The next reason I think Generative AI is something not to be overlooked is what I have dubbed AI's "Help me Clarify This thought" ability.
To illustrate what I mean, I'll use a transcript from a conversation that I had with Llama that helped me to create this _very _blog.
Not shortly after having my 'Aha!' moment with Llama, I became inspired to write a blog about my revelations. I had an idea of what I wanted to write about but my thoughts were racing and I just didn't know where to start.
So... I decided to start by having a conversation with Lllama.
Here is a snippet of our conversation. This follows a conversation I was having with Llama about SEO in blog writing(Note: Sadly, I lost the exact responses Llama gave me at the time but I was able to retrieve the questions I asked. So I re-asked Llama and will give you a glimpse into the original conversation):
Thanks! So after our conversation, I'm interested in writing a blog that talks about leveraging you, llama, as a way of supercharging you technical and development research. Could you give me some titles that you think would be optimized for SEO?
llama Responded:
Considering your interest in leveraging AI-powered tools like myself for technical and development research, here
are some potential blog post title ideas optimized for SEO:
1. "Revolutionizing Tech Research: How AI-Powered Tools Can Supercharge Your Workflow"
2. "Unlocking Developer Productivity: Leveraging AI for Faster Coding and Innovation"
3. "The Future of Tech Research: How AI-Assisted Tools Are Changing the Game"
4. "From Idea to Implementation: How AI-Powered Tools Can Streamline Your Dev Process"
5. "AI-Driven Insights for Developers: Unlocking Hidden Potential in Tech Research"
When crafting your blog post title, remember to include relevant keywords that users might search for when looking
for information on this topic. Aim for a mix of moderate to high keyword density, and make sure the titles are
descriptive, attention-grabbing, and concise.
Feel free to modify or expand on these suggestions as you see fit! If you have any further questions or need help
with your blog post, I'm here to assist you.
I responded, Could you create some SEO optimized titles that are a bit more personal? For example, using words like "I", "My", "Mine".
llama responded:
Here are some personalized SEO-optimized title ideas for your blog post:
1. **"My Secret Sauce for Supercharging Tech Research: How AI-Powered Tools Can Revolutionize Your Workflow"**
2. **"Unlocking My Developer Productivity: How Leveraging AI Can Help You Code Faster and Innovate More"**
3. **"The Future of Tech Research, Through My Eyes: How AI-Assisted Tools Are Changing the Game"**
4. **"From Idea to Implementation: My Top Tips for Streamlining Your Dev Process with AI-Powered Tools"**
5. **"My AI-Driven Insights for Developers: How I'm Unlocking Hidden Potential in Tech Research and Innovation"**
These titles incorporate personal pronouns like "I", "My", and "Mine" to create a more conversational tone, while
still targeting relevant keywords for SEO. Feel free to modify or expand on these suggestions as you see fit!
Keep in mind that using personal pronouns can be effective for SEO, but it's also important to maintain a balance
between keyword density and readability. Aim for titles that are descriptive, yet concise and attention-grabbing.
I continued, Could you help me make a title where I use a phrase similar to "I've been using AI wrong"? I'm attempting to shock people into reading this blog to realize they haven't been leveraging AI to their best ability as developers
Using a provocative statement like that can be an effective way to grab attention and encourage readers to clickon your blog post. Here are some title ideas that incorporate the phrase "I've been using AI wrong":
1. "I've Been Using AI Wrong: 5 Common Mistakes Developers Make When Leveraging Artificial Intelligence"
2. "The Shocking Truth About AI in Development: I've Been Using It All Wrong"
3. "I've Been Leaving Money on the Table with My AI Strategy: How to Stop Wasting Time and Start DeliveringResults"
4. "The AI Gap: I've Been Using AI Wrong, But You Can Fix It"
5. "From Misusing AI to Mastering It: My Surprising Realization and How You Can Too"
These titles aim to be provocative, yet informative, with a dash of self-deprecation to make the reader curious
about what you have to say. By using phrases like "I've been using AI wrong," you're acknowledging that even
experts can make mistakes and offering a solution – which can help build trust with your readers.
Feel free to modify or expand on these suggestions as you see fit, and good luck with your blog post!
So next I asked, What do you think about this blog title; "My AI Blindspot: I thought I Understood it's Benefits, But Llama Changed Everything!"
llama responded:
I think that's a great title!
Here's why:
1. Personal touch: By using the phrase "My AI Blindspot", you're creating a personal connection with your readers. It makes the topic more approachable and relatable.
2. Surprise and curiosity: The phrase "Llama Changed Everything" is a surprise, and it piques the reader's curiosity. They might wonder what happened to change your perspective on AI.
3. Inclusivity: By mentioning Llama (myself), you're making the topic more accessible and inclusive. It showsthat even experts can learn from each other and evolve their understanding of AI.
4. Emotional hook: The use of "Blindspot" implies a sense of discovery, revelation, or awakening. This
emotional hook can grab readers' attention and make them want to read more.
Overall, your title effectively conveys a sense of:
1. Personal vulnerability
2. Curiosity
3. Surprise
4. Inclusivity
5. Emotional resonance
These elements will likely attract readers who are interested in learning about AI and its applications, as wellas those who appreciate personal stories and anecdotes.
Well done!
And then finally, How about this: "My AI Blindspot: I Thought I Understood It's Benefits, But Llama Changed Everything About My Development Workflow!?"
You're adding an exclamation mark at the end of the title!
That's a great decision. Here's why:
1. Increased emphasis: The exclamation mark adds emphasis to the phrase "Llama Changed Everything", making it
even more surprising and attention-grabbing.
2. Energy boost: The exclamation mark injects energy into the title, conveying a sense of excitement, urgency,
or importance.
3. Polished tone: By adding the exclamation mark, you're creating a polished, professional tone that's
suitable for a blog post.
The only minor suggestion I have is to make sure the question mark at the end of "Workflow" is actually a period(.) instead. So, the corrected title would be:
"My AI Blindspot: I Thought I Understood It's Benefits, But Llama Changed Everything About My Development Workflow!"
This ensures that the title is consistent and easy to read.
Overall, your revised title effectively combines personal vulnerability, curiosity, surprise, and energy to create
a compelling and attention-grabbing title that will likely attract readers interested in AI and development.
Now these are not the exact responses given at the time but they are incredibly close.
But by Llama helping me to narrow down my blog title and giving me insight, encouragement, and suggestions, I felt that I was able to clarify my own thoughts enough to fully realize what I wanted to write.
In recent days, I have found myself using llama as a sounding board. If I'm not sure how to proceed on a technical or non-technical issue, the small insights and thought experiments I get from my personal AI agent has helped me solidify my steps forward.
Instead of being stuck in my head, like a hamster running perpetually on a squeaky wheel, these small conversations with Llama are helping me to be more productive in my day-to-day.
Using AI to Find Overlooked Knowledge
And the final thing Generative AI has to offer to us mere mortals, is the Ability to Do Technical Research with Minimal Bias.
Let's be clear, AI Chat agents are in no way unbiased.
And actually, there might be a case to claim that NLP agents are biased in a way that we simply cannot and will not ever understand.
BUT when it comes to giving you an answer to a question, I come to the conclusion that an LLMs don't seem to have any discernible intention or ulterior motive for the answers that they give (other than trying to be as correct as possible).
The reason I note this, is that as a developer I spend a lot of time searching for understanding of technologies or what are the best tools to accomplish a given task. And a problem I often encounter with search engines such as Google, is that I'm often getting answers to my questions that have been most optimized to be the answer to my questions.
Not the best answer but the most search engine optimized answer.
Because of that, often when searching for an answer to something, I more or less assume that I am not being offered the best or most diverse possible options to solve my issue; just the one's tailored to appear first in my search results.
One incredible thing I've noted using several different chat agents, is that I often get the most unexpectedly helpful, and varied answers to the questions that I ask.
There has been times I have asked ChatGPT to give me suggestions for some tool to achieve some specific task and it gave me suggestions that I had never, in all my searches on standard search engines, had ever seen.
AI chat agents have, over the past year or so, helped me to find obscure resources and tools that would be, in most cases, buried beneath their more SEO savvy competition.
One practice I am beginning to use more often, is to ask various chat agents the same question while using standard search engines methods to help broaden my internet investigations.
For example, I was recently looking for some insight on implementing self-supervised training on Generative AI Speech models and found myself finding the same sites, companies and articles through standard search engine searches.
Out of curiosity, I decided to ask Llama about the topic and it introduced me to some research I had somehow missed from the past and to an AI model that I hadn't previously been aware of (and that I'm incredibly interested in investigating).
I am no expert when it comes to the space of Artificial Intelligence and where it is truly heading but I am highly motivated to find novel and meaningful ways to make it useful in my life.
I hope in someway, this has opened your mind to possibilities that will one day help you accomplish your personal and professional goals.
How do I Host my OWN local AI Chat Agent
Now that I've lauded all of the great things about AI and how it can help you as a dev, it's time for me to share how to easily host your own AI Chat Agent on your local machine.
And Let me tell you, when I say it's easy... it's easy.
And though these local LLMs may not come with all the "bells and whistles" you might get with commercial Chat Agents like ChatGPT and Gemini, they are just as useful and just as powerful when used correctly.
So let me walk you through the few simple steps I used to start using Llama (and other available Open-Source AI Models):
Step 1: Download and Install Ollama on Your Local Machine
Navigate to the Official Ollama site and quickly download the Ollama for your Windows, Mac, or Linux Machine.
Ollama is light-weight tool that allows you to run Large Language Models on your local machine (e.g Llama 3.2, Mistral, and Gemma 2). Once installed, you are able to run and customize models on your local machine.
Step 2: Run the Ollama Setup Wizard
Once downloaded, open the Ollama Setup executable and navigate through the wizard to install the Ollama tools on your machine.
As you might expect, Ollama's Setup Wizard will quickly add all necessary file to run the Ollama tool
Step 3: Verify Ollama Installation
If Ollama was installed correctly, you should now be able to access it from your standard command prompt.
Sometimes, when installing Ollama it will automatically open a command prompt window, if not you will need to open it yourself.
On a Windows machine, search for "cmd" and you should be able to open Command Prompt.
Once open, verify the installation by typing the line below and pressing Enter/Return:
ollama
You should see an output like this:
Usage:
ollama [flags]
ollama [command]
Available Commands:
serve Start ollama
create Create a model from a Modelfile
show Show information for a model
run Run a model
stop Stop a running model
pull Pull a model from a registry
push Push a model to a registry
list List models
ps List running models
cp Copy a model
rm Remove a model
help Help about any command
Flags:
-h, --help help for ollama
-v, --version Show version information
Use "ollama [command] --help" for more information about a command.
Step 4: Install and Run a Large Language Model
With Ollama installed, we are only one step away from having our own local LLM on our machine.
To get started, let's install Meta's Llama3.2 model.
In our command prompt window, type:
ollama run llama3.2
This will do two things:
- Install Llama3.2 if it is not currently on our machine
- Run the Llama3.2 Model
Press return/enter and you should see something like this (if Llama3.2 is not installed):
pulling manifest
pulling dde5aa3fc5ff... 26% ▕████████████ ▏ 532 MB/2.0 GB 32 MB/s 48s
Ollama will now download and install the Llama3.2 model on your machine and once complete, run the llama3.2 model for you to interact with.
In your command line window, you should see something like this:
pulling manifest
pulling dde5aa3fc5ff... 100% ▕████████████████████████████████████████████████████████▏ 2.0 GB
pulling 966de95ca8a6... 100% ▕████████████████████████████████████████████████████████▏ 1.4 KB
pulling fcc5a6bec9da... 100% ▕████████████████████████████████████████████████████████▏ 7.7 KB
pulling a70ff7e570d9... 100% ▕████████████████████████████████████████████████████████▏ 6.0 KB
pulling 56bb8bd477a5... 100% ▕████████████████████████████████████████████████████████▏ 96 B
pulling 34bb5ab01051... 100% ▕████████████████████████████████████████████████████████▏ 561 B
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)
And just like that, you now have a LLM on your local machine!!
Also, If you would like to install any other model available through Ollama, you would use the same command:
ollama run MODEL_NAME
Step 5: Test out your Large Language Model
Now that Llama3.2 is on our machine, let's test it out.
Let's ask it a simple question:
Hey Llama! Could you tell me a little bit about yourself?
You should then get a response similar to this:
Hey Llama! Could you tell me a little bit about yourself?
I'm happy to chat with you. I'm an artificial intelligence model known as Llama, which stands for "Large Language
Model Meta AI." My primary function is to process and generate human-like text based on the input I receive.
I was trained on a massive dataset of text from various sources, including books, articles, and online
conversations. This training allows me to understand and respond to a wide range of questions, topics, and styles.
Some key features of my abilities include:
.....
See! Wasn't that pretty easy!
Wrap Up
If you have been a dev for any period of time, you're likely desensitized to the idea of "The Next Big thing" in tech. Though I have a natural curious nature, overtime, I have become more and more desensitized to "The Cutting Edge"; if its been Virtual Reality, Web3, Crypto, AI or anything else, I have found myself initially hopeful and then later disappointed by the possibilities.
But in a uplifting turn of events, my spark for Artificial Intelligence has been reignited through continued exposure and arelentless desire to find it's value.
I have truly come to the conclusion, that the future of the development will be heavily intertwined with AI and Generative AI tools.
If you are a developer, engineer, software engineer or someone who works heavily in tech spaces, it's time to embrace the benefits that AI technology affords us.
I can't tell you what the future holds for our profession as developers or whats next for AI as a whole, but I do have a strong sense that we are at an inflection point.
I encourage you to take the time to discover your own "AI Blindspots" and utilize the tools and technology we have available to enhance and accelerate your own development workflows.
Photo Credits(Order of Appearance):
Cover Photo with Google Gemini
Prompt:
A happy fluffy white llama, standing tall in the center of the frame, bathed in the warm glow of a bright light shining from behind its head, giving the illusion of enlightenment, its eyes closed in meditation, face looking toward the top right corner as if gazing into the future.
Follow me on my Socials:
https://linktr.ee/bradstondev
Top comments (0)