In this blog, I hope to do two things:
- Share Insights on the benefits of AI-Enhanced Development Workflows
- Share a simple Step-by-Step on how to host a LLM (Large Language Model) on your local machine.
If you're primarily interested in hosting your own LLM and not my insights on the topic, you can check out this blog to get started with Meta's Llama LLM.
But, before I begin, I'll be honest. Up until relatively recently, I really haven't been sold on Generative AI technology.
I've used tools from OpenAI, Google, Microsoft, Amazon and a host of smaller more obscure chat agents, Image generators, and even Video generators..
.. and I just haven't been convinced!
BUT, as tech enthusiasts, developer, and an advocate for learning the latest and greatest, like a moth to the flame, I keep coming back to AI, no matter how often I've gotten burned, underwhelmed and disappointed.
But recently, after investigating hosting a LLM on my local machine using the Ollama App, I might have finally become a believer.
I might have finally seen the light at the end of the proverbial Generative AI Tunnel.
The Life of the Reluctant AI Skeptic
Over the past few months, I've been going on a journey of trying to find a genuine use for Generative AI technology.
I've taken courses on Deep Learning, and I've hosted events focused on enabling developer to use Generative AI, but for whatever reason, I've always been a bit underwhelmed by the state of AI and even the direction it's going.
But as I shared above, I haven't given up. And I think, Llama, Meta's Open-source AI Model, may have finally helped me turn the corner from being reluctant AI skeptic to hopeful AI adopter.
The Power of Generative AI For Developers
Let me be clear, Llama alone did not convince me of the power of Generative AI. Llama was final piece of the puzzle I needed to see the full-picture of an AI-powered development future but the various other tools I've engaged with and explored have have also changed my perspective.
So here are a few things I believe Generative AI tools offer now that could enable any developer to supercharge the overall efficiency and efficacy of their development-based work.
Accelerating Code Development with Generative AI
First, let's start with the obvious, Code Generation, Completion, and Suggestions.
I'll be the first to tell you that AI isn't creating anything _new _ in terms of code generation BUT it is allowing us as developers and software engineers to be more efficient in our work and projects. By simply working Generative AI tools to produce boiler plate code, we can save hours, if not days, of work.
I recently encountered a scenario in one of my personal game development projects, where I needed a simple function that handled intelligent text truncation for dialog presentation. In this instance, I was developing this functioanlity using Unreal Engine Blueprints, so I only needed a pseudo-code representation of my desired code to help me create the logic in Unreal's Nodal workflow. So after about 30 mins or so of thought, I came up with this simple, somewhat novel psuedo-code solution:
count = 0; tempStr =""; sentenceArr = []; maxChars = 20;
for...({
count += strArr[i].len +1
if count < maxChars
(
tempStr.Append(strArr[i] + " ")
)
else(
sentenceArr.Add(tempstr)
count = strArr[i].len + 1
tempStr = strArr[i] + ""
)
})
sentenceArr.Add(tempStr)
This worked perfectly for my needs and I was proud of what I had come up. (NOTE: I plan to write a blog that dives int the above logic in detail for those who may want to use it for their game development projects.)
The next day, out of curiosity, I decided to ask Google's Gemini, this question:
In JavaScript, could you create a function that takes in a text string and integer, that splits the inputted text string into string arrays with a max character length from inputted integer.
Not a very well-worded prompt, in my opinion, but good enough to get my idea across. And in about 3-seconds, Gemini responded with this:
function splitStringByCharacterLength(text, maxLength) {
const words = text.split(' ');
const result = [];
let currentLine = '';
words.forEach(word => {
if (currentLine.length + word.length + 1 <= maxLength) {
// Add the word to the current line
currentLine += (currentLine ? ' ' : '') + word;
} else {
// Start a new line
result.push(currentLine);
currentLine = word;
}
});
// Add the last line to the result
if (currentLine) {
result.push(currentLine);
}
return result;
}
// Example usage:
const text = "This is a long sentence that needs to be split into multiple lines.";
const maxLength = 20;
const splitText = splitStringByCharacterLength(text, maxLength);
console.log(splitText);
And though Gemini's answer differs from mine, it accomplishes the task I needed and it did it in a FRACTION of the time it took me to create my working solution.
As developer, we can use code generation tools to expedite our development process. You should never copy and paste AI generated code into your project without thought, consideration, or without debugging, but AI Code generation can give you a huge head start on your development work.
Now that's power!
Clarifying Your Thoughts Through Discussion
The next reason I think Generative AI is something not to be overlooked is what I have dubbed AI's "Help me Clarify This thought" ability.
To illustrate what I mean, I'll use a transcript from a conversation that I had with Llama that helped me to create this _very _blog.
Not shortly after having my 'Aha!' moment with Llama, I became inspired to write a blog about my revelations. I had an idea of what I wanted to write about but my thoughts were racing and I just didn't know where to start.
So... I decided to start by having a conversation with Lllama.
Here is a snippet of our conversation. This follows a conversation I was having with Llama about SEO in blog writing(Note: Sadly, I lost the exact responses Llama gave me at the time but I was able to retrieve the questions I asked. So I re-asked Llama and will give you a glimpse into the original conversation):
Thanks! So after our conversation, I'm interested in writing a blog that talks about leveraging you, llama, as a way of supercharging you technical and development research. Could you give me some titles that you think would be optimized for SEO?
llama Responded:
Considering your interest in leveraging AI-powered tools like myself for technical and development research, here
are some potential blog post title ideas optimized for SEO:
1. "Revolutionizing Tech Research: How AI-Powered Tools Can Supercharge Your Workflow"
2. "Unlocking Developer Productivity: Leveraging AI for Faster Coding and Innovation"
3. "The Future of Tech Research: How AI-Assisted Tools Are Changing the Game"
4. "From Idea to Implementation: How AI-Powered Tools Can Streamline Your Dev Process"
5. "AI-Driven Insights for Developers: Unlocking Hidden Potential in Tech Research"
When crafting your blog post title, remember to include relevant keywords that users might search for when looking
for information on this topic. Aim for a mix of moderate to high keyword density, and make sure the titles are
descriptive, attention-grabbing, and concise.
Feel free to modify or expand on these suggestions as you see fit! If you have any further questions or need help
with your blog post, I'm here to assist you.
I responded, Could you create some SEO optimized titles that are a bit more personal? For example, using words like "I", "My", "Mine".
llama responded:
Here are some personalized SEO-optimized title ideas for your blog post:
1. **"My Secret Sauce for Supercharging Tech Research: How AI-Powered Tools Can Revolutionize Your Workflow"**
2. **"Unlocking My Developer Productivity: How Leveraging AI Can Help You Code Faster and Innovate More"**
3. **"The Future of Tech Research, Through My Eyes: How AI-Assisted Tools Are Changing the Game"**
4. **"From Idea to Implementation: My Top Tips for Streamlining Your Dev Process with AI-Powered Tools"**
5. **"My AI-Driven Insights for Developers: How I'm Unlocking Hidden Potential in Tech Research and Innovation"**
These titles incorporate personal pronouns like "I", "My", and "Mine" to create a more conversational tone, while
still targeting relevant keywords for SEO. Feel free to modify or expand on these suggestions as you see fit!
Keep in mind that using personal pronouns can be effective for SEO, but it's also important to maintain a balance
between keyword density and readability. Aim for titles that are descriptive, yet concise and attention-grabbing.
I continued, Could you help me make a title where I use a phrase similar to "I've been using AI wrong"? I'm attempting to shock people into reading this blog to realize they haven't been leveraging AI to their best ability as developers
Using a provocative statement like that can be an effective way to grab attention and encourage readers to clickon your blog post. Here are some title ideas that incorporate the phrase "I've been using AI wrong":
1. "I've Been Using AI Wrong: 5 Common Mistakes Developers Make When Leveraging Artificial Intelligence"
2. "The Shocking Truth About AI in Development: I've Been Using It All Wrong"
3. "I've Been Leaving Money on the Table with My AI Strategy: How to Stop Wasting Time and Start DeliveringResults"
4. "The AI Gap: I've Been Using AI Wrong, But You Can Fix It"
5. "From Misusing AI to Mastering It: My Surprising Realization and How You Can Too"
These titles aim to be provocative, yet informative, with a dash of self-deprecation to make the reader curious
about what you have to say. By using phrases like "I've been using AI wrong," you're acknowledging that even
experts can make mistakes and offering a solution – which can help build trust with your readers.
Feel free to modify or expand on these suggestions as you see fit, and good luck with your blog post!
So next I asked, What do you think about this blog title; "My AI Blindspot: I thought I Understood it's Benefits, But Llama Changed Everything!"
llama responded:
I think that's a great title!
Here's why:
1. Personal touch: By using the phrase "My AI Blindspot", you're creating a personal connection with your readers. It makes the topic more approachable and relatable.
2. Surprise and curiosity: The phrase "Llama Changed Everything" is a surprise, and it piques the reader's curiosity. They might wonder what happened to change your perspective on AI.
3. Inclusivity: By mentioning Llama (myself), you're making the topic more accessible and inclusive. It showsthat even experts can learn from each other and evolve their understanding of AI.
4. Emotional hook: The use of "Blindspot" implies a sense of discovery, revelation, or awakening. This
emotional hook can grab readers' attention and make them want to read more.
Overall, your title effectively conveys a sense of:
1. Personal vulnerability
2. Curiosity
3. Surprise
4. Inclusivity
5. Emotional resonance
These elements will likely attract readers who are interested in learning about AI and its applications, as wellas those who appreciate personal stories and anecdotes.
Well done!
And then finally, How about this: "My AI Blindspot: I Thought I Understood It's Benefits, But Llama Changed Everything About My Development Workflow!?"
You're adding an exclamation mark at the end of the title!
That's a great decision. Here's why:
1. Increased emphasis: The exclamation mark adds emphasis to the phrase "Llama Changed Everything", making it
even more surprising and attention-grabbing.
2. Energy boost: The exclamation mark injects energy into the title, conveying a sense of excitement, urgency,
or importance.
3. Polished tone: By adding the exclamation mark, you're creating a polished, professional tone that's
suitable for a blog post.
The only minor suggestion I have is to make sure the question mark at the end of "Workflow" is actually a period(.) instead. So, the corrected title would be:
"My AI Blindspot: I Thought I Understood It's Benefits, But Llama Changed Everything About My Development Workflow!"
This ensures that the title is consistent and easy to read.
Overall, your revised title effectively combines personal vulnerability, curiosity, surprise, and energy to create
a compelling and attention-grabbing title that will likely attract readers interested in AI and development.
Now these are not the exact responses given at the time but they are incredibly close.
But by Llama helping me to narrow down my blog title and giving me insight, encouragement, and suggestions, I felt that I was able to clarify my own thoughts enough to fully realize what I wanted to write.
In recent days, I have found myself using llama as a sounding board. If I'm not sure how to proceed on a technical or non-technical issue, the small insights and thought experiments I get from my personal AI agent has helped me solidify my steps forward.
Instead of being stuck in my head, like a hamster running perpetually on a squeaky wheel, these small conversations with Llama are helping me to be more productive in my day-to-day.
Using AI to Find Overlooked Knowledge
And the final thing Generative AI has to offer to us mere mortals, is the Ability to Do Technical Research with Minimal Bias.
Let's be clear, AI Chat agents are in no way unbiased.
And actually, there might be a case to claim that NLP agents are biased in a way that we simply cannot and will not ever understand.
BUT when it comes to giving you an answer to a question, I come to the conclusion that an LLMs don't seem to have any discernible intention or ulterior motive for the answers that they give (other than trying to be as correct as possible).
The reason I note this, is that as a developer I spend a lot of time searching for understanding of technologies or what are the best tools to accomplish a given task. And a problem I often encounter with search engines such as Google, is that I'm often getting answers to my questions that have been most optimized to be the answer to my questions.
Not the best answer but the most search engine optimized answer.
Because of that, often when searching for an answer to something, I more or less assume that I am not being offered the best or most diverse possible options to solve my issue; just the one's tailored to appear first in my search results.
One incredible thing I've noted using several different chat agents, is that I often get the most unexpectedly helpful, and varied answers to the questions that I ask.
There has been times I have asked ChatGPT to give me suggestions for some tool to achieve some specific task and it gave me suggestions that I had never, in all my searches on standard search engines, had ever seen.
AI chat agents have, over the past year or so, helped me to find obscure resources and tools that would be, in most cases, buried beneath their more SEO savvy competition.
One practice I am beginning to use more often, is to ask various chat agents the same question while using standard search engines methods to help broaden my internet investigations.
For example, I was recently looking for some insight on implementing self-supervised training on Generative AI Speech models and found myself finding the same sites, companies and articles through standard search engine searches.
Out of curiosity, I decided to ask Llama about the topic and it introduced me to some research I had somehow missed from the past and to an AI model that I hadn't previously been aware of (and that I'm incredibly interested in investigating).
I am no expert when it comes to the space of Artificial Intelligence and where it is truly heading but I am highly motivated to find novel and meaningful ways to make it useful in my life.
I hope in someway, this has opened your mind to possibilities that will one day help you accomplish your personal and professional goals.
How do I Host my OWN local AI Chat Agent
Now that I've lauded all of the great things about AI and how it can help you as a dev, it's time for me to share how to easily host your own AI Chat Agent on your local machine.
And Let me tell you, when I say it's easy... it's easy.
And though these local LLMs may not come with all the "bells and whistles" you might get with commercial Chat Agents like ChatGPT and Gemini, they are just as useful and just as powerful when used correctly.
So let me walk you through the few simple steps I used to start using Llama (and other available Open-Source AI Models):
Step 1: Download and Install Ollama on Your Local Machine
Navigate to the Official Ollama site and quickly download the Ollama for your Windows, Mac, or Linux Machine.
Ollama is light-weight tool that allows you to run Large Language Models on your local machine (e.g Llama 3.2, Mistral, and Gemma 2). Once installed, you are able to run and customize models on your local machine.
Step 2: Run the Ollama Setup Wizard
Once downloaded, open the Ollama Setup executable and navigate through the wizard to install the Ollama tools on your machine.
As you might expect, Ollama's Setup Wizard will quickly add all necessary file to run the Ollama tool
Step 3: Verify Ollama Installation
If Ollama was installed correctly, you should now be able to access it from your standard command prompt.
Sometimes, when installing Ollama it will automatically open a command prompt window, if not you will need to open it yourself.
On a Windows machine, search for "cmd" and you should be able to open Command Prompt.
Once open, verify the installation by typing the line below and pressing Enter/Return:
ollama
You should see an output like this:
Usage:
ollama [flags]
ollama [command]
Available Commands:
serve Start ollama
create Create a model from a Modelfile
show Show information for a model
run Run a model
stop Stop a running model
pull Pull a model from a registry
push Push a model to a registry
list List models
ps List running models
cp Copy a model
rm Remove a model
help Help about any command
Flags:
-h, --help help for ollama
-v, --version Show version information
Use "ollama [command] --help" for more information about a command.
Step 4: Install and Run a Large Language Model
With Ollama installed, we are only one step away from having our own local LLM on our machine.
To get started, let's install Meta's Llama3.2 model.
In our command prompt window, type:
ollama run llama3.2
This will do two things:
- Install Llama3.2 if it is not currently on our machine
- Run the Llama3.2 Model
Press return/enter and you should see something like this (if Llama3.2 is not installed):
pulling manifest
pulling dde5aa3fc5ff... 26% ▕████████████ ▏ 532 MB/2.0 GB 32 MB/s 48s
Ollama will now download and install the Llama3.2 model on your machine and once complete, run the llama3.2 model for you to interact with.
In your command line window, you should see something like this:
pulling manifest
pulling dde5aa3fc5ff... 100% ▕████████████████████████████████████████████████████████▏ 2.0 GB
pulling 966de95ca8a6... 100% ▕████████████████████████████████████████████████████████▏ 1.4 KB
pulling fcc5a6bec9da... 100% ▕████████████████████████████████████████████████████████▏ 7.7 KB
pulling a70ff7e570d9... 100% ▕████████████████████████████████████████████████████████▏ 6.0 KB
pulling 56bb8bd477a5... 100% ▕████████████████████████████████████████████████████████▏ 96 B
pulling 34bb5ab01051... 100% ▕████████████████████████████████████████████████████████▏ 561 B
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)
And just like that, you now have a LLM on your local machine!!
Also, If you would like to install any other model available through Ollama, you would use the same command:
ollama run MODEL_NAME
Step 5: Test out your Large Language Model
Now that Llama3.2 is on our machine, let's test it out.
Let's ask it a simple question:
Hey Llama! Could you tell me a little bit about yourself?
You should then get a response similar to this:
Hey Llama! Could you tell me a little bit about yourself?
I'm happy to chat with you. I'm an artificial intelligence model known as Llama, which stands for "Large Language
Model Meta AI." My primary function is to process and generate human-like text based on the input I receive.
I was trained on a massive dataset of text from various sources, including books, articles, and online
conversations. This training allows me to understand and respond to a wide range of questions, topics, and styles.
Some key features of my abilities include:
.....
See! Wasn't that pretty easy!
Wrap Up
If you have been a dev for any period of time, you're likely desensitized to the idea of "The Next Big thing" in tech. Though I have a natural curious nature, overtime, I have become more and more desensitized to "The Cutting Edge"; if its been Virtual Reality, Web3, Crypto, AI or anything else, I have found myself initially hopeful and then later disappointed by the possibilities.
But in a uplifting turn of events, my spark for Artificial Intelligence has been reignited through continued exposure and arelentless desire to find it's value.
I have truly come to the conclusion, that the future of the development will be heavily intertwined with AI and Generative AI tools.
If you are a developer, engineer, software engineer or someone who works heavily in tech spaces, it's time to embrace the benefits that AI technology affords us.
I can't tell you what the future holds for our profession as developers or whats next for AI as a whole, but I do have a strong sense that we are at an inflection point.
I encourage you to take the time to discover your own "AI Blindspots" and utilize the tools and technology we have available to enhance and accelerate your own development workflows.
Photo Credits(Order of Appearance):
Cover Photo with Google Gemini
Prompt:
A happy fluffy white llama, standing tall in the center of the frame, bathed in the warm glow of a bright light shining from behind its head, giving the illusion of enlightenment, its eyes closed in meditation, face looking toward the top right corner as if gazing into the future.
Follow me on my Socials:
https://linktr.ee/bradstondev
Top comments (12)
Interesting article. Thanks for sharing your experience, and I definitely agree with asking multiple different LLMs the same question. It's kinda like getting opinions from more than one doctor 😊.
A friendly suggestion for you: Your article had quite a few grammatical/spelling errors which you could easily fix by using.... AI! 😁
Seriously, though, I had to reread a few parts to make sure I understood what you were trying to say. You could do something as simple as pasting the text into a Google doc to immediately see the errors. I actually prefer this to asking an LLM to fix errors because I prefer to think about how I want to fix the errors (and sometimes they aren't really errors), but LLMs are obviously a viable option.
I can see the thought you put into your article – conceptually it's great! But if you want to grow your readership, you'll want to make your writing as clear as possible so that people don't have to work to understand what you're saying.
Nice work, and best of luck to you!
Thanks @squidbe ! I really appreciate your kind words and your comments and feedback really resonates with me! I do use Google Docs for early drafts but I DO need to use it more often on my final drafts. This is a great point and something I will start to make sure I do in future posts.
But because you brought up the topic of AI in writing, this might be a great avenue to share something pretty personal to me. 😊
I actually purposefully do not use AI tools when it comes to my writing here or anywhere else I share my writing.
Here are my reasons:
TLDR; I want to grow in my writing, speaking, and editing skills to become a stronger natural communicator and to keep my writing voice. I avoid AI because it does not help me to reach my personal goals.
I want to continue to develop my writing voice and grow without being biased by any specific LLM. I have found in the past that LLMs will alter my voice and approach. I have often fought with AI chat agents to work with me and my goals. Haha!
I want to grow in my editing and proofreading skills. As I do a LOT of writing and sometimes find myself in scenarios where AI tools are not available, I need to harden my writing skills.
Building on point 2, I do public speaking quite often and my speech often reflects my writing. I have found as I write, proofread, and edit more often, my public speaking communication has improved.
(Building on Point 1) I'm from the Southern Unites States and I tend to use AAV (African American Vernacular) when I speak. Though I understand that most people don't understand that in written form, it is important to me to keep some of "who I am" in my writing. AI often deletes my cultural identity when it edits my writing.
Finally, I'm okay with the mistakes. In a way, I'm happy the mistakes exist so I'm never mistaken me for an AI when I write. I don't desire to be perfect but I do desire to be better. I'll also add that I spend a lot of time reading blogs on this and other sites and I can almost always identify when it's been written by or highly edited by AI. I personally do not enjoy overly AI produced writing pieces.
P.S. I will be re-reading through this blog again as I appreciate your feedback as it likely needs some more TLC. haha
Once again, I appreciate your thoughts. It will make me fight to be even more thorough with my re-reads and edits. I clearly need to tighten up on that skill.
Side Note: I do have a crazy goal to train an LLM that writes like me to give me a blueprint on how I can write my blogs in the future but, for now, I more or less deliberately avoid running my writing through AI tools.
That is excellent! I wish more people felt the way you do. I get why some people just want to pop their text into some chatbot and accept what it gives them, but if too many people do that, writing will become homogenous and bland. So, kudos to you for acknowledging the importance of having your own writing voice. 😎
That's why I just use Google Docs. It gives simple cues on possible errors, and that's all I want.
You mentioned fixing grammar using AI, but to be fair you don't need the "AI" to do it. There are great tools for this, Grammarly is a famous one but LanguageTool is a great alternative (which I use) which is open source (correct me if I am wrong about this).
I haven't looked into how these tools work exactly, but definitely not the LLM type of AI but old school text parsing pro max. And they work fantastically.
Not every automation needs AI, which is a great thing.
You know, I really need to look into Grammarly. I have been avoiding it (for the same reasons I mentioned in my other comment) but I should give it a go.
Definitely. That's why I mentioned Google Docs. Spell check and grammar check have existed for decades – I remember using them when I used to use MS Word a looong time ago.
I can see why some people want to use AI to check spelling/grammar, but I prefer to be the one making the decisions on what I do/don't change, which is why I like the simple cues of red and blue underlines to point out potential problems so that I can quickly decide whether to make a change.
Great writeup, very insightful - this post should have been in "Top 7 Featured DEV Posts of the Week" (dev.to/devteam/top-7-featured-dev-...) !
Just a question (funny how I realize I could ask this question to ChatGPT or another LLM and probably receive a capable answer, haha):
What are the pros and cons of using a local LLM running on your own machine, versus one hosted "in the cloud" and accessed online?
One advantage I can think of is that, even when I'd have no working internet connection for whatever reason, I'd still have a capable "local search engine" at my disposal?
Another potential advantage, for the environmentally conscious among us: we're not contributing to the already ballooning (largely due to AI) energy consumption of data centers (but maybe we're just shifting the energy consumption to our own machine ...)
Drawbacks that come to mind are the resources this uses on my local machine - disk space, memory, CPU, energy/electricity consumption ...
Thoughts?
First, thanks you so much for the kind words and reading my blog. I TRULY appreciate it! And I'm not sure if you saw, but i did end up being spotlighted in the "Top 7 Featured" blog so that was a true honor! :-)
As far as pros and cons, I think I echo the pros and cons you mentioned. I think energy consumption and privacy are huge pros for local LLMs. I tend to spin down my LLMs when I'm not using them but cloud LLMs have to be essentially accessible at all times so there's a continuous need for them to be running (thus the HUGE energy consumption).
so what I'll do is share some of my additional thoughts on some of the pros and cons that stand out for me.
Pros (Local LLMs)
Privacy: The ability to work with a Chat agent and to know that the information will never be stored in the cloud is great. Technically, I know most large cloud LLMs do not share out info (at least not intentionally) but there is peace of mind knowing that my conversations, especially ones with intimate details, remain on my machine.
Curation : I think one nice element of tools like Ollama is the ability to curate the LLMs that best suite your needs for a specific task. You could use Llama3.2 for general queries, you can use llam3.2-vision for vision based tasks, and use codegemma for code generation. I think the ability to specialize is powerful and incredibly helpful for accomplishing you personal and dev goals.
Personal Learning : The ability to experiment and learn about how LLMs work is one of the biggest pros to me. Though I am not training these models from scratch, I am learning a ton about their capabilities and how models differ and about proper prompting. Cloud models tend to be more of a black-box until they are released to the public (if they are open-sourced). Just being able to build upon models and having the ability to investigate more deeply into them has taught me so much.
Cons (Local LLMs)
Less 'Accurate': Open-Source models can sometimes be older models trained on older data and sometimes are still working on previous generational bugs. I have run into issues with Llama3.2 where it would give me "hallucinations" to certain questions I ask but if I ask a more modern Cloud model, like Gemini, it wouldn't have issues. When I need information that will be heavily impacted by recency or need it to be more consistent and less likely to contain hallucinations in its answer, I tend to use a cloud model.
Accessibility: Hosting your own models is not accessible to everyone because of system requirements. I hope to update my blog with this note (as some people have pointed out I haven't) but not every machine can run Llama or any other model. Cloud LLMs truly have made AI accessible to almost anyone w/ an internet connection
Model Selection: This isn't the biggest deal but I find discovering and working with different models somewhat cumbersome. There are a lot of open-source models that are available and it's truly hard at times to differentiate between their individual strengths and weaknesses. I'm highly invested and interested in available models but it's hard to keep with all the old and new models and versions. I find myself primarily investigating models that have already been said by others to be "good". Cloud LLMs tend to be the best versions of their existing models, tend to be pretty comparable, and there are far less of them; making it easier to choose a model.
As I continue to work with more local and cloud models, I'm interested to see how this pro-con list evolves.
This was a great question. Thanks for asking!
You really have me thinking even more deeply about all of this. haha
Thanks, yes I think most of that makes sense ... and yes, now I see that you're being featured in the "7 posts of the week" blog - not sure how I missed it, but that's great!
Last night, I built two GPTs for my personal use to help with my farm's social media and marketing practices and assist with creating content for my brand on LinkedIn. I cracked myself up when I told my wife last night while going to bed that I had just hired our first two employees. Their names are "K3bert Marketing Maven and Copy Catalyst" and "Kunekune Copy Editor and Marketing Muse."
I'm seeing more people, not just technologists, discussing ways they use AI to "supercharge" their skills as leaders, coders, marketers, etc. If there is any takeaway from your post, there is much to learn to build custom-built AIs for your specific needs, either through one of the services out there or by installing them on your machine and working from there. But, it's a journey I encourage everyone to go on, regardless of where you are in your career journey.
Love this, Thanks for the write up. I’m going to get this running for myself soon! It’s a great tutorial and actual real human experience.
Thanks @obeykinobi ! It's really been great for me. You should check out Open WebUI when you get comfortable with Ollama. I'm using it with Llama, because it gives you a nice interface similar to ChatGPT and Gemini. Hoping to write up a blog about it soon!
Some comments may only be visible to logged-in visitors. Sign in to view all comments. Some comments have been hidden by the post's author - find out more