DEV Community: Jeremy Morgan

I attended POST/CON 2025. Here are my thoughts

Jeremy Morgan — Mon, 16 Jun 2025 16:54:10 +0000

I just returned from POST/CON 25, and I'm left with one big impression: Postman gets it.

They get that we're not just managing APIs. We're building complex systems, and increasingly, those systems involve AI. And as AI technologies race ahead Postman is right there to help you keep up.

I've been using Postman for years, and I was thrilled to be invited to POST/CON 2025 in Los Angeles. It was an amazing experience, and I want to share some of the highlights.

If you want the full rundown of the conference, you can check out the official recap here. I want to share my personal highlights and thoughts on the event.

Day 1: Full day workshop - "Mastering AI agent Automation with Postman Flows"

There I learned:

🔥 How easy it is to connect things together with flow

🔥 How to tie together AI agents into a seamless workflow and deploy it

🔥 Learn best practices for design

Then a fireside chat with Ryan Reynolds and Justine Davis that had almost nothing to do with code and everything to do with building great products that solve real problems, and taking care of people.

Day 2: Keynote and Sessions

Attended the Keynote where I learned about

🛠️ Postman VS code extensions

🛠️ GitHub Integration

🛠️ Postman Insights (lots of great analytics)

🛠️ AI Agent Builder

🛠️ Model Comparison Template

🛠️ MCP Server Generator (REALLY)

and so much more. You can get all the details here.

Sessions

I attended some amazing sessions, where I learned a ton about

🧠 Agent Orchestration using API-driven workflows

🧠 Building multi-agent experiences with Agent Builder

🧠 Automating API Testing in CI/CD pipelines with CircleCI and Postman

🧠 Composable API Ecosystems and the MCP Protocol

and tons of great lightning talks.

One of the best conferences I've attended

The energy at this conference was amazing. It's one of the best I've attended. Postman is at the forefront of tech. They've created great tools to keep YOU ahead of the game. They focus on building their product well and help you create your best products, too.

I highly suggest going to the next POST/CON. If you do, let me know and let's meet up!

Thank you so much to Postman for sending me to this conference. Now, if you'll excuse me, I have some new tools to try out!

-- Jeremy

Running DeepSeek R1 Locally on a Raspberry Pi

Jeremy Morgan — Sat, 15 Feb 2025 00:09:53 +0000

DeepSeek R1 shook the Generative AI world, and everyone even remotely interested in AI rushed to try it out. It is a great model, IMO. As you may know, I love to run models locally, and since this is an open-source model, of course, I had to try it out. It works great on my Mac Studio and 4090 machines.

But can it run on this? A silly little Raspberry Pi?

I received a few emails and private messages asking about this and had to try it out. In this tutorial, we'll walk through how to run DeepSeek R1 models on a Raspberry Pi 5 and evaluate their performance. Whether you want to get into running LLMs locally or build some edge AI stuff, this could be a fun tutorial to try out.

Here it is in video form if you prefer that:

So let's get started. Here's what you need:

Prerequisites

Hardware:

Raspberry Pi 5 (8GB or 16GB RAM recommended).
MicroSD card with Raspberry Pi OS (64-bit) installed.
Stable power supply and internet connection.

Software:

Basic familiarity with Raspberry Pi OS and terminal commands.
Docker (optional, for containerized applications).

Here's the machine I tried it with:

Step 1: Configure Your Raspberry Pi

Enable Remote Desktop (Optional)

To simplify GUI interactions, enable Remote Desktop:

sudo apt-get install xrdp -y
sudo systemctl start xrdp
sudo systemctl enable xrdp

Set the Pi to auto-login to the desktop GUI.

Type in

sudo raspi-config

And select 1 System Options

Then select S5 Boot/Auto Login

and choose B3 Desktop (Desktop GUI, requiring user to login)

Now, if you want, you can log into the desktop remotely using RDP and access a desktop.

Note: on my Raspberry Pi I could not get any browsers to run properly. They would not display in a readable form, I'll be looking it and trying to fix that at some point.

Your browser may also look like this:

Step 2: Install Ollama

Ollama is a tool for running LLMs locally. Since Raspberry Pi uses ARM64 architecture, download the Ollama Linux ARM64 build:

curl -fsSL https://ollama.com/install.sh | sh

Verify the installation:

ollama --version

Note that it will say

WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.

This is because the Raspberry Pi's GPU is not CUDA compatible. However, we can see how it runs on just the CPU.

Step 3: Run DeepSeek R1 Models

Test the 1.5B Parameter Model

Let's start with the smallest model available to try it out. That would be the deepseek-r1 1.5b model model, which has 1.5 billion parameters.

ollama run deepseek-r1:1.5b

Example Prompt:


>>> Tell me a funny joke about Python.

Performance Notes:

Speed: ~6.12 tokens/second.
RAM Usage: ~3GB.
CPU: Consistently maxed at 99%.

So that's not too bad. It's not great, but it might be useful for prototyping and experimenting. You could use this for many tasks as long as it isn't real-time chat or something immediately interactive.

Let's try a bigger model. We know it will be slower, but I want to see if it's possible.

Test the 7B Parameter Model (Optional)

For this test, we'll try the deepseek-r1 7B parameter model. This one is considerably bigger and is a pretty decent model for many projects.

ollama run deepseek-r1:7b

Performance Notes:

Speed: ~1.43 tokens/second (extremely slow).
RAM Usage: ~6GB (requires 8GB+ Pi).

This one is really slow and barely usable. But it's still impressive that you can run a 7B parameter model on a Raspberry Pi, in my opinion. There's not much use for it, but it's possible.

Step 4: Deploy a Dockerized Chat Application

Let's try running a dockerized chat application. This is an application I built for a previous YouTube video. It's a VueJS application that uses the DeepSeek R1 models.

The source code is available here.

Here's how to run it on a Pi:

Clone the Demo App (replace with your own project):

git clone https://github.com/JeremyMorgan/DeepSeek-Chat-Demo
cd deepseek-chat-demo

Build and Run with Docker:

docker compose up

Access the app at http://localhost:5173 and test prompts.

Hey! It runs! Awesome. The responsiveness isn't too bad, it's a little slow but usable.

Step 5: Experiment with a Raspberry Pi Cluster (Extra Bonus Round!)

I decided to drop this on my Raspberry Cluster just to try it out. While this doesn't improve speed (LLMs run on single nodes), it's a fun experiment for distributed workloads.

These Raspberry Pis are all 8GB models instead of the 16GB. Here's how you can set it up:

Set Up Kubernetes using micorok8s or similar tools.
Deploy the App by creating via a YAML manifest (deepseek-deployment.yaml):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deepseek-deployment
  labels:
    app: deepseek
spec:
  replicas: 4
  selector:
    matchLabels:
      app: deepseek
  template:
    metadata:
      labels:
        app: deepseek
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app: deepseek
              topologyKey: "kubernetes.io/hostname"
      containers:
        - name: deepseek
          image: jeremymorgan/deepseek-1-5b-chat-demo:ARM64
          ports:
            - containerPort: 11435
              hostPort: 11435

Create a service (deepseek-service.yaml):

apiVersion: v1
kind: Service
metadata:
  name: deepseek-service
spec:
  type: NodePort
  selector:
    app: deepseek
  ports:
    - name: http
      protocol: TCP
      port: 80           # service port (cluster-internal)
      targetPort: 80     # container port
      nodePort: 30080    # external port on every node
    - name: chat
      protocol: TCP
      port: 11436        # service port (cluster-internal)
      targetPort: 11436  # container port
      nodePort: 31144    # external port on every node

The responses remain slow, as clustering doesn't parallelize model inference. However, the performance difference between 8GB and 16GB is not noticeable with the 1.5B parameter model.

You can grab the Docker image from here and check it out:

https://hub.docker.com/r/jeremymorgan/deepseek-1-5b-chat-demo/

Key Takeaways

So here's what you'll learn while deploying DeepSeek models to the Raspberry Pi 5.

Feasibility:
- The 1.5B model runs acceptably on an 8-16GB Raspberry Pi 5 for lightweight tasks.
- The 7B model will run, but is impractical due to speed (1.43 tokens/sec).
Use Cases:
- Educational experiments.
- Prototyping edge AI applications.
Optimization Tips:
- Use quantized models (e.g., 4-bit GGUF) for better performance.
- Avoid GUI overhead by running Ollama headless.

Conclusion

While DeepSeek R1 won't replace cloud-based LLMs on a Raspberry Pi, it's a fun way to explore AI on budget hardware. Consider upgrading to a Jetson Nano or used GPU server for better performance.

It's a fun experiment!

Final Results:

Model Size	Tokens/sec	RAM Usage	Usability
1.5B	~6.12	3GB	Basic Q&A
7B	~1.43	6GB	Not recommended

Try it yourself and see how it goes, if you have any comments or questions Yell at me!!

Review: The New NVIDIA Jetson Orin Nano

Jeremy Morgan — Tue, 24 Dec 2024 00:57:02 +0000

Hello, friends! If you're a reader of my nerdy articles you've probably heard about NVIDIA's Jetson. It's a great platform for prototyping apps and putting AI at the edge.

I got lucky and got my hands on the newest, very affordable Jetson, the Jetson Orin Nano.

Today, we’ll dive into everything from unboxing this little gem to testing its performance with AI models. Ready to explore what makes this device so cool? Let’s get started!

If you'd rather see a video version of this review here it is:

What's Inside the Box?

First impressions matter, and the Orin Nano’s packaging is as clean and straightforward as you’d expect. Here’s what you get:

A robust power supply with both European and American plugs.
A quick start manual (kinda)
And, of course, the star of the show: the Jetson Orin Nano.

You do need to purchase an SD card for it, aas it's not shipped with one.

The device is sleek, lightweight, and full of potential. It includes HDMI, USB, Ethernet, GPIO, and video input ports. Whether you're making an AI robot or building computer vision stuff (I know I will), this device has you covered.

The Specs: Why It’s Called a "Raspberry Pi on Steroids"

Let’s talk hardware. Look at what's packed into this small device:

CPU: A six-core ARM Cortex 64-bit processor.
GPU: NVIDIA Ampere architecture, tuned for AI workloads.
RAM: 8 GB of high-speed memory.
Connectivity: Ethernet and wireless ready.
Storage Options: NVMe support for all your AI model needs.

Think of it as a supercharged Raspberry Pi with a serious focus on AI.

Setting It Up: The Good, The Tricky, and The Rewarding

Getting the Orin Nano up and running wasn’t exactly plug-and-play, but it wasn’t rocket science either. Here’s how it went down:

Firmware Update: Before diving in, I updated the firmware to version 36.x. This step required temporarily installing Jetpack 5.1.3. Then, you update the firmware from there. (Pro tip: don’t skip this. You want the new firmware and Jetpack)
Jetpack 6.1 Installation: After the firmware update, I installed Jetpack 6.1, NVIDIA’s tailored OS for the Jetson lineup. It’s smooth, intuitive, and optimized for AI tasks.

If you're purchasing this devcie, then burning SD cards and flashing firmware shouldn't be a problem.

Performance Tests: How Does It Handle AI Models?

Here’s where the fun begins—I tested the Orin Nano with AI models using Ollama, a platform designed for running large language models (LLMs).

Check out Ollama here

Here’s what I discovered:

1 Billion Parameter Models: Smooth sailing! These models ran effortlessly, perfect for tasks like chatbots or real-time applications.
3 Billion Parameter Models: Still impressive, though with slightly longer loading times.
7 Billion Parameter Models: Houston, we have a problem. The Orin Nano hit its limits here, occasionally freezing or locking up.

While it’s not designed for massive models, this performance is impressive for a device in this price range.

You can view the performance tests I did in this video.

Pros and Cons: The Real Deal

What I Loved:

Compact and Lightweight: Perfect for edge AI projects where space is a premium.
Impressive Performance: At $250, you’re getting serious bang for your buck.
Quiet Operation: Even under load, it’s quiet—a big plus for home projects.
User-Friendly OS: Jetpack is intuitive and packed with features.

What Needs Improvement:

Limits on Larger Models: It’s not a deal-breaker, but don’t expect it to handle anything beyond 7 billion parameters comfortably.
Occasional Glitches: Some random lockups and an odd “system throttled due to overcurrent” error. I'm looking into it. Manageable, but worth noting.

Final Verdict: Is It Worth It?

Absolutely. The NVIDIA Jetson Orin Nano is a fantastic choice for developers exploring AI at the edge or prototyping innovative solutions.

It’s not built for heavy-duty production workloads, but for $250, it’s hard to beat the value and potential.

What’s Next?

I'll explore the Orin Nano's features more in upcoming projects, so stay tuned. If you're interested in edge AI or want to try out large language models, this device is a perfect start.

Have questions or suggestions for tests? Just drop a comment or message me on social media. Let's see what this small device can really do!

Happy experimenting! 🚀

You can order the NVIDIA Jetson Orin Nano from the following places (Though right now there is no stock, check back!)

Questions, comments? Want more details? Yell at me!!

Video version of this tutorial:

How to Make a Retro 2D JavaScript Game Part 3

Jeremy Morgan — Thu, 19 Dec 2024 19:39:26 +0000

Let’s make the game more fun with scoring and difficulty progression.

1. Adding a Score

Modify the create function to display a score:

this.score = 0;
this.scoreText = this.add.text(16, 16, 'Score: 0', { fontSize: '24px', fill: '#fff' });

Now you'll see a score in the upper left hand corner of the screen:

Update the score when the player catches an item. In the update function where we created collision detection, add the following to update the score:

this.score += 10;
this.scoreText.setText('Score: ' + this.score);

So change this:

if (Phaser.Geom.Intersects.RectangleToRectangle(this.player.getBounds(), this.item.getBounds())) {
        console.log('Caught an item!');
        this.item.y = 50;
        this.item.x = Phaser.Math.Between(50, 750);
}

to this:

if (Phaser.Geom.Intersects.RectangleToRectangle(this.player.getBounds(), this.item.getBounds())) {
        console.log('Caught an item!');
        this.item.y = 50;
        this.item.x = Phaser.Math.Between(50, 750);
        this.score += 10;
        this.scoreText.setText('Score: ' + this.score);
}

And now every time you catch a block, the score updates:

2. Increasing Difficulty

Let's make this more difficult and make the items fall faster as the score increases:

In our create function, add this:

this.itemSpeed = 3;

in our update() function, delete this:

    // Move falling item
    this.item.y += 3;

And replace it with this:

  this.item.y += this.itemSpeed;

    if (this.score > 0 && this.score % 500 === 0) {
        this.itemSpeed += 0.5;
        console.log('Going faster! Speed is now: ' + this.itemSpeed);
    }

Now, every time you get 500 points, your speed increases, and it gets more difficult!

3. Retro Feel Enhancements

So let's add some graphics to this to make it more exciting.

I just drew up these images, don't judge me.

in preload, add the following:

// Load the player sprite
this.load.image('player', 'pail.png');

Then remove this:

// Player (Blue rectangle)
this.player = this.add.rectangle(400, 550, 50, 50, 0x0000ff);

And replace it with this:

   // Replace rectangle with sprite
    this.player = this.add.sprite(400, 550, 'player');
    this.player.setScale(1); // Adjust this value if needed to match desired size

Let's turn the falling objects into apples.

In preload(), add:

this.load.image('apple', 'apple.png');  // Loa

In create(), delete this:

    // Falling item (Green rectangle)
    this.item = this.add.rectangle(400, 50, 50, 50, 0x00ff00);

and replace it with this:

    // Apple sprite
    this.item = this.add.sprite(400, 50, 'apple');
    this.item.setScale(1); // Adjust scale if needed

And now you'll see a different look!

Now we see an apple and a pail, but it's in a dark room. Let's enhance our look by adding in a background.

In preload():

 this.load.image('background', 'background.png');  // Load background image

and in create():

    // Add background first so it appears behind other sprites
    this.add.image(400, 300, 'background');  // Position at center of game (800/2, 600/2)

And let's make the score black so we can see it:

this.scoreText = this.add.text(16, 16, 'Score: 0', { fontSize: '24px', fill: '#fff' });

Now save it and reload it:

Hey that looks awesome!

Final Thoughts

Congratulations! 🎉 You’ve built a working "Catch the Items" game using Phaser 3.

Next Steps

Customize the shapes with images or sprites.
Add sound effects.
Experiment with game parameters like speed and item spawn rates.

Keep practicing, and have fun creating your games. The possibilities are endless—go make something awesome! 🚀

Note: If you'd rather have a video tutorial, here it is:

Source Code
Playable Version

How to Make a Retro 2D JavaScript Game Part 2

Jeremy Morgan — Thu, 19 Dec 2024 19:36:10 +0000

Let’s make this game interactive! We’ll add a player, movement controls, and falling items.

Note: If you'd rather have a video tutorial, here it is:

1. Adding Placeholder Graphics

Update the create function to add shapes representing the player and falling items:

function create() {
    // Player (Blue rectangle)
    this.player = this.add.rectangle(400, 550, 50, 50, 0x0000ff);

    // Falling item (Green rectangle)
    this.item = this.add.rectangle(400, 50, 50, 50, 0x00ff00);

    // Enable physics
    this.physics.add.existing(this.player);
    this.physics.add.existing(this.item);

    // Player controls
    this.cursors = this.input.keyboard.createCursorKeys();
}

2. Moving the Player Left and Right

Add movement logic to the update function:

function update() {
    // Move player left
    if (this.cursors.left.isDown) {
        this.player.x -= 5;
    }
    // Move player right
    else if (this.cursors.right.isDown) {
        this.player.x += 5;
    }
}

The player can now move left and right using arrow keys!

3. Adding Simple Collision Logic

We need to update our config:

change:

   type: Phaser.AUTO, // Auto-detect WebGL or Canvas
   width: 800,        // Game width
   height: 600,       // Game height

To this, to add physics and gravity to our scene.

type: Phaser.AUTO,
            width: 800,
            height: 600,
            physics: {
                default: 'arcade',
                arcade: {
                    gravity: { y: 0 }
                }
            },

Make the falling item reset its position when it reaches the bottom:

function update() {
    // Move falling item
    this.item.y += 3;

    // Reset item position
    if (this.item.y > 600) {
        this.item.y = 50;
        this.item.x = Phaser.Math.Between(50, 750); // Random x-position
    }

    // Check for overlap
    if (Phaser.Geom.Intersects.RectangleToRectangle(this.player.getBounds(), this.item.getBounds())) {
        console.log('Caught an item!');
        this.item.y = 50;
        this.item.x = Phaser.Math.Between(50, 750);
    }
}

Now you'll see a screen that looks like this: and you should see green blocks falling. You can also move the player with your arrow keys:

📝 Recap: You added player movement, falling items, and basic collision detection. Your game is interactive! Now on to part 3

Note: If you'd rather have a video tutorial, here it is:

Source Code
Playable Version

How to Make a Retro 2D JavaScript Game Part 1

Jeremy Morgan — Thu, 19 Dec 2024 19:34:38 +0000

Welcome, aspiring game developers! 🚀 In this beginner-friendly guide, we’ll build a simple, retro-themed "Catch the Items" game using Phaser 3, a powerful JavaScript game development framework. This series of tutorials is designed for absolute beginners, so don’t worry if you’re new to coding or Phaser—you’re in good hands.

Note: If you'd rather have a video tutorial, here it is:

Tutorial 1: Project Setup & Basics

Let’s start by setting up the environment, introducing key JavaScript concepts, and creating our first Phaser scene.

Note: In many of our tutorials we use Node/Vite to set up our games. But I want to keep this as simple as possible. To run this you can install the NPM package serve or just open the document with Chrome.

1. What is Phaser 3?

Phaser 3 is a popular 2D game development framework for creating browser-based games using JavaScript. It’s beginner-friendly, flexible, and powerful—perfect for making retro-inspired games like this one!

2. Setting Up Your Development Environment

To keep things simple, we’ll use Phaser via its CDN link. Here’s the setup:

Create a project folder:
- Make a new folder on your computer, e.g., catch-game.
- Inside the folder, create a file named index.html.
Add the basic HTML structure:
Paste the following code into your index.html file:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Catch the Items</title>
    <script src="https://cdn.jsdelivr.net/npm/phaser@3.55.2/dist/phaser.js"></script>
</head>
<body>
    <script>
        // Phaser Game Configuration
        const config = {
            type: Phaser.AUTO, // Auto-detect WebGL or Canvas
            width: 800,        // Game width
            height: 600,       // Game height
            scene: {
                preload: preload,
                create: create,
                update: update
            }
        };

        const game = new Phaser.Game(config);

        function preload() {
            // Preload assets (none yet)
        }

        function create() {
            // Add game objects here
            this.add.text(300, 250, 'Hello, Phaser!', { fontSize: '32px', fill: '#fff' });
        }

        function update() {
            // Game loop (empty for now)
        }
    </script>
</body>
</html>

Run your game:
- Open the index.html file in any browser (Chrome is recommended).
- You should see a simple canvas with the text "Hello, Phaser!".

🎉 Congratulations! You’ve set up Phaser and displayed your first scene!

3. JavaScript Basics for Phaser

Before diving deeper, let’s quickly review some JavaScript concepts:

Variables: Store data.

   let playerName = 'PhaserHero';

Functions: Reusable blocks of code.

   function greetPlayer() {
       console.log('Welcome to the game!');
   }
   greetPlayer();

Objects: Collections of properties and methods.

   let player = {
       name: 'Hero',
       score: 0,
       jump: function() {
           console.log('Player jumps!');
       }
   };
   player.jump();

You’ll use these concepts as you develop your game in Phaser.

4. Creating a Minimal Phaser Scene

Modify the create function to display a basic game canvas and some text:

function create() {
    this.add.text(300, 250, 'Catch the Items!', { fontSize: '32px', fill: '#fff' });
    this.add.rectangle(400, 300, 50, 50, 0xff0000); // Red square
}

Reload the browser—you should see a red square in the center of the screen.

What does create() mean?

Phaser uses three core functions to manage the game lifecycle: preload, create, and update.

Let's break down what each function does:

preload(): This function is called before the game starts. It's where you load your game assets like images, sounds, and more.
create(): This function is called after the assets are loaded. It's where you create your game objects, set up the game world, and define initial game settings.
update(): This function is called repeatedly throughout the game loop. It's where you update the game state, handle user input, and make things happen in your game world.

We'll be using these functions extensively as we build our games in this tutorial and future tutorials.

📝 Recap: You’ve set up a Phaser game, learned some JavaScript basics, and displayed simple shapes on the canvas. Great start! Now on to part 2

Source Code
Playable Version

Get Started with Python

Jeremy Morgan — Mon, 16 Dec 2024 18:39:36 +0000

Hey there, fellow geeks and future coders! Welcome to Part 1 of our series, "Learn Python"! If you've ever wanted to learn Python but felt overwhelmed by where to start, you’re in the right place. We’re going to break it down in easy, bite-sized chunks, walking you through each concept step by step. So grab your coffee (or your favorite beverage), and let’s dive in!

Why Python?

First off, why should you learn Python? Well, Python is one of the easiest programming languages to get started with. It’s used everywhere—web development, data science, automation, artificial intelligence, you name it! Whether you’re automating boring tasks or building the next big app, Python has your back.

But the best part? Python is super beginner-friendly. Its syntax (fancy word for how code is written) is clean and straightforward, so you can focus more on what you want to do rather than how to do it.

Pretty cool, right?

What Can You Build with Python?

Websites (ever heard of Django or Flask?)
Automation scripts (goodbye repetitive tasks!)
Data analysis and machine learning models
Games (even 2D ones like the classics!)
IoT projects (you know, like controlling lights with a Raspberry Pi)

Okay, enough hype—let’s get started!

Step 1: Setting Up Python

Before we can write any Python code, we need to set up our environment. Don’t worry, this is easier than it sounds.

Install Python

First, you'll need to install Python if you don't have it already. Go to the official Python website python.org, download the latest version, and follow the instructions.

For Windows: During installation, make sure to check the box that says “Add Python to PATH.” This will let you run Python from anywhere on your system.

For macOS/Linux: Python usually comes pre-installed, but if you need the latest version, you can install it via a package manager like Homebrew (brew install python3).

If you want, you can also run Python in a Web Browser here.

Verify the Installation

Once installed, open a terminal (or Command Prompt on Windows) and type:

python --version

If you see something like Python 3.x.x, congratulations! Python is ready to roll.

Running Python Code

You can run Python code in two ways:

Interactive Mode: Just type python in your terminal, and you can start writing Python code line-by-line. This is great for quick tests.

Demo idea: Try it out right now! Type python in your terminal and then type:

   print("Testing in interactive mode!")

Hit enter, and you’ll see the result immediately. This is a fun way to experiment.

Script Mode: Write your Python code in a .py file (like hello.py), then run it by typing:

   python hello.py

Step 2: Your First Python Program

Alright, let’s write your very first Python program. We’re going to create the classic “Hello, World!” program.

Write Your Code

Open your favorite text editor (VSCode, Sublime, or even Notepad), and type the following:

print("Hello, World!")

Save the file as hello.py, then run it from the terminal:

python hello.py

Boom! You should see:

Hello, World!

🎉 Congrats, you just wrote your first Python program! 🎉

Make It Interactive (Optional Demo)

Want to take this a step further? Try prompting the user for input:

name = input("What's your name? ")
print("Nice to meet you, " + name + "!")

Save it as hello_interactive.py and run it. When prompted, type your name and see how Python responds! This little demo shows how Python can interact with users, making it feel more like a conversation and less like static code.

Step 3: Understanding the Basics

Let’s go over some fundamental concepts in Python. These are building blocks that we’ll use in the next parts of this series.

Variables

Think of variables as boxes where you can store information, like numbers or text.

# This is a variable storing a number
age = 30

# This is a variable storing a string (text)
name = "Alice"

You can use these variables later in your code:

print("My name is", name)
print("I am", age, "years old")

Data Types

Python has several data types you’ll use frequently:

Integers: Whole numbers (e.g., 42)
Floats: Decimal numbers (e.g., 3.14)
Strings: Text (e.g., "Hello, World!")
Booleans: True or False

You can even check the type of any variable with:

print(type(age))  # Outputs: <class 'int'>
print(type(name)) # Outputs: <class 'str'>

Comments

Sometimes you need to add notes to your code. Python ignores comments, so they’re just for you (or anyone else reading the code).

# This is a comment. Python won't run this.

Math Operations

Python can handle math like a calculator:

x = 10
y = 5

print(x + y)  # Addition
print(x - y)  # Subtraction
print(x * y)  # Multiplication
print(x / y)  # Division

Make a Simple Calculator (Demo)

Now that you know variables and math, let’s make a tiny calculator program:

num1 = int(input("Enter a number: "))
num2 = int(input("Enter another number: "))
print("Sum:", num1 + num2)
print("Product:", num1 * num2)

Run this, and you can quickly see Python doing something practical—performing math operations on the fly!

A Quick Variable Experiment

Try this short experiment to understand how strings can be combined:

message = "Hello"
message += ", World!"
print(message)

This shows you how variables can evolve as your program runs.

Step 4: Next Steps

Now that you've got Python set up and written your first program, you're ready to take the next step. In Part 2 of this series, we’ll explore conditions and loops, where Python will start making decisions for you and performing repetitive tasks. It’s going to be fun!

Practice Challenge

Before we wrap up, try writing a Python script that does the following:

Stores your name in a variable.
Prints a greeting like "Hello, [Your Name]!"
Multiplies two numbers and prints the result.

Here’s a hint for the multiplication part:

result = 8 * 7
print(result)

Got it? Awesome! You’re on your way to becoming a Python pro. 🚀

See you in Part 2, where we’ll make Python even more powerful!

Happy coding!

AI New Hotness for December 13th, 2024

Jeremy Morgan — Sat, 14 Dec 2024 05:22:53 +0000

Welcome to this week's edition of the AI New Hotness Newsletter, where we talk about new and exciting stuff to happen in the world of Generative AI, particularly for software developers. It’s a fast moving, wacky world for sure.

Reminder: This is also available on LinkedIn if you'd rather receive it there.

AI in the News

Introducing ChatGPT Pro

OpenAI releases ChatGPT Pro, a subscription service with powerful new features

OpenAI has released a new subscription service called ChatGPT Pro. This service provides unlimited access to OpenAI's most advanced models, including o1, o1-mini, GPT-4o, and Advanced Voice. It also includes o1 pro mode, which uses more compute to provide more reliable and accurate answers to difficult questions. ChatGPT Pro is designed for researchers, engineers, and other professionals who need to use AI for complex tasks.

Details:

ChatGPT Pro provides unlimited access to OpenAI's most advanced models, including o1, o1-mini, GPT-4o, and Advanced Voice.
It also includes o1 pro mode, which uses more compute to provide more reliable and accurate answers to difficult questions.
ChatGPT Pro is designed for researchers, engineers, and other professionals who need to use AI for complex tasks.

Why it Matters:

ChatGPT Pro is a powerful new tool that can be used for a variety of tasks, including writing, research, and development. With its unlimited access to OpenAI's most advanced models, ChatGPT Pro can help you to be more productive and creative.

Click here to read more

Make Movies with Text! This AI Writes Videos for You

OpenAI Unveils Sora, a Powerful New Video Generation Model

OpenAI has released a new AI model called Sora that can create realistic videos from just a text description. This means you can create movies, explainer videos, or even home videos using just your imagination and some text!

Details:

Sora can generate videos in a variety of styles, from photorealistic to cartoonish. It can also be used to create videos with special effects or to animate existing images.

Why it Matters:

Sora is a powerful new tool that could revolutionize the way videos are created. It could make it possible for anyone to create professional-quality videos, even if they don't have any filmmaking experience.

Click here to read more

Unleash Your Creativity with Amazon Nova, the Powerful New AI

Generate Stunning Videos and Images, and More!

Do you ever feel limited by your imagination? Amazon Nova is here to help! This new generation of AI can be used to generate all sorts of creative content, from videos and images to who-knows-what-else. Amazon Nova is also being used to improve the customer shopping experience on Amazon. Keep reading to learn more about how Amazon Nova can help you!

Details:

Amazon Nova is a new generation of foundation models that can be used to generate creative content.
Amazon Nova is also being used to improve the customer shopping experience on Amazon.

Why it Matters:

Amazon Nova is a powerful new tool that can be used to generate creative content and improve the customer shopping experience. If you're looking for a way to boost your creativity or make shopping on Amazon even easier, then Amazon Nova is definitely worth checking out.

Click here to read more

Google Releases Revolutionary AI Model, Gemini 2.0

Google unveils its latest AI model, Gemini 2.0, promising to revolutionize the field of AI agent creation. Gemini 2.0 boasts advanced capabilities that push the boundaries of what's possible in AI. But with great power comes great responsibility, and Google emphasizes its commitment to developing AI responsibly, with safety and security at the forefront.

Details:

Gemini 2.0 is more capable than previous versions, with native image and audio output and tool use.
Gemini 2.0 Flash is available to developers and trusted testers, with wider availability planned for early next year.
Google is exploring agentic experiences with Gemini 2.0, including Project Astra, Project Mariner, and Jules.
Google is committed to building AI responsibly, with safety and security as key priorities.

Why it Matters:

Gemini 2.0 has the potential to revolutionize various fields by enabling the creation of powerful and intelligent AI agents. Google's commitment to responsible AI development ensures these advancements are made with safety and security in mind.

Click here to read more

Apple's New AI Can Turn Your Thoughts Into Emojis!

Tired of boring emojis? Apple Intelligence has just received a major upgrade, bringing you Image Playground and Genmoji to revolutionize the way you express yourself.

Image Playground lets you unleash your creativity and create unique images using your imagination. It's like having a mini art studio right on your device!

Genmoji takes emojis to a whole new level. Generate custom emojis that perfectly capture your mood or personality.

Apple Intelligence is available now as a free software update, so you can start creating and expressing yourself in exciting new ways. But hurry, this update is only available for iPhone 16 and later, iPad with A17 Pro or M1 and later, and Mac with M1 and later!

Why it Matters:

Apple Intelligence's new features offer a fun and creative way to express yourself through images and emojis. It's a free update available for compatible devices, so there's no reason not to try it out!

Click here to read more

Apple Secretly Developing AI Chip to Rival NVIDIA

Apple is reportedly working with Broadcom to develop its own AI chip, code-named Baltra, to power its AI services and reduce reliance on Nvidia. This move positions Apple alongside other tech giants like Google in the race to develop cutting-edge AI hardware.

Details:

Apple is developing its first server chip specifically for AI processing.
This move aims to reduce reliance on Nvidia's high-priced and often unavailable processors.
Baltra is expected to be ready for mass production by 2026.
Apple and Broadcom have a history of collaboration, including a recent multi-billion-dollar deal for 5G components.
This development highlights the growing importance of AI hardware and the increasing competition in the AI chip market.

Click here to read more

Talk to Santa AND See Him Live!

OpenAI just dropped a major update for the holiday season! You can now chat with Santa Claus in real-time using video calls and screen sharing. This new "Advanced Voice Mode" feature allows you to:

Ask Santa anything: Get advice on beard care, hear about life at the North Pole, or have him tell you a story!
See Santa's jolly face: Experience the magic of Christmas firsthand with live video chat.
Get help with tricky responses: Brainstorm the perfect reply to a friend's Christmas message using screen sharing.
This feature is rolling out now:

Available on mobile apps first, then desktop apps and web.
Free for everyone to chat with Santa on the first try, even if you've used your voice limit.
Full access to video and screen sharing for Plus, Pro, and Teams users (Europe rollout coming soon).
Enterprise and EDU plans get access early next year.

Click here to read more

Anthropic's New AI Model: Is it Worth the Hype (and the Higher Price)?

Claude 3.5 Haiku is here, but does it live up to the promise?

Anthropic has released Claude 3.5 Haiku, a powerful new AI model that boasts improved coding, data analysis, and content moderation capabilities. However, this upgrade comes with a higher price tag, sparking debate among developers.

Details:

Claude 3.5 Haiku surpasses its predecessor, 3 Opus, in key areas, offering enhanced performance in coding, data extraction, and content moderation.

It can generate longer pieces of text and boasts a more up-to-date knowledge base.
However, 3.5 Haiku lacks image analysis capabilities, limiting its versatility compared to other Anthropic models.
The model's release was initially met with controversy due to an unexpected price increase.

Click here to read more

AI Tools

Create Engaging Video Training Content in Minutes with Lupo

Lupo is a platform that helps businesses create high-quality video training content quickly and easily. With Lupo, you can use Markdown and Marp to create courses, and even leverage their agency services if needed.

Why you might need this tool:

Create high-quality video training content quickly and easily
Use Markdown and Marp to create courses
Leverage their agency services if needed

Click here to read more

Want to Build AI Tools Like a Pro? This Low-Code Platform Makes it Easy (Even Without Coding Skills!)

AISmartCube is a low-code AI tool platform that allows users to build AI tools without any coding required. The platform provides access to global large models, various plugins, a shared knowledge base, and ready-to-use tools and assistants. It also offers flexible pricing with points and a free credit allowance.

Click here to read more

Want to run any program automatically?

Pinokio is a browser that lets you install, run, and programmatically control any application, automatically. You can use Pinokio scripts, which are shared by the community, to automate tasks or create new functionalities.

Why you might need this tool:

With Pinokio, you can:

Automate repetitive tasks
Create custom functionalities for your favorite applications
Be more productive by letting Pinokio handle the boring stuff

Click here to read more

Build Stunning Websites in 3D, Without Code - Dora!

Dora is a no-code platform that allows you to create 3D animated websites. With Dora, you can create beautiful and engaging websites without having to write any code.

Why you might need this tool:

Create stunning and unique websites that will stand out from the competition.
No coding required, so even those with no coding experience can create professional websites.
Large community of users to help you learn and grow.

Click here to read more

Build Websites and Landing Pages in Minutes - No Coding Required!

Sitekick AI is an AI-powered tool that allows you to create landing pages and websites easily. It can generate text, images, and code for your website, so you don't need any coding or design skills.

Why you might need this tool:

Easy to use: Sitekick AI does not require any coding or design skills, so it is perfect for people who do not have any experience with web development.
Affordable: Sitekick AI is affordable, and it has a 30-day money-back guarantee, so you can try it out risk-free.
Saves time: Sitekick AI can help you create websites and landing pages quickly and easily, so you can save time and focus on other important tasks.

Click here to read more

Learn about AI

Learn Anything Faster with the Power of AI

KodeKloud has recently released an AI tutor that can personalize the training for you, providing real-time feedback and guides to craft a custom learning sequence just for you. Each task is validated for accuracy and it supports multiple languages.

Personalized learning: The AI-powered learning suite tailors the learning experience to your individual needs and goals.

Real-time feedback: The AI Assistant provides real-time feedback on your progress, so you can identify and address any areas of difficulty.

Custom learning sequences: The AI Tutor creates a custom learning sequence for any tech topic you want to master.

Accuracy validation: Each task is validated for accuracy, so you can be sure that you are learning the correct information.

Multilingual support: The AI Assistant supports multiple languages, so you can learn in your preferred language.

Click here to read more

Build a Multimodal AI Agent with Gemini 2.0

In this tutorial, we'll build a Multimodal AI Agent using Google's Gemini 2.0 Flash model that can simultaneously analyze videos and conduct web searches. This powerful combination allows the agent to provide comprehensive responses by understanding both visual content and related web information.

Click here to read more

Learn AI Assisted Development

A course built by yours truly, you can learn AI-assisted programming, project planning, backend and frontend development, plus documentation using tools like ChatGPT, GitHub Copilot, BlackboxAI, Tabnine, and Cursor

Click here to read more

Thanks for reading!!

I started this newsletter to share all the cool stuff I’m finding in AI over time. Keep updated on the latest stuff with this newsletter. I'm glad you're here.

Share this on your LinkedIn page if you think your friends might be interested in this stuff.

BTW Check out my YouTube Channel for more cool stuff with Generative AI. Tips, tutorials, reviews of AI tools and more.

Python Basic Syntax and Indentation: The Complete Beginner's Guide

Jeremy Morgan — Thu, 12 Dec 2024 00:21:51 +0000

When you're first learning to program, Python stands out for a special reason: it's designed to be read almost like English. Unlike other programming languages that use lots of symbols and brackets, Python relies on simple, clean formatting that makes your code look like a well-organized document.

Think of Python's syntax like the grammar rules of a language. Just as English has rules about how to structure sentences to make meaning clear, Python has rules about how to write code so both humans and computers can understand it.

Understanding Python's Basic Syntax

The Building Blocks

Let's start with the simplest elements of Python syntax:

# This is a comment - Python ignores anything after the '#' symbol
student_name = "Alice"    # A variable holding text (string)
student_age = 15         # A variable holding a number (integer)

# Using variables in a sentence (string formatting)
print(f"Hello, my name is {student_name} and I'm {student_age} years old.")

In this example, we're using several basic elements of Python:

Comments (lines starting with #)
Variables (student_name and student_age)
String formatting (the f"..." syntax)
The print function

Basic Operations

Python can perform calculations and comparisons just like a calculator:

# Basic math operations
total_score = 95 + 87    # Addition
average = total_score / 2 # Division

# Comparisons
if student_age >= 15:
    print(f"{student_name} can take advanced classes")

The Heart of Python: Understanding Indentation

Here's where Python gets truly unique: instead of using brackets or special symbols to group code together, Python uses indentation. This might seem strange at first, but it makes Python code exceptionally clear and readable.

How Indentation Creates Structure

Think of indentation like the way you might organize a detailed outline:

def make_sandwich():
    print("1. Get two slices of bread")  # First level
    if has_cheese:
        print("2. Add cheese")           # Second level
        print("3. Add tomatoes")         # Still second level
    else:
        print("2. Add butter")           # Second level in else block
    print("4. Put the slices together")  # Back to first level

Each indented block tells Python "these lines belong together." It's like creating a sub-list in an outline – everything indented under "if has_cheese:" is part of that condition.

The Rules of Indentation

Let's look at the key rules for Python indentation:

def process_grade(score):
    # Rule 1: Use exactly 4 spaces for each indentation level
    if score >= 90:
        print("Excellent!")
        if score == 100:
            print("Perfect score!")

    # Rule 2: Aligned blocks work together
    elif score >= 80:
        print("Good job!")
        print("Keep it up!")  # This line is part of the elif block

    # Rule 3: Unindented lines end the block
    print("Processing complete")  # This runs regardless of score

Nested Indentation: Going Deeper

As your programs get more complex, you'll often need multiple levels of indentation:

def check_weather(temperature, is_raining):
    # First level: inside function
    if temperature > 70:
        # Second level: inside if
        if is_raining:
            # Third level: nested condition
            print("It's warm but raining")
            print("Take an umbrella")
        else:
            print("It's a warm, sunny day")
            print("Perfect for outdoors")
    else:
        print("It's cool outside")
        print("Take a jacket")

Complex Structures and Indentation

Let's look at a more complex example that shows how indentation helps organize code:

def process_student_grades(students):
    for student in students:            # First level loop
        print(f"Checking {student['name']}'s grades...")

        total = 0
        for grade in student['grades']: # Second level loop
            if grade > 90:              # Third level condition
                print("Outstanding!")
            total += grade

        average = total / len(student['grades'])

        # Back to first loop level
        if average >= 90:
            print("Honor Roll")
            if student['attendance'] > 95:  # Another level
                print("Perfect Attendance Award")

Common Patterns and Best Practices

Handling Multiple Conditions

# Good: Clear and easy to follow
def check_eligibility(age, grade, attendance):
    if age < 18:
        return "Too young"

    if grade < 70:
        return "Grades too low"

    if attendance < 80:
        return "Attendance too low"

    return "Eligible"

# Avoid: Too many nested levels
def check_eligibility_nested(age, grade, attendance):
    if age >= 18:
        if grade >= 70:
            if attendance >= 80:
                return "Eligible"
            else:
                return "Attendance too low"
        else:
            return "Grades too low"
    else:
        return "Too young"

Working with Functions and Classes

class Student:
    def __init__(self, name):
        self.name = name
        self.grades = []

    def add_grade(self, grade):
        # Notice the consistent indentation in methods
        if isinstance(grade, (int, float)):
            if 0 <= grade <= 100:
                self.grades.append(grade)
                print(f"Grade {grade} added")
            else:
                print("Grade must be between 0 and 100")
        else:
            print("Grade must be a number")

Common Mistakes and How to Fix Them

Indentation Errors

# WRONG - Inconsistent indentation
if score > 90:
print("Great job!")    # Error: no indentation
    print("Keep it up!")   # Error: inconsistent indentation

# RIGHT - Proper indentation
if score > 90:
    print("Great job!")
    print("Keep it up!")

Mixing Tabs and Spaces

# WRONG - Mixed tabs and spaces (don't do this!)
def calculate_average(numbers):
    total = 0
    count = 0    # This line uses a tab
    for num in numbers:    # This line uses spaces
        total += num

Practice Exercise: Putting It All Together

Try writing this program to practice indentation and syntax:

def grade_assignment(score, late_days):
    # Start with the base score
    final_score = score

    # Check if the assignment is late
    if late_days > 0:
        if late_days <= 5:
            # Deduct 2 points per late day
            final_score -= (late_days * 2)
        else:
            # Maximum lateness penalty
            final_score -= 10

    # Ensure score doesn't go below 0
    if final_score < 0:
        final_score = 0

    # Determine letter grade
    if final_score >= 90:
        return "A", final_score
    elif final_score >= 80:
        return "B", final_score
    elif final_score >= 70:
        return "C", final_score
    else:
        return "F", final_score

# Test the function
score = 95
late_days = 2
letter_grade, final_score = grade_assignment(score, late_days)
print(f"Original Score: {score}")
print(f"Late Days: {late_days}")
print(f"Final Score: {final_score}")
print(f"Letter Grade: {letter_grade}")

Key Takeaways

Python uses indentation to understand code structure
Always use 4 spaces for each level of indentation
Be consistent with your indentation throughout your code
Simpler, flatter code structure is usually better than deeply nested code
Proper indentation makes code more readable and helps prevent errors

Next Steps

Now that you understand Python's basic syntax and indentation:

Practice writing simple programs focusing on proper indentation
Learn about different data types (strings, numbers, lists)
Explore functions and classes
Study loops and control structures
Start working with Python modules and libraries

Remember: Good indentation habits form the foundation of becoming a skilled Python programmer. Take your time to master these concepts, and the rest will follow naturally!

Understanding Python Syntax and Variables

Jeremy Morgan — Tue, 03 Dec 2024 18:57:44 +0000

Hey there, Python enthusiasts! If you’re diving into the world of Python or brushing up your skills, mastering Python’s syntax and variables is a fantastic place to start. Python is known for its simplicity and readability, making it a top choice for developers of all levels. In this guide, we’ll unravel the basics of Python syntax and variables with plenty of practical examples and best practices. So, grab a coffee (or your favorite beverage) and let’s dive in!

Why Python Syntax and Variables Matter

First things first—why should we care about syntax and variables in Python? Here’s the deal:

Readability: Python’s clean, intuitive syntax means less time decoding code and more time solving problems.
Efficiency: Proper use of variables keeps your code efficient and streamlined.
Debugging: A solid grasp of syntax helps you pinpoint errors faster than a debugger.
Scalability: Writing clear, organized code ensures that your projects can grow without turning into a tangled mess.

Convinced? Great. Let’s start with the basics.

Python Syntax Basics

Indentation: Python’s Secret Sauce

In Python, indentation isn’t just for looks—it’s how you define blocks of code. Forget braces ({}) and semicolons—just align your code with consistent spacing.

Here’s an example:

if True:
    print("Hello, Python!")

That’s it. The print statement is indented to show it belongs to the if block. Forget to indent, or mix spaces and tabs, and Python will call you out with a syntax error.

Comments: Talk to Your Future Self

Comments in your code are lifesavers when you revisit it months (or years) later. Python supports:

Single-line comments: Start with #.
Multi-line comments: Enclose with triple quotes (''' or """).

Here’s how:

# Single-line comment
"""
Multi-line comment
spanning several lines.
"""

Python is Case-Sensitive

Python distinguishes between Variable, variable, and VARIABLE. Keep this in mind to avoid pesky bugs.

Variables in Python

What Are Variables?

Think of variables as labeled storage containers for your data. Python is dynamically typed, so you don’t need to declare types upfront. Here’s a quick example:

x = 10  # Integer
y = 3.14  # Float
z = "Hello, World!"  # String

Naming Variables

To keep your code clean and readable, follow these rules:

Rules:
- Start with a letter or underscore, not a number.
- Use only letters, numbers, and underscores—no spaces or special characters.
- Avoid Python keywords like if, class, or def.
Conventions:
- Use snake_case (e.g., user_name).
- Choose meaningful names—score is better than s.

Assigning Values

Assigning values is as simple as:

a, b, c = 1, 2, 3  # Multiple assignments

Common Python Data Types

Here’s a rundown of Python’s built-in data types:

Numeric:
- int: Whole numbers (e.g., 42)
- float: Decimal numbers (e.g., 3.14)
Strings: Enclosed in single, double, or triple quotes:

  greeting = "Hello, Python!"

Booleans: True or False

  is_active = True

Lists: Ordered, mutable collections:

  fruits = ["apple", "banana", "cherry"]

Dictionaries: Key-value pairs:

  person = {"name": "Alice", "age": 25}

Performing Operations with Variables

Arithmetic

Python handles math like a champ:

x = 10
y = 3

print(x + y)  # Addition
print(x - y)  # Subtraction
print(x * y)  # Multiplication
print(x / y)  # Division

Strings

You can concatenate or repeat strings easily:

name = "Alice"
print(name + " Smith")  # Alice Smith
print(name * 3)  # AliceAliceAlice

Logical Operations

Logical operators (and, or, not) are super handy:

x = True
y = False

print(x and y)  # False
print(x or y)  # True
print(not x)  # False

Best Practices

Write clean, efficient Python by following these tips:

Descriptive Names: Use meaningful variable names.
DRY Principle: Don’t Repeat Yourself—reuse your code.
Follow PEP 8: Stick to Python’s style guide.
Comment Smartly: Explain why, not what.
Avoid Globals: Keep variables local to their functions when possible.

Common Pitfalls (And How to Avoid Them)

Indentation Errors: Stick to spaces or tabs (not both), and use four spaces per level.
Scope Issues: Know the difference between local and global variables.
Type Mismatches: Python doesn’t mix types:

   x = "5"
   y = 10
   print(x + y)  # TypeError

FAQ

Q: What’s the difference between variables and constants?

Variables can change; constants stay fixed. Use all caps to indicate constants (e.g., PI = 3.14).

Q: How can I check a variable’s type?

Use type():

x = 10
print(type(x))  # <class 'int'>

Q: Can I change a variable’s type?

Sure can! Python allows dynamic typing:

x = 10
x = "Now a string"

Wrapping Up

Mastering Python syntax and variables is your gateway to writing cleaner, more effective code. With practice, these basics will become second nature.

Questions? Leave them in the comments here!

The Ultimate Guide to Running Local LLMs on Your Mac

Jeremy Morgan — Thu, 17 Oct 2024 17:17:46 +0000

Hello friends! Want to run LLM (large language models) locally on your Mac? Here's your guide! We'll explore three powerful tools for running LLMs directly on your Mac without relying on cloud services or expensive subscriptions.

Whether you are a beginner or an experienced developer, you'll be up and running in no time. This is a great way to evaluate different open-source models or create a sandbox to write AI applications on your own machine.

We'll go from easy to use to a solution that requires programming.

Products we're using:

LM Studio: User-Friendly AI for Everyone
Ollama: Efficient and Developer-Friendly
Hugging Face Transformers: Advanced Model Access

If you'd rather watch a video of this tutorial, here it is!

So let's get started.

1. LM Studio: User-Friendly AI for Everyone

LM Studio is an excellent starting point for both beginners and experts. It provides an intuitive interface for exploring and using various AI models.

Getting Started with LM Studio

Visit lmstudio.ai and download the appropriate version for your Mac.
Install LM Studio by dragging the downloaded file into your Applications folder.
Launch LM Studio and accept any security prompts.

Exploring Models

In the main interface, click "Load a model" or select a model from the "New and Noteworthy" list.

we'll use the "llama2 3B" model for this tutorial. Click on it to download.
Once downloaded, click "Load model" to activate it.

Using the Chat Interface

With the model loaded, you can start interacting with it in the chat interface.
Try asking a question like "Tell me a funny joke about Python."

Observe the model's response and the performance metrics (tokens per second, context usage).

Utilizing the API Server

LM Studio also provides an OpenAI-compatible API server, making it easy to integrate with your applications:

Click on the server icon in the left sidebar.
Start the server by clicking the "Start Server" button.
Copy the provided server address (usually http://localhost:1234).

You can see a set of endpoints available:

GET http://localhost:1234/v1/models
POST http://localhost:1234/v1/chat/completions
POST http://localhost:1234/v1/completions
POST http://localhost:1234/v1/embeddings

You can now use this address to send requests to the model using tools like Postman or your own code. Here's an example using Postman:

Create a new POST request to http://localhost:1234/v1/chat/completions.
Set the body to raw JSON with the following content:

{
        "model": "lmstudio-community/Qwen2.5-14B-Instruct-GGUF/Qwen2.5-14B-Instruct-Q4_K_M.gguf",
        "messages": [
            {
                "role": "system",
                "content": "You are a helpful jokester who knows a lot about Python"
            },
            {
                "role": "user",
                "content": "Tell me a funny Python joke."
            }
        ],
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "joke_response",
                "strict": "true",
                "schema": {
                    "type": "object",
                    "properties": {
                        "joke": {
                            "type": "string"
                        }
                    },
                    "required": [
                        "joke"
                    ]
                }
            }
        },
        "temperature": 0.7,
        "max_tokens": 50,
        "stream": false
    }

Send the request and observe the model's response.

LM Studio is perfect for quickly testing different models and integrating them into your projects with minimal setup.

2. Ollama: Efficient and Developer-Friendly

Ollama is a lightweight and powerful tool for deploying LLMs, which is ideal for developers who prefer working from the command line.

Installing Ollama

Visit the Ollama website and download the Mac version.
Install Ollama by dragging the downloaded file into your Applications folder.
Launch Ollama and accept any security prompts.

Using Ollama from the Terminal

Open a terminal window.
List available models by running: Ollama list

To download and run a model, use: Ollama run <model-name>
For example: Ollama run qwen2.5-14b
Once the model is loaded, you can interact directly with it in the terminal.

Ollama Commands and Features

Use /? to see available commands within a model session.
Exit a model session with /bye.
Run models with verbose output using --verbose flag.

Using Ollama's API

Ollama also provides an API for integration with your applications:

Ensure Ollama is running (you'll see the icon in your menu bar).
Send POST requests to http://localhost:11434/api/generate.

Example using Postman:

{
      "model": "qwen2.5:14b",
      "prompt": "Tell me a funny joke about Python",
      "stream": false
}

Ollama Python Library

For Python developers, Ollama offers a convenient library:

Install the library: pip install ollama
Use it in your Python scripts:

import ollama
response = ollama.chat(model='qwen2.5:14b', messages=[
  {
    'role': 'user',
    'content': 'Tell me a funny joke about Golang!',
  },
])
print(response['message']['content'])

Ollama provides a great balance between ease of use and flexibility, making it an excellent choice for developers building AI-powered applications.

3. Hugging Face Transformers: Advanced Model Access

Hugging Face Transformers is a powerful library that gives you access to many models and more control over their usage.

Setting Up Hugging Face Transformers

Create a new Python virtual environment:

python -m venv env
source env/bin/activate

Install required libraries:

pip install torch transformers accelerate

Install Hugging Face CLI:

pip install -U huggingface_hub[cli]

huggingface-cli login

(You'll need to create a user access token on the Hugging Face website)

Using a Model with Transformers

Here's a simple example using the LLaMA 3.2 3B model:

import torch
from transformers import pipeline

model_id = "meta-llama/Llama-3.2-3B-Instruct"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Advantages of Hugging Face Transformers

Access to a vast library of models
Fine-grained control over model parameters
Ability to fine-tune models for specific tasks
Integration with popular deep learning frameworks

While Hugging Face Transformers requires more coding knowledge, it offers unparalleled flexibility and access to cutting-edge AI models.

Conclusion

We've explored three powerful tools for running AI models locally on your Mac:

LM Studio: Perfect for beginners and quick experimentation
Ollama: Ideal for developers who prefer command-line interfaces and simple API integration
Hugging Face Transformers: Best for advanced users who need access to a wide range of models and fine-grained control

Each tool has its strengths, and the choice depends on your specific needs and technical expertise. By running these models locally, you gain more control over your AI applications, ensure data privacy, and avoid the costs associated with cloud-based services.

Consider your Mac's specifications when working with larger models, as they can be resource-intensive. Start with smaller models and work up as you become more familiar with the tools and your hardware capabilities.

Happy coding, and enjoy exploring the world of local AI on your Mac!

If you have any questions or feedback feel free to reach out or leave a comment below.

The Top Conversational AI Models of 2024

Jeremy Morgan — Fri, 30 Aug 2024 20:02:19 +0000

Like you, I use Generative AI nearly every single day. One common request I get is when folks have wet their feet with ChatGPT and ask me "What else is out there?" I send them a list of links and a little blurb about the sites. I decided to share that list and insights with you.

Bookmark this page because it will be updated frequently. I'll also do this with some other tools I use (image generation, code generation, video, etc.).

Am I missing anything? Feel free to reach out to me!

Here are the top conversational AI tools out there right now.

Perplexity

URL: https://www.perplexity.ai/
Title: Perplexity AI
Description: Perplexity - Where Knowledge Begins
Why it's awesome: It pulls from different sources to verify the information presented. Very magic.

Claude

URL: https://claude.ai
Title: Claude
Description: Talk with Claude, an AI assistant from Anthropic
Why it's awesome: Excels in writing, and analysys.

Globe Explore

URL: https://explorer.globe.engineer/
Title: Globe Explorer
Description: Find everything humans know about...
Why it's awesome: It pulls from many different sources to really deep dive into a subject.

Poe

URL: https://poe.com/
Title: Poe - Fast, Helpful AI Chat
Description: Poe lets you ask questions, get instant answers, and have back-and-forth conversations with AI. It gives access to GPT-4, GPT-3.5-turbo, Claude from Anthropic, and various other bots.
Why it's awesome: There are so many connections to other services, and the interface is very cohesive.

Grok

URL: https://x.com/i/grok
Title: x.com
Description: AI Assistant, powered by X
Why it's awesome: It uses X as an additional source, and it is very up to date.

Gemini

URL: https://gemini.google.com/app
Title: Gemini - chat to supercharge your ideas
Description: Bard is now Gemini. Get help with writing, planning, learning, and more from Google AI.
Why it's awesome: Backed by Google, exciting and factual results.

Groq

URL: https://groq.com/
Title: Groq is Fast AI Inference
Description: The LPU™ Inference Engine by Groq is a hardware and software platform that delivers exceptional compute speed, quality, and energy efficiency. Groq provides cloud and on-prem solutions at scale for AI applications.
Why it's awesome: Screaming fast and smart, great for applications.

ChatGPT

URL: https://openai.com/chatgpt/
Title: ChatGPT by OpenAI
Description: A conversational AI system that listens, learns, and challenges
Why it's awesome: It's the OG, and Custom GPTs are awesome

Hugging Face Chat Models

URL: https://huggingface.co/chat/models
Title: HuggingChat - Models
Description: The first open source alternative to ChatGPT.
Why it's awesome: A wide variety of models to choose from

Bing AI (Copilot)

URL: https://www.bing.com/chat
Title: Microsoft Copilot in Bing
Description: Chat, get answers, create amazing content, and discover information effortlessly with Bing's AI-powered chat. Transform the way you search and get answers with Microsoft Copilot in Bing.
Why it's awesome: Fast and great for programming/coding stuff.

Conclusion

This is the list of Conversational AI models I use constantly. I will update this as I go. If you can think of one I'm missing out on, Contact me!