Book Review: Build Applications with Local AI Models on a Mac

Feedback and synthesis on of my latest readings: “Build Applications with Local AI Models on a Mac”

Introduction

As cloud-based AI infrastructure and API subscription costs continue to rise, running Large Language Models (LLMs) locally on consumer-grade hardware has transitioned from a niche hobby to a compelling architectural shift. The primary drivers behind this movement are privacy and economy. From a privacy perspective, processing data locally guarantees that confidential text, proprietary source code, internal business records, and personally identifiable information (PII) never leave your physical device. There are no remote logging pipelines, third-party data retention policies, or potential cloud leaks to worry about. On the economic front, local execution removes pay-per-token API metering entirely. Once you own the hardware, running inference across millions of tokens carries zero incremental API cost, making local models extremely cost-effective for high-volume tasks, continuous agent loops, embedding generation, and rapid local prototyping.

To be clear, one cannot do all with local LLMs in my opinion, but it helps for some tasks…

I, myself use Ollama on a regular basis with different models for agents, embeddings, etc. I was curious to read this book and figure out whether I would learn something new or not.

Disclaimer: I have no ties, commercial relationships, or affiliations with Manning Publications (manning.com) or the author of this book, Keiji Kamigusa. This review and breakdown are purely independent.

All images are from the book’s author.

---

Part 1: First Steps — Entering the World of the Terminal

Chapters 1 to 6 are really basic, destinated for beginners! You can ignore big parts of them.

Chapter 1: Getting Started with Local AI

This introductory chapter establishes the conceptual framework for local AI development on Apple Silicon and Intel Macs. It introduces how local inference works on modern hardware — highlighting the unified memory architecture (UMA) of Apple’s M-series chips — and compares local runtime environments against traditional cloud APIs in terms of latency, privacy, and cost predictability.

Chapter 2: Installing and Using Homebrew

Focusing on developer tooling, this chapter guides us through setting up Homebrew, the standard package manager for macOS. It covers terminal basics, environment path setup, and how command-line utilities form the foundational stack for managing dependencies in local machine learning projects.

Chapter 3: Installing and Setting Up Ollama

Here, the book dives into Ollama, the central engine used throughout the book to serve local models. It covers installation, service startup, verifying running instances via terminal commands, and managing local model storage.

# Pulling and running a lightweight model locally via Ollama terminal CLI
# for example...
ollama run granite3.2

Part 2: Core Skills — Building Your First AI Chatbot

Chapter 4: Downloading an LLM and Your First Conversation

This chapter demonstrates basic CLI interactions with quantized open-weights models (such as Granite, Llama, Gemma, and Qwen). We learn how to pull models from the registry, run interactive shell sessions, and monitor memory usage during inference.

Chapter 5: Setting Up VS Code and Your Python Development Environment

Moving from CLI usage to application development, this chapter covers configuring Visual Studio Code, installing Python, and structuring a clean directory workspace for building custom AI tools.

Chapter 6: Creating and Managing Python Virtual Environments

To avoid package dependency conflicts, this chapter explains Python virtual environments (venv). It covers environment creation, activation, and managing dependency lists with requirements.txt.

# Creating and activating a clean Python virtual environment
python3 -m venv .venv
source .venv/bin/activate
pip install ollama streamlit

Chapter 7: Controlling LLMs via the Ollama Python API

This chapter introduces programatically driving local models using the official ollama Python client. It demonstrates synchronous text generation, chat completions, and real-time response streaming.

import ollama

# Querying a local model programmatically with streaming enabled
response = ollama.chat(
    model="gemma2:2b",
    messages=[{"role": "user", "content": "Explain vector embeddings in one sentence."}],
    stream=True,
)

for chunk in response:
    print(chunk["message"]["content"], end="", flush=True)

Chapter 8: Building a Web UI with Streamlit

Demonstrating how to wrap Python backend code into an interactive graphical web interface using Streamlit. The chapter demonstrates rapid prototyping of UI elements like text inputs, buttons, and message display blocks.

Chapter 9: Recording Audio and Transcribing Speech with MLX Whisper

Expanding beyond text input, this chapter integrates Apple Silicon-optimized speech recognition using MLX Whisper. It details audio input capture and converting spoken voice into clean text transcripts locally. (I liked this chapter 🙂 and the following one)

Chapter 10: Building a Voice-Enabled AI Chat Application

Combining speech recognition with LLM generation, this chapter walks through building a hands-free, voice-driven AI conversational agent that listens, transcribes, processes queries, and outputs responses.

Chapter 11: Managing Session State and Chat History

Personally I really appreciated this part of the book!

To handle multi-turn conversations, this chapter addresses managing state inside Streamlit applications using st.session_state. It demonstrates how to maintain chat history, trim context windows, and calculate token counters.

import streamlit as st

# Managing chat state across Streamlit reruns
if "messages" not in st.session_state:
    st.session_state.messages = []

# Displaying chat history
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

Part 3: Advanced Topics — Deeper Understanding and Customization

Chapters 12 and 13 (part 3 of the book) interested me personally the most.

Chapter 12: Comparing and Selecting LLM Models

This chapter provides a comparative framework for evaluating open-weight models (e.g., Granite4, Gemma 3, Qwen 2.5, Phi-3). It contrasts parameter counts, context window sizes, quantization impact, and inference speed benchmarks across different hardware setups.

import time
import ollama

# Benchmarking inference speed across multiple local models
models = ["gemma3:4b", "qwen2.5:3b", "phi3:mini"]
prompt = "What are three key benefits of local AI inference?"

for model_name in models:
    start_time = time.time()
    response = ollama.chat(
        model=model_name,
        messages=[{"role": "user", "content": prompt}],
    )
    elapsed = time.time() - start_time
    print(f"Model: {model_name} | Response Time: {elapsed:.2f}s")

Chapter 13: System Prompts and Parameter Tuning

Focusing on output control, this chapter details how to shape model behavior using system prompts (assigning personas or strict rules) and adjusting hyperparameters like temperature to control randomness.

import ollama

# System prompt persona assignment and parameter tuning
response = ollama.chat(
    model="gemma2:2b",
    messages=[
        {"role": "system", "content": "You are a concise expert. Answer strictly in bullet points."},
        {"role": "user", "content": "Summarize the benefits of virtual environments."},
    ],
    options={"temperature": 0.2},
)
print(response["message"]["content"])

Chapter 14: Offline LLMs and Their Benefits

This chapter focuses on air-gapped operations, demonstrating how locally hosted models function entirely without active internet connectivity — ensuring high security and complete system resilience against remote network failures.

Chapter 15: Common Errors and Troubleshooting

A practical reference chapter covering common execution issues, including missing package dependencies, port binding errors, out-of-memory (OOM) GPU allocation failures, and Ollama daemon connectivity fixes.

Chapter 16: Advancing to RAG, Web UI, and Fine-Tuning

This chapter introduces advanced application design patterns, primarily focusing on Retrieval-Augmented Generation (RAG) using vector stores like ChromaDB, agent tool integrations with LangChain, and high-level interfaces like Open WebUI.

Personally, I prefer other vector databases such as AstraDB or Milvus, but the book focuses on ChromaDB.

import chromadb
import ollama

# RAG setup: Injecting context into local LLM query
client = chromadb.Client()
collection = client.create_collection("my_knowledge")

collection.add(
    documents=["Ollama runs LLMs locally on macOS, Linux, and Windows."],
    ids=["doc1"],
)

results = collection.query(query_texts=["Where can Ollama run?"], n_results=1)
context = "\n".join(results["documents"][0])

prompt = f"Context:\n{context}\n\nQuestion: Where can Ollama run?\nAnswer:"
response = ollama.chat(
    model="gemma3:4b",
    messages=[{"role": "user", "content": prompt}],
)
print(response["message"]["content"])

Chapter 17: The Open Model Revolution of 2026

Taking a broader architectural view, this chapter explores the landscape of open-weight artificial intelligence models, discussing open ecosystem standards, model safety guardrails, and hardware developments shaping modern on-device execution.

Chapter 18: Where to Go from Here

The concluding chapter provides a roadmap for further developer exploration, suggesting paths toward custom agent orchestration frameworks, fine-tuning techniques, and deployment options beyond local development environments.

Conclusion & Personal Opinion

Build Applications with Local AI Models on a Mac provides a practical, step-by-step introduction to running LLMs on consumer hardware and integrating them into functional Python applications. By guiding the readers through terminal setup, Ollama configuration, Streamlit web UI creation, speech transcription via MLX Whisper, and foundational RAG patterns using ChromaDB and LangChain, the book serves as a clear starting guide for on-device AI integration.

Personal Opinion

Overall, this was mostly an enjoyable read . It is particularly well-suited for novices, beginners, or developers who are new to running models locally using Ollama and want a structured, practical path from terminal installation to building functional UI wrappers and voice agents. While experienced developers already familiar with agentic loops, custom embedding pipelines, and low-level quantization mechanics might find much of the initial material elementary, it remains a clean, cohesive refresher. The accompanying code repository provided with the book is well-organized, making it easy to follow along, test code snippets, and quickly adapt the examples for personal experiments.

Thanks for reading 📔

Links

• The book on editor’s site: https://www.manning.com/books/build-applications-with-local-ai-models-on-a-mac