DEV Community

ijayasurya
ijayasurya

Posted on • Originally published at oksurya.com

Ollama + Apple Shortcuts + Voice Control (Hey Mac! 🚀)

#ios #apple #shortcuts #ai

Hey there, fellow tech explorer! 👋 Ever wished your Mac could just listen and do what you ask? Like:

"Hey Mac, generate an AppleScript for me!"

Well guess what? You can make that happen — without any cloud magic. It's all local, private, and totally in your control 😎

Let me walk you through how I created a simple AI agent using:

  • 🦙 Ollama (running AI models locally)
  • 🍏 Apple Shortcuts (automation awesomeness)
  • 🗣️ Voice Control (so I can just speak commands)
  • 🎩 Custom phrase: “Hey Mac!”

🎯 What We’re Gonna Build

"Hey Mac, write an AppleScript to mute the volume."

This command will:

  1. Use your voice to trigger a shortcut
  2. Send the task to Ollama
  3. Return just the AppleScript
  4. Run it automatically!

Boom. Magic. 🔮✨

🛠️ Step-by-Step Guide

1️⃣ Install Ollama & Run a Model Locally

Go to ollama.com and install it.

Then, open Terminal and run:

ollama run llama3
Enter fullscreen mode Exit fullscreen mode

(Or any model you like — Mistral, Llama 2, etc.)

It’ll start a local API at:

📍 http://localhost:11434/api/generate

2️⃣ Create the Shortcut in Apple Shortcuts

Here’s what the shortcut does (you saw this in the screenshot):

  • 🎙️ Dictate Text – You speak the task
  • 📝 Sends: "Return only valid AppleScript code. Do not explain. The task is: <your task>"
  • 🌐 GET request to: http://localhost:11434/api/generate
  • 🪄 Extracts the pure AppleScript (removes backticks and tags)
  • 🖥️ Runs it using osascript
  • ✅ Done!

3️⃣ Connect Voice Control with Custom Command

Here’s the cherry on top 🍒

  1. Go to System Settings > Accessibility > Voice Control
  2. Turn it ON
  3. Click Commands…
  4. Click ➕ to create a new command:
    • When I say: Hey Mac
    • While using: Any app
    • Perform: Run Shortcut → Select your shortcut (e.g., "AutoLama")

Now say:

"Hey Mac"

It triggers your shortcut 🎤🤖

😍 Example in Action

Let’s say I want to create a new folder on my Desktop. I say:

“Hey Mac, create a folder named LlamaMagic on desktop.”

Your shortcut will:

  • Send that to the local LLM
  • Receive: 
  tell application "Finder"
      make new folder at desktop with properties {name:"LlamaMagic"}
  end tell
Enter fullscreen mode Exit fullscreen mode
  • Run it instantly!

You just spoke an idea and your Mac obeyed like a friendly genie 🧞‍♂️

🧠 Bonus Tips

  • You can change the prompt to ask for shell scripts, Python, or whatever!
  • Add more automation, like asking for confirmation before running
  • Use clipboard instead of voice if you're shy 😅

🥳 Final Thoughts

This was super fun to build, and it makes your Mac feel like a personal AI assistant. Everything is local — no internet, no tracking, no middleman!

All you need is:

  • Ollama 🦙
  • A Shortcut ⚡
  • Voice Control 🎙️
  • A little creativity 💡

Now go impress your friends and make your Mac say "Your wish is my command." 😄

If you liked this guide, share it with your geeky friends 🧙‍♀️ and tell them:

“You don’t need Siri. You need Shortcuts + Ollama!”

Need help setting it up? Drop your questions below! 💬👇

Top comments (3)

Collapse
 
olivero profile image
Oliver O •

Can it be run in a sandbox?

Collapse
 
jayasurya_mailsamy profile image
Jayasurya Mailsamy •

no ,its for mac computer only

Collapse
 
olivero profile image
Oliver O •

It sounds interesting. Are you planning to open-source the project?