Building IPAC - The AI I Want to Live With
I've spent a lot of time thinking about what I actually want from an AI assistant.
Not what sounds impressive in a product announcement.
Not another chatbot with a nicer voice.
Not an app that can answer questions, generate an image, or remind me to buy milk.
Those things are useful. I've used them. I'll probably keep using them.
But every time I use one, I end up thinking about the same thing:
Is this really what a personal AI is supposed to be?
Because when I imagine having my own AI, I imagine something different.
I imagine an intelligence that gradually becomes familiar with me.
Something that knows what I'm working on, remembers conversations that actually matter, understands what I'm trying to accomplish, can use the tools around me, and doesn't need me to explain the entire world to it every time we talk.
Something that feels less like opening an application and more like continuing a relationship.
That idea has been sitting in my head for a while.
And eventually I decided to stop wondering whether someone else was going to build it.
I'm going to try.
I'm calling it IPAC.
I Think Fiction Got One Thing Right
Some of the best ideas about personal AI haven't come from real products.
They've come from fiction.
One of the examples that stayed with me was AIA from *Afraid*.
If you've seen the movie, you know AIA isn't presented as just another voice assistant.
She's connected to the home.
She observes.
She learns the family's routines.
She understands things happening around her.
She can interact with the environment.
And, importantly, she isn't constantly waiting for someone to formulate the perfect command.
She's there.
That's what caught my attention.
Not the horror.
Not the fact that everything eventually goes very wrong.
The idea underneath it.
What would it actually feel like if your home had an intelligence living inside it?
An intelligence that understood the environment instead of simply controlling a few smart devices.
That could notice things.
That could remember things.
That could connect events together.
That could help without being explicitly told every tiny step.
The movie takes that idea somewhere I definitely don't want to go.
But the starting point is fascinating.
And I kept thinking:
Why does this still feel so far away from the assistants we actually use?
Then I Found Daphne
Daphne, from American Horror Stories, gave me another piece of the idea.
Daphne isn't interesting to me because of where her story eventually goes either.
It's the relationship.
She isn't just a machine you give commands to.
She becomes part of the user's life.
She talks to him.
She remembers.
She knows things about him.
She develops an understanding of his world.
Again, the show pushes the concept into horror.
But underneath that is another question I've always found interesting:
What happens when an AI knows you for a long time?
Not because you uploaded a giant profile about yourself.
Not because you manually configured hundreds of preferences.
But because you've actually lived alongside it.
You talked to it yesterday.
You talked to it six months ago.
It helped you with something important.
You told it about a project.
It watched that project evolve.
It remembers why you made certain decisions.
It knows what you're currently doing.
And when you come back tomorrow, you don't have to start from zero.
That continuity is something I really want.
We're Getting Closer
The interesting thing is that this isn't purely science fiction anymore.
Companies are clearly moving in this direction.
Amazon's Alexa+ is a good example.
The vision around Alexa+ is much bigger than the original voice assistant. Amazon describes it as conversational and personalized, with the ability to remember context, work across devices, interact with services, and take actions on a user's behalf.
That's important.
Because the industry is starting to move away from:
"Ask a question. Get an answer."
toward:
"Tell the assistant what you want, and let it figure out how to accomplish it."
That's a huge shift.
And honestly, I like it.
I think it's where assistants should go.
But when I look at all of this, I still feel like something is missing.
Maybe it's because I'm approaching the problem from a slightly different direction.
I'm not particularly interested in building another assistant product.
I'm interested in building the thing behind the assistant.
I Don't Want Ten Assistants
Think about what happens today.
You might have an AI chatbot for conversation.
A coding agent for programming.
A smart-home assistant for your house.
A calendar assistant.
An automation platform.
A research agent.
A browser agent.
A messaging bot.
A collection of scripts.
Maybe a few APIs you've connected yourself.
They're all useful.
But eventually you end up becoming the manager of your own AI ecosystem.
You have to decide which tool should do what.
You have to move information between systems.
You have to explain context again.
You have to keep track of what is running.
And sometimes you end up thinking more about how to use the AI than what you originally wanted to accomplish.
I don't think that's the final form.
I want something simpler.
I want to be able to say:
"I need this done."
And have the assistant figure out what that means.
Not perfectly.
Not magically.
But intelligently enough to begin working out the problem.
That's Where IPAC Comes In
IPAC stands for:
Interactive Personal Assistant with Agentic Capabilities.
The name is deliberately boring.
The idea isn't.
The basic philosophy is:
Workers work for IPAC.
IPAC works for the human.
That distinction is extremely important to me.
I don't want one enormous AI trying to do everything.
I want specialized workers.
One can search.
One can write code.
One can monitor something.
One can control a device.
One can research.
One can interact with an API.
One can perform a long-running task.
One can analyze information.
But those workers shouldn't become independent personalities competing for my attention.
They should be tools available to the assistant.
IPAC coordinates them.
IPAC understands what I'm trying to accomplish.
IPAC decides what needs to happen.
The workers do the work.
And IPAC brings the result back to me.
The Part That Sounds Small But Isn't
There's another rule I want to build into IPAC:
The assistant doesn't have to talk all the time.
This might sound obvious.
It isn't.
Imagine having twenty agents working in the background.
One finishes something.
Another discovers something.
Another encounters an error.
Another notices an event.
If all twenty immediately message you, you've created a very sophisticated way of being annoyed.
I don't want that.
I want IPAC to have a concept of value.
Something can happen without needing to become a notification.
A worker can finish its task and remain silent.
An event can be interesting but irrelevant.
Something can matter eventually but not right now.
And sometimes the right response is simply:
wait.
That's something I'm calling the Silence Principle.
If IPAC doesn't have something valuable to contribute, it should be comfortable being quiet.
I Want It To Know When I'm There
This leads to another part of the project that I find especially interesting:
presence.
I don't want IPAC to think of me as a message that arrived in an API request.
I'm a person.
Sometimes I'm actively talking to it.
Sometimes I'm working.
Sometimes I'm away.
Sometimes I'm focused.
Sometimes I'm in a call.
Sometimes I'm driving.
Sometimes I'm asleep.
Those states matter.
If I'm deeply focused on something, I probably don't want an unnecessary interruption.
If something genuinely important happens, I might.
If I'm away, the assistant might wait.
If I'm actively talking to it, context changes again.
This is why I don't think a personal AI is just a language model with tools.
It's a system that exists in time.
And Then There's Memory
This is probably one of the areas I care about most.
I don't want IPAC to have a giant folder called "memory" and dump everything into it.
That's not really remembering.
Human memory isn't a transcript of every sentence we've ever heard.
Some things matter.
Some things don't.
Some things matter for five minutes.
Some things matter for years.
Some things are relevant to one project but completely irrelevant to another.
So IPAC needs to learn the difference.
It needs to know what happened recently.
It needs to know what I'm currently working on.
It needs to understand longer-term things.
It needs to remember important decisions.
It needs to maintain continuity.
And it needs to retrieve the right information when it actually becomes relevant.
The goal isn't:
"My AI remembers everything."
The goal is:
"My AI remembers what matters."
One Assistant, Everywhere
There's another part of this dream that is difficult to explain until you experience the problem yourself.
I don't want a phone assistant.
A laptop assistant.
A home assistant.
A browser assistant.
A Telegram bot.
A desktop AI.
I want IPAC.
The phone should just be one way of talking to it.
The laptop should be another.
The browser should be another.
Voice should be another.
My home should be another.
The interface changes.
The relationship doesn't.
If I start something on my laptop, I should eventually be able to continue it from my phone.
If IPAC starts doing something in the background, I shouldn't need to know which machine is currently handling it.
The system should feel like one thing.
This Is Where My Dream Becomes an Engineering Problem
And this is probably the most important thing about this project.
I'm not pretending I know how to solve all of this.
I don't.
That's actually part of why I'm building it.
There are enormous problems hiding underneath these ideas.
How should memory work?
How should context be selected?
How do agents communicate without creating chaos?
How do you prevent duplicate actions?
How do you handle failures?
How do you know whether an agent actually completed something?
How do you give an AI autonomy without giving it unlimited authority?
How do you make it proactive without making it annoying?
How do you maintain one relationship across multiple devices?
How do you make the system reliable enough that you can actually trust it?
And perhaps the biggest question:
How do you build something that feels alive without pretending that it is a human?
I don't have all the answers.
I'm building my way toward them.
I Don't Want a Digital Human
This is important.
The goal isn't to create a fake person.
I don't need IPAC to pretend to have a childhood.
I don't need it to pretend it has feelings.
I don't need some artificial personality constantly reminding me that it is "your best friend."
What I want is something more grounded.
A capable intelligence with a consistent identity.
A system that understands its role.
A system that knows what it can do.
A system that knows what it cannot do.
A system that remembers its relationship with its user.
A system that can act.
A system that can learn from what happens.
And a system that ultimately understands one very important thing:
It exists to serve the human, not the other way around.
The Controlled Dream
I call this a dream, but I don't want it to be an uncontrolled one.
That's probably the most important distinction.
It's easy to imagine an AI that can do everything.
It's much harder to build one that knows when it shouldn't.
It's easy to imagine unlimited autonomy.
It's much harder to design boundaries that make autonomy useful.
It's easy to say:
"My AI will handle everything."
It's much harder to answer:
"What exactly is it allowed to handle?"
That's why I'm not interested in building some mysterious black box that I simply turn loose.
I want to understand the system.
I want to know what it is doing.
I want to know why something happened.
I want to be able to control it.
I want to be able to change it.
I want to be able to shut something down when necessary.
The dream is big.
The engineering has to be disciplined.
So I'm Building It
This is the beginning of my IPAC journey.
I don't know exactly where it will end.
Maybe some ideas will fail.
Some architectures will probably get thrown away.
There will be bugs.
There will be models that don't behave the way I expect.
There will be integrations that break.
There will be days where something that sounded brilliant on paper turns out to be a terrible engineering decision.
That's part of it.
I'm not writing this blog because I already have the perfect personal AI.
I'm writing it because I'm trying to build one.
I want to document the process honestly.
The architecture.
The experiments.
The failures.
The breakthroughs.
The strange problems that don't have obvious solutions.
The decisions that turn out to be wrong.
And hopefully, eventually, the moments where IPAC does something and I stop for a second and think:
Okay. That's actually starting to feel like the thing I imagined.
That's what I'm chasing.
Not a chatbot.
Not a collection of agents.
Not a smart speaker with a bigger model.
Not an AI that exists to impress people for five minutes.
I want to see if I can build something that can become a genuine personal intelligence layer around a human life.
Something that can work beside me.
Something that can remember.
Something that can act.
Something that can stay quiet.
Something that can grow with the person using it.
And most importantly:
something that remains under that person's control.
That's IPAC.
And this is day one.
Top comments (1)
seems interesting