This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
Ever wish you could talk to a deceased loved one? Pretty sure there's a Black mirror episode on that. I tried to do something similar. Training on an actual dead person's conversations sounds like an ethical conundrum that I don't want to push my luck with in this post. So instead, I had to take a few thousand conversations of mine and fine tuned a few open-weight models on top of it.
Why? So that my humor outlives me.
Demo
Code
https://github.com/Arghyann/helloGrandma
How I Built It
I gathered my conversations from Whatsapp, Signal and Instagram which gave me about 100k question answer pairs. After a bit of digging around, I had the disappointing realization that most of my texts weren't profound and that I was an unusually dry texter. So i trimmed down the dataset to 30k long-ish messages with a few short messages here and there. I trained Qwen 14B on an Nvidia L4 for 4 epochs. That took about 10 hours.
I tried a few other approaches. I've added scripts and metrics along with the code. I changed models, tried tweaking hyperparameters, trained it on some long form content I'd written. Lastly, I tried training Qwen 14B on 1.5k pairs of just funny things I'd said. I did this by filtering texts that were followed by laughing emojis and other things. Did this mean i was funny only 1.5% of the time? Remember that the data depends on the other person laughing or even understanding all of my other comedic masterpieces. The actual number is far higher I'd like to believe. During the second world war, allies wanted to add more armor to the parts of the plane that were the most damaged of the planes that returned. A statistician pointed out that they should armor parts that weren't hit since those were the planes that never made it back. I believe a similar thing is true for my funny percentage.
All the training and inference was done on Modal. I made a small Go API and deployed it on Render along with the front end. When someone types a prompt in the frontend, it would send a request to the backend which would wake a GPU on Modal for inferencing. I also got hellograndma.tech and hosted the frontend on there for jokes.
Now since the models were trained on my personal conversations, giving away access to everyone on the internet would be unwise. If the judges contact me however, I'm willing to give them invite codes. So, I'll only share snippets of my conversations on the domain.
Why Does Open Innovation Matter?
A few years ago, building a project like this would've seemed impossible. Training a large language model is an incredibly hard and expensive task to do from scratch. It would require an infeasible amount of compute and data. Open weight models allow individuals like me to take already trained models and fine tune them on personal tasks. The barrier was low enough that I could give it a shot.
Also, like I've mentioned, the model were trained on years of private data between me and the people I know. Handing it over to a model company made me uncomfortable. Training on a rented GPU with storage is a real benefit.
Another thing about open innovation is that it lets people use existing technologies and extend it in ways the creators could never have anticipated. The goal for me was not really to make anything with real world utility. I just wanted to see how close I could get a model to pick up on my humor and other things that make me recognizable for people I know.
That's probably my favorite part of Open Innovation: the ability to take personal curiosities and turn them into fun experiments without a whole data center or a research lab.
The Spirit of It
I gave access to my girlfriend who was kind enough to let me use some of our conversations for training. This is what it said to her:
Her: What kind of women are you into?
Model: You
One unintended consequence of most of my conversations coming from my girlfriend was that the model was a flirt. Also, it was moody. I have mixed feelings about that.
Prize Categories
Best Use of Render
Used a Render Blueprint to deploy both the frontend and the Go API. The database is a SQLite file on a persistent disk. Render runs a cheap always on machine that wakes up a GPU everytime someone wants to talk to a model, so i don't have to pay for an always on GPU.
Best Use of a .tech Domain
My grandma would've loved it.
Top comments (1)
I wish i could eat your brain.