TL;DR: I'm experimenting with creating a cognitive structure using HDC as its foundation. Here's an explanation of my thoughts on the matter, and below is a link to a README with a more technical explanation of how the system works.
A Little Backstory
I'd like to explain my thought process so my thoughts seem a little less abstract.
Here I'll talk about something I'm still actively creating, and I don't have a demo yet. I became fascinated with this thing almost nine months ago, when I decided to give Character AI a try as a casual game, as my peers were so into it. And... for me, it turned out to be incredibly boring. At that point, I became curious about how such a thing works, and I started studying neural networks.
At first, of course, I started with simple topics, trying to understand what Markov chains are. But this quickly gave way to a more interesting topic: Transformers. That's when I really got hooked; it was extremely interesting and enjoyable to study. Then I was captivated by the exciting idea: would I manage to write my first Transformer before I turned 18? In the end, I was only a day late, haha. But this topic, too, seemed... Simple at its core? That wasn't enough for all the magic emanating from the black boxes of neural networks.
And then a silly thought occurred to me. "What if we give a neural network emotions as numbers from 0 to 1?" So banal. In my first attempt to recreate this idea, I quickly gained momentum in what I wanted: hormones, simulating parts of the brain, morality... I plagued Obsidian with notes on over 60 modules for such a venture, ahaha. Then I changed direction: I wanted to simulate the psyche, not the brain. And again, tons of notes, months of thought and reflection.
But I had to give up. Writing a system so voluminous in the number and complexity of modules is simply unrealistic for a teenager like me, right? At least that's what someone told me in a Telegram chat called Real Programmers. But I didn't take this as a call to stop; on the contrary, I was captivated by the excitement—could I simplify the system to a realistic scale? Could I even create what I had in mind?
From that moment on, I pondered a lot before the main idea struck me: I needed thinking, not a simulation of the brain or psyche. And thinking is built on three pillars (as I thought then): narrative memory, subjective perception, and those very priors that the creator of the ARC test, who had so captivated me back then, had spoken of. I wasn't aiming for AGI, and I still am, but I was captivated by the idea that a built-in ability to perceive reality was essential. I still agree with that.
And after that, I simply forgot about the project for a while. Due to life's events, I simply upped and gave up. But about a month and a half ago, something prompted me to revisit what I had been doing. And I had an epiphany.
For over eight months, I've been observing how I think. So why should I try to replicate the psyche as it works? I can imitate the processes it creates! And then I was swept away by a wave of inspiration and a sudden, clear understanding of how to build this project.
How can you turn HDC into 'thinking'?
As you can see, I put the word 'thinking' in quotation marks, which is very important. This is an imitation of processes, not a claim that I'm creating something that actually thinks like a human being. But an imitation of that—yes, that's true.
If you grasp the idea that thinking is a process, a multitude of possibilities opens up. Now, looking back, I thought thinking was built on three things: narrative memory, subjective experience, and priors. But now I've changed my mind slightly: what if thinking is a process based on two things—associative memory and subjective perception? Yes, priors are important, but essentially, associative memory and subjective perception are those very same priors.
And what does this give us? It gives us a fantastic chance to imitate human thinking! Recreate emotions, hormones, bodily sensations, hearing, but as state parameters, like in a game. But the main trick: all states are interdependent and influence how received information is remembered. With what speed, confidence, and what kind of 'emotional trace' is the information stored. For what purpose? So that even the same input is perceived by the neural network in a new way each time.
But that's not enough!
And you're right. Thinking isn't just a state, as I said; associative memory is also necessary. In my architecture, the memory alone consists of 14 modules. That's an insanely large number, right? But each part processes a specific piece of information and does so in its own way; they can't even be created according to a template, let alone combined into one. Because memory isn't a static storage of information. So how do we solve this understanding of memory?
And that's where HDC comes in. This ingenious system is ideal for associative connections! And that's our key to this very memory. Now, just think about what happens if information is processed uniquely each time, and the system's state affects the proximity of vectors in HDC memory? We'll get a system capable of gaining experience.
So what?
Actually, I agree, experience itself isn't particularly powerful. But think about how you learned something yourself. You made mistakes. But you had reasons to continue. "This is a matter of my future career!" or even "Mom will be very angry if I get a bad grade!" Mistakes are your way of finding direction in the labyrinth. And someone who knows everything from the start... They will never improve.
But simple mistakes aren't quite enough. The suffering from mistakes is the purest motivation. Only by feeling the pain of a burn will you stop touching fire. In other words, you learn to avoid what causes harm. This is easily replicated in code.
In Conclusion
I think I've said enough to explain what I do. And I would be incredibly happy if there were someone I could discuss this with! Thank you for reading. You can also read more technically about how I create this system in this README on GitHub.
Top comments (0)