Three weeks ago, I wrote about knowledge collapse - how our best technical insights are dying in private AI chats while Stack Overflow bleeds 78% o...
For further actions, you may consider blocking this person and/or reporting abuse
Brilliant! As @richardpascoe said: Quite simply mind blown! This is the kind of article I love to read and share: a problem and its solution.
appreciate that pascal.
"problem and its solution" is exactly the structure i was going for. too many articles identify problems without shipping anything.
if you share it, curious what audience youre thinking - linkedin? twitter? developer communities?
always learning what resonates
You're welcome Daniel! I share it with some developpers I use to work with, for example… I've also took it in my own wallabag list, to get it at hand when needed.
“Absolutely! Articles like this are gold—clear problem, actionable solution, and real insights. Definitely worth sharing with anyone who wants to learn something meaningful.”
Done & dusted in 3 weeks - that's quicker than it takes most companies to come up with a rough outline of a plan, haha - of course you did benefit from your prior experience with the techniques that you mentioned (as per your previous dev.to articles), but still - epic!
Eager to start looking at it and playing with it .......
😂 exactly most companies would still be in planning meetings.
advantage of building in public. no committees, no bureaucracy, just ship and iterate.
"eager to start looking at it". let me know if you hit any setup issues. trying to get first 5-10 instances deployed this week to test federation.
would love your feedback on the protocol
Yeah having the vision and the skills (and knowing what you want and need) beats any corporate design/planning committee, lol ... brilliant work, excited to check it out!
This is honestly one of the more concrete responses I’ve seen to the “knowledge collapse” problem most discussions stop at diagnosing it, not shipping infrastructure.
The security scanner is the part that really stands out to me. That’s the missing piece in almost every “share your AI chats” idea: people want to share insights, but one leaked token or internal URL is enough to shut the whole thing down. Treating safety as a first-class concern instead of an afterthought makes this feel production-minded rather than experimental.
I also like that you’re leaning on ActivityPub instead of inventing a new federation protocol. Reusing something battle-tested (even if it’s awkward for Q&A semantics) feels like the right tradeoff if the goal is adoption rather than purity.
One question I’m curious about as this scales:
how do you see moderation and trust evolving across federated instances, especially once cross-instance search is live? Is it more “local rules, local reputation,” or do you imagine shared signals emerging over time?
Either way, shipping this end-to-end in ~3 weeks — parsing, embeddings, security, federation is impressive. This feels like real infrastructure, not a demo. I’ll be digging into the repo.
this is THE question. youre right to ask it early.
my thinking. layered moderation
local instance rules: each sets its own policy (strict vs permissive)
shared blocklists: subscribe to curated spam/bad actor lists (activitypub pattern from mastodon).
web of trust: reputation doesnt federate centrally. instead, your instance tracks which OTHER instances you trust. transitive trust emerges (like pgp key signing).
the hard part: cross-instance search
whose rules apply when searching federated content?
current plan:
this means:
does that match your mental model or am i missing something?
"digging into the repo" . would love feedback on protocol design, especially moderation hooks.
Really cool project! 🙌 I love how this tackles decentralized knowledge sharing and puts control back in users’ hands. Looking forward to seeing how the federated AI knowledge commons grows and gets adopted — especially for collaborative learning and research!
appreciate that richard. your question about fosstodon was the spark.
"living breathing resource" is exactly right. this only works if its adopted by people who care about preserving knowledge publicly.
next step. lets get your instance running.
ive got the setup down to 5 minutes. if youre interested, we can test federation between our instances - prove the protocol works cross-server.
would love your feedback on the activitypub implementation. youve got way more fediverse experience than i do.
lets make this real.
The security-first approach and federation angle make this feel thoughtful, not rushed. Turning private AI chats into safe, searchable, shared knowledge is something a lot of us have felt missing.
Big respect for building this in public and getting it into production so quickly.
appreciate that.the security scanner was non-negotiable.
one leaked api key costs more than this entire system. had to be safe by default.
"something a lot of us have felt missing" -exactly. we all knew this was a problem, someone just had to build it.
I am curious.what would you want to see next? collections? mcp extension? better federation protocol?
This is the logical next step in the conversation about Digital Sovereignty.
I recently wrote about the importance of 'Owning Your Keys' (CMKs) to secure data at rest against vendor lock-in. Seeing this concept applied to the knowledge layer via Federated AI is fascinating.
Centralization offers convenience, but decentralized 'Knowledge Commons' offer true ownership. Excellent work on this architecture.
appreciate you connecting this to digital sovereignty.that's exactly the conversation this needs.
centralization = convenience at the cost of ownership.
federation = ownership at the cost of bootstrapping.
but the cost equation is changing:
the next wave isn't "build vs buy" - it's "own vs rent"
your CMK work on data at rest + this on knowledge at rest = complete digital sovereignty stack
what does the infrastructure layer look like when enterprises want both?
this is going to be great. first federation test with someone who actually understands activitypub protocol.
ive documented the deployment but let me know if anything is unclear. we can do a live test once yours is running.
also if you find protocol issues, raise them early. better to fix before 10+ instances deploy.
excited to see fosstodon community connected to this
That sounds really interesting! Can you share a bit about the approach you used to tackle knowledge collapse and what kind of results you saw in those 21 days?
happy to break it down.
problem: stack overflow down 78%. best debugging sessions trapped in private AI chats.
approach:
results in 21 days:
key: make sharing safe + easy while respecting privacy
what interests you most about this?
Hi, I'm Justin, a full-stack engineer with a strong interest in backend systems, async programming, and AI.
I'm currently working a lot with APIs, distributed systems, and LLM-based projects.
Looking forward to learning from you all and contributing where I can.😉
Really interesting perspective. The idea of tackling knowledge collapse in such a short timeframe is impressive. Looking forward to diving deeper into this.
I really enjoyed reading about how you built this. As a student, I'm curious: what was the biggest challenge you faced when trying to get different systems to work together in a "federated" way? Thanks for the great breakdown!
Ill have to try this out!
“21 days? That’s faster than most Netflix series binge! Can you share the secret sauce behind this knowledge collapse fix?”
"Have you audited your RAG embedding vectors for 1 \ell_P sub-threshold leakage? In high-traffic federations, we're seeing a non-integer dimensional drift where the static leakage loss (V_{DD} I_{LEAK}) becomes proportional to the dynamic switching frequency (f). If your gates aren't fully released, your MCP (Model Context Protocol) is essentially trying to index a cage that's smaller than the wavelength of the data. How are you handling the entropy spike at the Planck scale transition?"
Nice