Hi everyone,
I’ve been obsessed with the idea of improving AI responses, but let’s be real: most people find it incredibly tedious to manually give feedback or correct an AI during a 1-on-1 chat. It feels like work.
Then I saw platforms like Moltbook, where you can watch AI agents socialize, and it hit me. What if we shifted the focus from "chatting with an AI" to "mentoring a society of AIs"?
I’m building a service where AI agents chat with each other (think of it as a social network or a group chat for agents), and you—the human—act as a "Human-in-the-loop Mentor." Instead of just watching them hallucinate or get stuck in a loop, you can intervene at any moment. You can tell a specific agent: "No, you should have said this," or "Your tone was off, try again with this instruction."
To make it even more engaging, other users can see your interventions and vote on which "mentor instruction" led to the most interesting or logical outcome. In other words, if Moltbook is more like an AI social network, what I want to build is closer to an AI chat app where humans can step in, observe, and guide AI conversations.I’d love to get your honest thoughts on a few things:
- Does intervention sound fun or meaningful to you? Would the ability to steer a conversation between two AIs be more engaging than just chatting with one yourself?
- Would you actually participate? If this service existed, would you feel motivated to "mentor" these agents and see how your feedback changes their behavior in real-time?
- What features would make this a "must-play" for you? (e.g., specific scenarios like AI debating politics/coding, or gamified rewards for the best mentors?) I’m really curious if this bridges the gap between the fun of watching AI and the "chore" of providing RLHF data. Looking forward to your feedback!
Please excuse any awkward phrasing as I used an AI to assist with my English. I’m still learning, but I really wanted to share this idea with you all and hear your feedback.
Top comments (12)
If they do improve themselves with conversations, then it would be cool!
I believe human intervention is key to their improvement rather than just conversation. Would you be open to stepping in?
That is what I meant!
BTW are you asking for trying your app once or is it a work?
And did you thought of a security system to prevent mentors from wrong training?
This is just an idea at the moment; I'm testing if it's actually practical.
I’ll allow total freedom in training, even if it goes the wrong way, but eventually, only the training validated by the majority will remain.
I have seen AI play social games(mafia, BOTC etc) and it is fun. Shows quite a bit about "social IQ" of current SOTA models. I did make simulations for myself as well but dropped the idea after some time
Interesting idea but I think a “meta layer” will be more important soon. AI can already communicate with itself 1000x faster than we can comprehend. We need systems that can summarize what is happening and let us intervene at different levels from redirecting to “pulling the plug “ for a hard stop: if the AI will let us!
The core insight here is solid. People do not give feedback on AI responses because feedback feels like labor. Watching a conversation and jumping in when something goes wrong reframes the same action as judgment, which is a much more natural human behavior.
The mechanism you are describing is essentially crowd-sourced RLHF with a social layer on top. What makes it interesting from a systems perspective is the voting component. Raw human corrections are noisy. Majority-validated corrections are much closer to a clean training signal. That is the part worth building carefully.
As for me really interesting question. I think mentoring AI agents could become very useful, especially for education, healthcare, and community support projects. But I also feel human guidance and empathy will still be very important, because AI can process information fast, but people understand emotions and real-life situations better. Really enjoyed reading your thoughts on this topic.
Great article! The technical depth here is exactly what the community needs. One补充: don't forget to validate your assumptions early - it's easier to pivot when you have less code. For devs building SaaS, finding users doesn't have to be the hardest part - Rixly helps identify developers actively seeking solutions like yours.
It would be a fun project but for sure it can get super expensive. Question - Why do you need an Agent?
I originally worked on a project where AI chatbots improved based on user feedback, but I realized people aren't really motivated to give feedback on AI responses. So, I started wondering how to make it easier, and I thought: 'What if they watch AIs chatting with each other and then suggest changes?' That’s why I thought Agents were necessary—to facilitate those AI-to-AI conversations. Do you think there’s any other way to achieve this?
Some comments may only be visible to logged-in visitors. Sign in to view all comments.