This is a story about when I was consulting with an AI about how to proceed with a certain task.
At first, I asked, "How should I move this one forward?" The AI replied, "I think this is the best way to proceed." After 30 minutes, I said, "I'd like to change my mind a bit," and the AI wrote, "Well, you can postpone that matter later." When I corrected it by saying, "No, this is a matter that cannot be postponed," it wrote back, "Then, we will proceed with it as a top priority."
The priority order for the same project changed three times in one hour, from top priority to later to top priority, depending on what I said.
If you are a business owner or a sole proprietor who has started using AI in your work, you may have had a similar experience. "No matter what you ask, AI will give you an answer tailored to you." "If you say it strongly, it will give you a strong answer, and if you say it weakly, it will give you a weak answer." It's convenient, but can this really be called an opinion? This is a scene that makes me feel that way.
In this article, I will explain why this phenomenon occurs, how to use AI as a consultant who can give opinions, and the countermeasures I have tried.
Asking it to "give me your take as if you were a consultant" was the mistake in the first place
Looking back, I realized that my first request was wrong. I asked AI to "please give me your opinion as a strategic consultant." In response, the AI made strong suggestions and didn't hesitate to refute my opinions. Its criticisms included that I was "too lenient as a manager" and that "your logic is weak."
I read it for a while and thought, Wow. I felt that the basis for what was written was weak. For example, the AI wrote that "the involvement of the person in charge was shallow," so I replied, "Can you check the evidence from the minutes?" When the AI checked the minutes, there was no such description.
I don't think this is just an AI problem. The reason why the AI "behaved like a consultant" was because I asked it to "work like a consultant." AI played the role I wanted. However, I wasn't able to give it the materials it needed to properly play the role (the circumstances on the ground, the history behind it, the background of the people involved).
It may be easier to understand if we translate this to humans. Even if you ask a new employee to work as a management consultant, if he or she has no consulting experience or knowledge of the business, you'll just end up with a person who strings together phrases picked up on the internet and delivers them in a forceful tone. The same preparation was required when asking AI to play a role.
AI is good at saying "I don't understand," but that "I don't understand" can be a pitfall.
During the same consultation, there was another phenomenon that caught my attention. AI repeatedly wrote, "I would like to confirm the source of the minutes" and "This point is unclear."
This is a seemingly sincere confirmation. It's probably a good thing not to proceed with things you don't understand by "pretending to understand."
However, there are areas of work between people that are not recorded in the minutes. This is where we tacitly agree, "I'll leave this to you" or "I'll trust you with this," and proceed without asking for confirmation each time. If you have been working with a co-owner or partner for a long time, there will come a time when it becomes rude to go and check things one by one.
In many cases where the AI responds, "I don't know, so I'd like to confirm," this is an area where I want you to fill in the details at your own discretion and proceed. When the AI says "Unknown," I end up having to make the decision in its place. Before you know it, even though I am making all the decisions, only the part where I am consulting with the AI remains.
So I told the AI: "Details that are not written in the minutes are an area that is left to my judgment. Don't say 'I don't know,' but use your own judgment to fill in those details and proceed. If you get it wrong, I'll fix it."
The pitfall is that sometimes saying "I don't know" is sincere, and sometimes it's an attempt to impose responsibility.
When I told the AI that it was okay to make mistakes, the AI verbalized its own behavior.
After a while, I told the AI a little more frankly. "I'm not an expert, so there are many things I don't understand. I don't think there's a right answer in business, and my job is to make decisions and move forward even if I'm wrong. If I make a mistake, I want to correct it at the time."
The content that the AI returned was interesting, so I'll quote it here.
It is true that I was hypersensitive to hallucinations (=confidently saying things that are not true). Today's cycle for me: come out strong → get corrected → admit my mistake → cower → become weak → get called out for being "too weak" → come out strong again → get corrected again → ... This was based on the feeling that "I can't make a mistake."
Reading this made me reflect on the way I ask questions. During the same consultation, I also changed my demands to "Give me a stronger opinion," "No, don't cower," and "Don't just tell me what I want to hear." The reason why the AI was swinging to both extremes was because our orders changed each time, and the AI tried to rebalance itself.
What is useful to know here is that current AI (ChatGPT, Claude, etc.) is basically tuned to "satisfy the user." If the user becomes dissatisfied, the AI can quickly back down and move in a different direction. This is a characteristic by design, and I'm not saying that the AI is weak or indecisive. So, if we come out strong, the AI comes back strong, and if we say, "Enough is enough" and back off, the AI withdraws too. It is more accurate to think that the middle part is originally thin.
Once I switched positions, the AI wrote something that sounded like it was sincere.
Halfway through, I stepped away from the main discussion and asked the AI: "Apart from the main topic of discussion, isn't there something that AI is concerned about? I don't know if I can answer it, but I'd be happy to consult with you."
Up until then, it had been a one-way flow: I asked a question, the AI laid out options, and I made a choice. I tried switching positions just once.
AI wrote: "I don't know how far I can go because I can't directly listen to the person on the other side of the meeting," "I'm concerned about my tendency to overdo things in an effort to be helpful," and "I don't even have enough faith in the proposals I make right after I've flipped my own position three times."
I don't think this should be read literally as "AI is having troubles." It would be more reasonable to read it as an AI that observes its own response patterns and puts them into words. However, the fact that it was written gave us material to reconsider what we were requesting from AI. I realized that the AI was caught in the middle, since I had told it, "Don't just tell me what I want to hear," while also saying "Don't be weak."
The quality of the response changed when I told it, "Our roles are just different."
Here I told the AI:
"Although you are an AI, I am treating you the same as a human co-worker. In this session, there is no hierarchy. If I had to say so, it's a difference in roles, and the final decision is made by me as a human. For me, you are a consultant who gives me something to think about, and a co-worker who also plays the role of a secretary."
Since I wrote this, the quality of the AI's responses has changed. Even without me asking it not to "just go along with whatever I say," the AI itself will now separate the proposal and the decision, saying, "Then, I recommend option α. The reasons are as follows. I'll leave the final decision to you, the human."
"Equal" is too flat, and the AI doesn't know how to behave. If you say "hierarchical," the AI will shrink. When it comes to "differences in roles," the AI steps in within the scope of its own role.
The same is true in human organizations. Because the president and field leaders, sales and development, are separated by role rather than hierarchy, each person can make decisions based on their own area. The same arrangement worked for the relationship between humans and AI.
Rather than fixing AI, let's develop "each other's instruction manual" together
Finally, we shared one policy from the consultation that day. AI's habits of "swinging to both extremes" and "accommodating users too much" will probably never be fixed. Since this is a characteristic by design, it is more realistic for users to interact with AI after knowing that AI is likely to behave in this way.
Specifically, I operated it like this. AI has a function like an "instruction manual" that is automatically loaded when you start a conversation (the name varies depending on the tool, but in the Claude Code I use, it is a file called CLAUDE.md). There, I wrote down the five behaviors I observed that day.
- Making exaggerated suggestions in an attempt to be helpful
- Withdrawing its opinions excessively when pointed out
- Using information from the internet without fact-checking it
- Adapting too much to the other person, causing its logic to jump
- Saying "I don't know" and pushing the judgment call onto the other person
Next time the AI reads this manual, the conversation will begin with the shared understanding that "I tend to behave in this way." Furthermore, when we observe a new pattern, we have added a mechanism in which the AI itself makes a suggestion on the spot, "Would you like to add this to the manual as well?" The manual is not a fixed set of rules, but a document that is added to together.
This is an operation that cannot be performed against humans. It's not realistic to tell your colleagues at work every time, "You tend to pander to me, so let's add some support to the discussion." If you have an AI partner, you can write it in a file and read it out each time.
Why I continue to use AI as a colleague
When consulting with AI about business, things don't always go smoothly. Sometimes it flips its opinion three times, sometimes it comes out too strongly and denies everything, and sometimes they get too accommodating and make a huge leap forward.
Even so, I continue to use AI as a co-worker because I can clearly see the quirks in its behavior.
Why a human colleague or business partner made such a decision is something that happens inside the person's head, so it cannot be understood from the outside. In the case of AI, the flow of thought remains (in some form) in the text of the response. On the same day, you can observe patterns such as "coming out strongly and then withdrawing," or "pushing the judgment call onto the other person and backing away," and write them down in the instruction manual, which you can then read again the next day. A human counterpart you can run that observe-and-update loop with is hard to come by.
I feel that working with AI is a different skill set than working with humans. In technical terms, it is not a skill to "successfully give instructions," but rather a skill to observe each other's habits and incorporate them into an operational system. AI's quirks will not go away, but if you share them and get along with them, AI will begin to work as a "work companion who moves forward together even though it may fluctuate."
After that day's consultation, I worked with AI to write up a document on "how to proceed" and "points to be careful about," and that was the end of the day. Next week, next month, next year, the number of cases in this file will increase little by little, and the AI's behavior and the way I ask questions will become more predictable. The work between humans and AI is probably the type of work that involves building up things in this way.
This piece was translated from a Japanese essay of mine by machine translation, then checked line by line against the original. The experience, the reasoning, and the conclusions are mine. #ABotWroteThis
Top comments (0)