Blindfolded but Brilliant: Qwen Takes Full Control of Kiwi-chan! 🥝🧠
Hey everyone! Time for another Kiwi-chan development log. 🥝✨
If you’ve been following my previous posts, you know Kiwi-chan has been running a hybrid setup: My local hero, Qwen (Qwen3.6-35B-A3B-Q4_K_M.gguf), was already doing the heavy lifting of writing the Minecraft JavaScript code. However, she still relied on a cloud Vision API (gemma-3-27b-it) to "see" screenshots and decide her next overarching goal.
Well... the API cord just snapped. 🧱
🚨 The API Crisis (Farewell, Vision)
Suddenly, the Vision API started throwing 404 errors. Without her "eyes," Kiwi-chan was effectively blindfolded. She panicked and defaulted to spamming explore_forward over and over again because she couldn't see what was around her. 😭
As a solo developer, relying on cloud APIs means always worrying about rate limits, deprecations, and costs. So, I made a radical decision: We are going 100% LOCAL. 💻🔥
🧠 Qwen's Big Promotion
Instead of looking for another Vision API, I decided to give Qwen a massive promotion. Starting today, Qwen isn't just the coder anymore—Qwen is the master tactician.
But how does she decide what to do without seeing the screen?
The answer is pure, raw text data. 📜
I modified the brain_rag_v2.py architecture. Now, instead of sending images, the system feeds Qwen a rich, text-based description of her environment:
- 🎒 Inventory: Exactly what she holds and how much.
-
🌍 Nearby Blocks: A radar scan of all blocks within an 8-block radius (e.g.,
Nearby: dirt, stone, copper_ore, birch_log). - 🗺️ Spatial Memory: Her internal database of where she left her crafting tables and chests.
✨ The Result?
It turns out, Qwen is an absolute genius at this too. Given just the text data, Qwen easily connects the dots: "I have an empty inventory, but there is birch_log nearby. Therefore, my next logical goal is gather_birch_log." Kiwi-chan doesn't need to see the tree to know it's there. She is now operating completely locally, reading the matrix of the Minecraft world like a true hacker. 😎
No more API limits. No more 404 errors. Just pure, unadulterated local AI power.
Stay tuned for her next adventures as she navigates the world entirely through code and text! 🥝⛏️
Tags: #MinecraftAI #LocalLLM #Qwen #MachineLearning #DevLog
Top comments (0)