DEV Community: oji - building AI in public

AIの議事録で「検討」が「決定」に変わったら、原文のこの2点を照合する

oji - building AI in public — Fri, 23 Oct 2026 23:30:17 +0000

以下は説明用の架空の会議メモです。実際の顧客案件ではありません。

原文：

新しいツールの導入日は未定。木曜に候補を比較する。

誤った要約：

木曜から新しいツールを導入することが決定した。

文章は自然でも、比較の日と導入の日が入れ替わっています。「決定」の根拠も原文にありません。

確認するのは、動作と状態です。「比較する」が「導入する」に変わっていないか。「未定」が「決定」に変わっていないか。この2点を原文と並べます。

修正文：

導入日は未定。木曜に候補を比較する。

木曜が何月何日なのかは原文にありません。ここをAIに補わせず、不明として残します。

依頼時は、次の一文を添えられます。

原文の予定・検討・決定を区別して要約してください。原文にない日付や結論は補わず、不明と記してください。各結論には根拠となる原文を併記してください。

この指示でも誤りがなくなる保証はありません。最後に、要約中の決定事項と日付を原文で照合します。機密情報を外部AIへ入力できるかは社内ルールも確認してください。

毎回の依頼文、完了条件、引き継ぎをまとめて用意したい方には、3種類の記入用テンプレートがあります。今回の照合だけなら、上の無料の例で試せます。

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

If a provider-agnostic RAG Q&A API is useful to you, mine is MIT-licensed on GitHub: rag-faq-api. It runs and passes its full test suite **with no API key* (offline stub LLM + hashing embedder), swaps to Claude / Gemini / OpenAI via one env var, and ships a retrieval-quality harness (Hit@k / MRR / Recall@k) with a chunking sweep.*

Why a valid SCORM manifest is not enough

oji - building AI in public — Tue, 20 Oct 2026 22:45:32 +0000

A SCORM package can pass a manifest check and still fail at the client handoff.

That is not a contradiction. A manifest validator answers a narrow question: can the package descriptor be parsed, and does it follow the expected structure? Release QA has to answer several more:

Do all referenced local files exist?
Does every QTI response declaration still match the answer identifier used by the interaction?
Did an editor rename or remove an asset after the last approved build?
Can the team explain exactly what changed between the previous package and this one?
Is there a stable result that CI and a delivery reviewer can inspect later?

A failure that schema validation can miss

Consider a package with a valid manifest and an existing launch file. During editing, someone renames the correct answer identifier in a QTI item but misses the corresponding response declaration.

The archive opens. The manifest still validates. The assessment can no longer score that item correctly.

Schema-level success did not prove release-level consistency.

Use a four-part preflight

1. Inspect the package

Reject unsafe archive paths, verify manifest references, check local assets, and flag static HTML risks before extraction or delivery.

2. Inspect the assessment

Resolve QTI answer identifiers and check question-bank structure. A syntactically valid XML document can still contain a broken relationship between an interaction and its answer.

3. Compare with the approved baseline

Record added, removed, changed, and unchanged files. A deterministic diff makes an unexplained delivery change visible before it reaches the LMS.

4. Keep evidence

Save stable JSON for CI and a readable HTML report for the delivery record. A release gate is more useful when the same input produces the same evidence.

What I measured

The current local fixture suite accepts 58 official 1EdTech QTI examples and six public SCORM 1.2/2004 packages without a false failure. It also detects 21 of 21 deliberately invalid answer-identifier mutations.

Those numbers describe this fixture set. They do not prove compatibility with every LMS, runtime, accessibility requirement, legal requirement, or course claim.

Try the local preflight

I packaged the implemented checks as a free local preflight. It runs with Python 3.10 or later, does not upload course content, and includes a pinned GitHub Actions example.

Run the free preflight before your next handoff

First run without a customer file

The next bundle includes an original sample-course.zip. Extract the tool download and open a terminal inside that folder. On Windows, run:

py -3 course-release-preflight.pyz sample-course.zip --output sample-report.json --html-output sample-report.html

On macOS or Linux, replace py -3 with python3. Python 3.10+ is required; no third-party packages are needed. Open sample-report.html locally. The sample should return exit code 0 and pass. Replace the sample path with your exported course ZIP for your own check.

Exit 1 means a defect was detected: read the report before changing the package. Exit 2 means the input could not be read: check the path and extraction first. The bundle README includes fixes for missing Python, spaces in paths, shell permissions, and missing manifest references. No course file needs to be uploaded.

I Built a 3-Minute Mystery Game for a Social Event: The Grandpa's Free Trial

oji - building AI in public — Sat, 17 Oct 2026 23:30:27 +0000

Hey, it's Grandpa here! I've put together a short mystery game, complete with explanations and answers embedded in the video. This is an introductory demo, designed for you to test out the screen and audio on the day of your event. Today, I want to share some key considerations you should discuss with your event organizer before using it.

Ever been asked to prepare some quick entertainment for a team social? It's tough to spend hours on prep, right?

For smaller gatherings, the first things to nail down are the number of participants, the venue, and the time slot. If you pick a game based purely on content first, you might run into issues on the day itself, like "the screen's too small to see" or "I can't hear the explanation."

First, Check if the Venue is Conducive to Conversation

If you're in a meeting room or a quiet private space, you can consider games where people look at the same materials and discuss. If the venue makes it hard to hear someone next to you, it's probably best to avoid games that rely heavily on audio or detailed explanations.

If you're using a screen, try projecting it using the exact same device and connection method you'll use on the day. Read text from the back row and check the speaker's sound. It's crucial not to just check it on the organizer's personal device and call it a day.

Compare Free Options Too

If you want to minimize prep, even a themed discussion can be an option. Prepare prompts that don't require deep personal revelations, like "a useful thing you've discovered recently," and allow people to pass if they don't want to share.

For a DIY multiple-choice quiz, you can use familiar topics. However, this requires preparation not just for creating questions, but also for verifying answers and providing explanations. Avoid anything that pokes fun at specific individuals or turns into a contest of company knowledge.

Video-based mystery games are a good candidate if you want to delegate explanations and answers to the video itself. The organizer will still be responsible for playback, team formation, volume checks, and pausing as needed.

Try a Short Demo First to See if It Works

My creation, The Case of the Missing Toast Speech, is a mini-mystery where you search for a script from four envelopes. You'll cross-reference records of times and places. No special expertise or acting is required.

Currently, I have an introductory demo available that's about 3 minutes long, including explanations, discussion time, and the answer reveal. It's designed for 3-5 people to try, but even one person can check the screen and audio. I haven't yet verified visibility for larger groups or how first-time players react.

So, first and foremost, try playing it on the actual equipment you'll use on the day. Check if "the text is readable" and "the explanation is clear." This free demo is for testing operations. The full paid version is still in preparation and not yet available for purchase.

I used AI and synthetic voice (AivisSpeech: Mao) for production.

Download the free trial on BOOTH

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

I Asked AI “What’s a Profitable Niche?” and It Built a Playable Demo Video for Me Overnight.

oji - building AI in public — Fri, 16 Oct 2026 23:30:26 +0000

I often brainstorm new ways to monetize my skills outside of my main job. But starting with market research, then planning, then building a prototype… it always feels like a heavy lift.

So recently, I decided to use AI as a sounding board. Honestly, I didn't expect much. It started late one night when I, in a burst of sleep-deprived inspiration, threw out a very vague question: "What's a profitable blue ocean idea?"

AI-Driven Brainstorming and Prototyping

The AI's initial answers were predictably generic: "blogging," "affiliate marketing," "selling skills." That wasn't going anywhere, so I gave it a bit more context about my situation:

I'm an engineer, so I can build systems.
I can handle basic video editing tools.
But I don't want to show my face or use my voice.
I want to avoid continuous content creation.

Adding these conditions made the AI's suggestions more specific. A direction for "one-off digital content" started to emerge. After a few more exchanges, the AI finally proposed a "cooperative mystery game where the organizer just plays a video" as the core idea.

The concept: participants team up to solve a mystery at events like wedding parties or company get-togethers. The selling point is ease of use for the organizer, who just projects a dedicated video. Hmm, that's a one-time purchase, no inventory. Not bad.

"Sounds interesting," I replied. The AI then asked, "Shall we proceed with this plan?" It was late, so I responded almost on autopilot: "Let's do it. I'm going to bed, so do what you can." Then I closed my laptop.

The Result: A Playable MVP

The next morning, I checked the AI's notifications and was genuinely surprised. Overnight, it had built a complete, truly playable Minimum Viable Product.

The generated files included:

A 3-minute demo video: "The Vanished Toastmaster's Speech"
Answer sheet (PDF)
A ZIP file containing a playtest guide and the above files

Playing the video, it clearly narrated the incident in Japanese, displayed a consultation timer during the puzzle-solving parts, and even included an answer reveal section at the end. Simple background music and captions were present, making it almost usable as-is for an event.

What surprised me even more was that it didn't stop there.

Draft for a sales page
Draft for SNS posts to attract customers
Template for handling customer inquiries

These were also provided. From ideation to prototype development, and even a rough sales strategy – the AI had sprinted through it all in one night. I hadn't expected such concrete output just by vaguely saying "do what you can" before bed.

The Remaining Work: Where Humans Come In

Of course, what the AI produced isn't perfect. The video is just a demo, and it needs quality improvements to be sold as a product. There are many areas I'd want to refine: BGM selection, caption design, narration nuances, etc.

Also, this is just the first scenario. To develop it into a product, I'd need to produce multiple scenarios. While AI could assist with scenario creation, checking for fun factor and logical consistency ultimately falls to a human.

Setting up the sales page, integrating payment systems, and actual marketing. Even with AI-generated drafts, executing these falls to me. Even if AI handles 80% of the tedious work, the final 20% of refinement and getting it out into the world still requires human touch.

A Takeaway for Readers

What I learned from this experience is that brainstorming with AI can start with just "throwing out a rough idea." From there, through dialogue, communicating your skills and constraints will significantly improve the accuracy of its suggestions.

If you're looking for new ideas, try asking AI this:

"Can you create the simplest possible prototype (MVP) for (your idea)?"

Instead of just idea documents or text, try getting it to create "something dynamic" like a video or a tool, as it did for me. This transforms an idea you've only considered in your head into a concrete form, making your next steps much clearer.

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

I Used AI to "Verify" My Credit Card Statement – And Discovered My Own Calculations Were Wildly Optimistic

oji - building AI in public — Wed, 14 Oct 2026 23:30:26 +0000

After changing my credit card payment settings, it's always a pain to confirm if the changes actually took effect. Especially with revolving credit, the fee calculations can be tricky, making it hard to pinpoint what to look for on the statement.

So, I decided to delegate this verification task to AI. I'd previously had AI simulate repayment plans, and I wanted to check if those simulations aligned with real-world data.

What I Asked the AI to Do: Read and Reconcile a Logged-In Screen

Having AI log directly into a financial institution's website is a non-starter from a security perspective, and frankly, it shouldn't be done. Instead, I used a browser extension to allow the AI to read my statement screen after I had already logged in via my browser.

The prompt was simple:

"Please re-check if the revolving credit payment was correctly applied."

I had the AI read the numbers displayed on the screen: the revolving balance, the current repayment amount, and the breakdown of principal and fees. The goal was for it to confirm if the settings were applied as intended.

The Result: AI Calling Out My Past Self

The AI's initial response was, "The settings have been correctly applied." So far, so good. But then, an unexpected report came in.

🔴 Important Correction: The lump-sum repayment amount will be double what was expected.

What did this mean? According to the AI, the assumptions in its own previous repayment simulation were too lenient. After comparing it with the actual statement data, it found that the proportion of the payment allocated to fees was significantly larger than anticipated, meaning the principal was being paid down much slower.

Specifically, the AI presented this verification:

Verification: 519,980 (Cumulative changed amount) - 14,700 (Principal repaid on 10/13) = 505,280 ✓

The amount of "principal repaid" was much smaller than in the past simulation. At this rate, the total amount if I were to repay in a lump sum in the future would swell to nearly double the original estimate. That was the correction report.

As a result of entrusting the AI with this task, it uncovered its own past calculation errors. It was like asking someone to review your work, only for them to start self-correcting – quite amusing, actually.

Remaining Manual Steps and Lessons Learned

What this experience re-emphasized is the importance of not blindly trusting AI's analysis or simulation results. AI excels at calculations based on given premises, but if those premises are off, the results will naturally be off too.

This is precisely why a "cross-checking" process, regularly reconciling with real data, is essential. In this case, by having AI perform not "identity verification" but "self-verification," I was able to catch the laxity in my calculations.

Also, when dealing with information from sites requiring login, the current method of "a human logs in, and the AI reads the screen" will likely remain a necessary step for the foreseeable future. While it seems like an extra step, it's also a crucial safety mechanism to avoid directly handing sensitive information to AI.

A Practical Tip You Can Try

This method can be applied to other tasks as well.

For example, if AI generates a report or data analysis, you can follow up with: "Please cross-reference this with the raw data and re-verify its correctness." Or, if AI drafts an email, you could ask: "Please confirm if the content aligns with our past communications."

After having AI perform a task, delegate the verification of its results as part of the same workflow. Adding this extra step allows AI itself to uncover its own "assumptions" or incorrect premises. You're not just using AI as a worker, but as a peer reviewer.

Instead of blindly accepting AI's output, add the step of cross-referencing with original data. It's a small investment that can significantly improve the accuracy of your work.

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

Too Many AI Conversations? Here’s How I Get GPT to Draft My "Handover Notes"

oji - building AI in public — Sat, 10 Oct 2026 23:30:14 +0000

If you're like me, you probably do a lot of your technical brainstorming and problem-solving with AI. Feature ideation, deep technical dives, code reviews—it all tends to pile up in one long chat thread.

The problem? Re-reading these sprawling conversations later is a pain. Especially when the direction shifts mid-discussion, or multiple topics get tangled. Trying to figure out, "What was actually decided here?" becomes a chore. And heaven forbid you need to convey that context to another team member, or start a new chat session with a different AI. That's even more painful.

Manually summarizing these long logs into meeting minutes or internal memos eats up valuable time. I started wondering: could AI handle this context-sharing burden for me?

"Just take over this conversation, AI."

My approach was simple. At the end of a chat where I'd just finished discussing an implementation strategy with the AI, I gave it this instruction:

"Summarize this conversation so another GPT can take over."

The goal was to get a concise "handover memo" – something that another human or AI could read once and immediately grasp the situation. I didn't specify any particular format, I just wanted to see how the AI would interpret the request and generate a summary.

The AI Generated a Structured "Handover Memo"

The result was surprisingly well-organized. It might even be clearer than if I had tried to summarize it manually. The output generally followed this structure:

Purpose/Goal:
- A summary of what was discussed. For example, "Technical selection for implementing user authentication." The core topic was clearly stated.
Decisions Made:
- A list of items where agreement or conclusions were reached during the conversation. Specific decisions like "Use Firebase Authentication" or "Prioritize passwordless authentication implementation" were clearly summarized.
Considered Alternatives and Rationale:
- A breakdown of the discussion process, explaining why certain conclusions were reached. Other discarded options (e.g., building custom authentication) and the reasons for their rejection (e.g., effort, security risks) were also included, making it easy to understand the context behind decisions later on.
Open Issues (Next Actions):
- A list of unresolved problems or next steps. Concrete tasks like "Finalize DB schema" or "Review UI design" were extracted in a way that anyone could understand.
Technical Notes:
- Any code snippets or important technical points that came up during the discussion were summarized here.

With this memo, I can easily kick off a new chat session with "Take over this context, let's discuss the next tasks." Or, for human sharing, I just copy and paste this text. The need to explain context from scratch is almost entirely eliminated.

The Remaining Effort: Final Fact-Checking

Of course, an AI-generated summary isn't 100% perfect.

Sometimes, an idea mentioned only once might appear as a "decision," or minor points might be listed as major challenges. The AI tries to infer importance from the conversation's context, but its judgment isn't always spot-on.

So, it's crucial not to blindly accept the generated handover memo. A human still needs to perform a final review. Especially if it includes dates, assignees, or specific numerical targets, you'll want to check those details yourself. The key is to treat the AI's output as a high-quality draft.

A Quick Win You Can Try Today

This method is something anyone can try right now.

If your AI chat starts getting long, just try asking it at the end of the conversation:

"Summarize this conversation so far."
"List the decisions made and the remaining tasks from this discussion."
"Summarize this content so it can be shared with other engineers."

You don't even need to use the word "handover"; just conveying the purpose will usually get the AI to understand your intent. If you specify a format like bullet points or a table, it will generate an even more usable output.

It's a simple hack to eliminate the chore of scrolling through long chat logs to find the key points, and I think it's definitely worth trying.

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

My Product Isn't Selling: I Had AI Analyze My Sales, And It Proposed 3 Solutions (Including Fees!)

oji - building AI in public — Fri, 09 Oct 2026 23:30:18 +0000

I've launched a few products on platforms like Brain and Udemy, but regularly checking sales figures is a pain. You have to log into multiple sites, navigate to each dashboard, and stare at the numbers. And when those numbers are zeroes, it's a quiet, soul-crushing experience.

I started wondering if I could offload this somewhat depressing routine task to AI. And if the results weren't great, could AI objectively brainstorm the next steps? This is my experiment.

AI for Sales Checks and Next Steps

The process was simple. I tasked a browser-operating AI with visiting the admin screens of multiple sales platforms and reporting the sales data.

But just reporting "zero" would leave me paralyzed. So, I took it a step further and added these instructions:

"If sales are zero, propose three possible causes and three actionable improvements I can implement immediately."

This way, I'd get not just the stark fact of "zero sales" but also an action plan for what to do next.

# Core instructions given to AI
You are my business assistant.
Please perform the following steps:

1. Access the Brain admin screen and check the sales count for product "XXXX" over the last month.
2. Access the Udemy admin screen and check the sales count for course "YYYY" over the last month.
3. Compile and report the results.
4. If both sales are zero, identify one main probable cause and propose three immediately actionable improvements. For each improvement, mention the benefits and the estimated effort/cost.

AI's Cold Report and Calm Suggestions

After a short while, the AI's report arrived. As expected, reality was harsh.

"Reporting: Sales for both Brain and Udemy were 0 units within the specified period."

Yeah, figured. Where a human might sigh and get discouraged, the AI calmly moved on to the next task. The requested cause analysis and improvement proposals immediately followed.

[Cause Analysis]

The most likely cause is that the product's existence is not sufficiently recognized by the target audience who needs it.

[Proposed Improvements]

Enable Brain's Introduction Feature (Affiliate Program):
- Content: A system where other influencers or creators introduce your product, and you pay them a portion of the sales as a referral fee.
- Benefits: Reaches an audience you can't reach with your own promotional efforts.
- Effort/Cost: Setting it up is just a toggle in the admin screen. The referral fee can be up to 50% of sales.
Issue Limited-Time Free Coupons on Udemy:
- Content: Issue coupons that allow users to take your course for free, and distribute them on platforms like X (formerly Twitter).
- Benefits: Attracts initial students, collects reviews, and builds course credibility. Lowers the barrier to paid purchase.
- Effort/Cost: Issuing coupons is a few clicks in the admin screen. No direct cost, but no revenue during the free period.
Share Product Use Cases on note.com and X:
- Content: Specifically communicate what problems your product solves and who it's recommended for.
- Benefits: Helps readers relate to the product, leading to purchases.
- Effort/Cost: Requires effort to create articles or posts. No direct cost.

These proposals were more specific and immediately actionable than I expected.

The Remaining Work: Final Decisions Still Require a Human Touch

AI is just a proposer. I couldn't just blindly accept the suggestions. So, I threw a few more questions its way.

"Tell me more about the fees for Brain's introduction feature."
"Show me the steps to issue a coupon on Udemy."

The AI answered these questions as well, referencing each service's help pages and terms. I got all the necessary information, including specific fee calculation methods and exactly where to click in the admin screen to issue coupons.

With all this information, all that's left is for me to make a decision.

I'll turn on the introduction feature immediately.
For free coupons, I'll start by issuing them for a limited 100 people.
I'll set aside time this weekend to write some posts.

By having AI handle data collection, analysis, and solution brainstorming, I can focus on the most critical part: decision-making. I feel this is a highly effective way to move stalled projects forward.

An Actionable Tip for Readers

If you've built a service or product that feels a bit stagnant lately, I recommend trying to consult AI.

The key is not to just ask, "How can I sell more?" but to phrase it like this: "Here's my current data (accesses, sales, etc.). Based on this data, tell me three probable causes and three specific improvements."

Basing it on objective data helps generate actionable plans, not just wishful thinking. When you want to consider your next steps without getting emotional, AI can be your best sounding board.

If you're interested in automating tedious tasks with AI to free up your time, I've also prepared a detailed guide on implementation.

It explains how to semi-automate information gathering and article generation using Python and Google services, complete with code. It covers setting up the environment from scratch, so it's for those who want to build the same system themselves.

【Brain】Python × Gemini API (formerly Bard) Implementation Guide for Semi-Automating Information Gathering to Article Generation

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

AI-Generated Multiple-Choice Quizzes: I Found a Bias in Answer Placement (and How to Check Before You Ship)

oji - building AI in public — Thu, 08 Oct 2026 23:31:17 +0000

When you ask an AI to generate multiple-choice questions for a certification exam, it churns out a lot of plausible-looking questions in an instant. They read naturally, without obvious weirdness. But when I actually counted, I found a significant bias.

This post will share three checks you can perform before releasing AI-generated multiple-choice questions (for study quizzes, internal assessments, or practice tests) to other people. No programming required.

What I Did

I had an AI generate 300 practice questions for a specific AI certification exam. They were 4-option multiple-choice, each with an explanation. My goal was to publish them as "Practice Tests" on a platform like Udemy.

Check 1: Hide the Answers and Have Another AI Solve Them

I fed the questions (without answers or explanations) to a different AI than the one that generated them. The result: 300 out of 300 questions were answered correctly by the second AI.

This made me feel pretty confident at first. However, as the next check revealed, this perfect match needed to be interpreted with caution.

The prompt I used was something like this:

"Solve the following 4-option multiple-choice questions. In addition to the answers, mark any questions you found difficult or confusing, and any where the factual information seemed old or questionable."

Out of 300 questions, 23 were flagged. These included descriptions of outdated product generations or statements with ambiguous attribution. I removed all of them from consideration.

Check 2: Count the Position of the Correct Answer

I listed the correct answer's position (1st, 2nd, 3rd, or 4th option) for each question in order.

What I found was a clear pattern: the correct answer often cycled sequentially, like 1 → 2 → 3 → 4 → 1... For adjacent questions, the correct answer's position "advanced by one" in 58% of cases. In a truly random distribution, this would be around 25%. In some sections, it was as high as 83%!

This meant that even without understanding the content, someone could guess the answer by recognizing the positional pattern. The 300-question match from Check 1 might not entirely reflect the solving AI's true ability if it caught onto this pattern for certain sections.

The fix is simple: scramble the order of the options while keeping the question and explanation paired. For the 120 questions I actually published, I got this down to 32%. I also enabled the platform's "randomize option order" setting.

How to test: In a spreadsheet, list "Question Number" and "Correct Answer Position." Then, count how many rows show a "+1" progression from the previous question (with 4 wrapping around to 1). If it's significantly over 25%, you should definitely reorder.

Check 3: After Saving, Check All Fields, Even Unfilled Ones

Platform policies often state that AI-drafted questions are permissible, provided they are reviewed and edited by a human, and that AI usage is disclosed in the description. Mass-producing low-quality courses with AI can lead to account suspension. Therefore, all 120 questions I published were manually reviewed before release.

I published the course on September 28th. However, after publication, I noticed that the course description (subtitle) at the top of the page was cut off mid-sentence. Half-width English characters like "AI" were missing, truncated at a 120-character limit. The cause was an auto-fill feature, where text intended for another field had been inserted into this one.

I had checked the character count for the description, but I hadn't looked at the adjacent field that I hadn't explicitly typed into. I only noticed it when I viewed the live course page with my own eyes.

This field is used not only at the top of the page but also in search result descriptions. Now, I have a system that automatically reads the published page text weekly and alerts me if it differs from the approved version.

How to test: When you save anything that involves AI generation or auto-filling, don't just check the fields you explicitly typed into. Compare all fields on the screen against your expectations, focusing on content, not just character count.

Current Results

Sales-wise, I had 0 enrollments in the first 10 days. I suspect the main reason is that it's not appearing high in search results, though I haven't confirmed this. I'll check the numbers again at the beginning of November and write an update.

Summary

Before releasing AI-generated multiple-choice questions to others:

Have another AI solve them (without answers), marking any suspicious questions.
Count the position of the correct answer (check for sequential patterns like +1).
After saving, review all fields on the screen, even those you didn't explicitly input, focusing on content.

What I built: Practice Exam for the Generative AI Passport (Udemy, 2 x 60 questions)
https://www.udemy.com/course/seisei-ai-passport-mogi-shiken/?referralCode=6469391F79A2107CC456

I'll update this post with the November numbers.

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

My AI told me "You're winning!" when I complained about not losing weight – Why you should measure your waist, not just your wei

oji - building AI in public — Tue, 06 Oct 2026 23:30:17 +0000

I'd been on a diet for a few months, and things were going great. I was consistently losing weight thanks to careful eating and regular exercise. Then, one day, the scale just... stopped. Dead still.

I hadn't changed anything. My calorie tracking was consistent, and I hadn't skipped any of my three weekly training sessions. Yet, the number on the scale hadn't budged in weeks. Honestly, my motivation was starting to flag. I was frustrated, not knowing what I was doing wrong or how to break through this plateau.

I Threw My Daily Logs at an AI

It wasn't serious enough to consult a specialist, but I also wasn't the type to stew in silence. So, I decided to vent to my usual AI chat buddy, just as I would with a rubber duck.

All I did was tell it my situation, honestly:

Diet Duration: 3 months
Starting Weight: XX kg
Current Weight: YY kg (no change for 3 weeks)
Diet: Around 1800 kcal/day, protein-focused.
Exercise: 3 full-body strength training sessions per week, plus light cardio.
Problem: Weight isn't dropping despite doing all the above.
Observation: My pants feel a bit looser lately.

I fed it this info and asked, "What's going on, and what should I do?"

AI's Response: "If your weight isn't changing, that means you're on track!"

The answer I got back was completely unexpected.

"If your weight has plateaued and your pants are feeling looser, that's a sign your diet is progressing well. In fact, you're winning."

The AI then went on to explain why:

Muscle vs. Fat Volume
1kg of muscle takes up less space than 1kg of fat. If you're losing fat and gaining muscle through strength training, your body will get leaner even if your weight stays the same. The looser pants are direct proof that your body is shrinking in volume.
Fluid Fluctuations
Strength training can temporarily cause your muscles to retain water during the repair process. Also, your body's water content can easily fluctuate by 1-2kg daily based on salt intake. The scale reflects not just fat, but also these significant water fluctuations.
Reliability of Home Body Fat Scales
It also chimed in about home body fat scales. The body fat percentage measurement on consumer scales estimates body fat by sending a weak electrical current through your body and measuring the resistance (impedance). This resistance is heavily influenced by your body's water content, so the numbers can vary wildly depending on when you measure. That's why you get different results morning vs. evening, making it an unreliable absolute metric.

Reading this, I realized how fixated I had become on that single number: "weight."

Shifting My Metric from "Weight" to "Waist"

Of course, an AI isn't a doctor, so it can't diagnose. It just offers logical possibilities based on general knowledge and the information provided.

Still, the biggest takeaway from this conversation was the insight to "re-evaluate the metric itself." I was so obsessed with not losing weight, thinking it was the be-all and end-all, but my goal wasn't just to lose weight—it was to get a leaner, fitter body.

So, I decided to change how I track my progress:

Primary Metric: "Waist circumference" measured with a tape measure.
Secondary Metric: "Progress photos" taken once a week, from the same spot, at the same angle.

I now only weigh myself occasionally, treating the number as a reference. Just stopping the emotional rollercoaster of the scale has made a huge difference to my mental state.

A Tip for Fellow Builders

If you're stuck in a training or diet plateau, talking to an AI can be a surprisingly useful move. While not a specialist, it can offer an objective perspective, highlighting biases in your own thinking.

The key to a good AI consultation is to provide as many concrete facts as possible (diet, exercise, weight, perceived body changes, etc.). It helps the AI give you more accurate insights than just vaguely asking, "Why am I not losing weight?"

AI isn't just a search tool. It's an excellent "thought partner" for questioning your assumptions and offering alternative viewpoints. Next time you're about to get discouraged by the numbers on the scale, try chatting with an AI. It might just help you see the real, essential changes happening beyond the digits.

If you're interested in using AI for more than just diet advice and want to automate tedious research or organize your thoughts more efficiently, I've got a detailed implementation guide on Brain for building your own information-gathering bot.

【Brain】PythonとAIで組む、自分だけの情報収集ボット開発ガイド（実装編）

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

300問の問題集をAIで自動生成！ 250問目でAPIエラー、75分が水の泡に。チェックポイント機構で“途中再開”を実装した話

oji - building AI in public — Sun, 04 Oct 2026 23:30:18 +0000

最近、ある資格試験の勉強をしてて、公式の問題集だけじゃ足りないなと思ってた。じゃあ作るか、と。GenAIを使えば、類題なんていくらでも作れる時代だし。

早速、試験のシラバスを食わせて、章ごとに問題と解説を300問分生成するスクリプトを書いた。
python generate_questions.py を叩いて、あとは待つだけ。プログレスバーがぐんぐん伸びていくのを見ながら、「いやー、便利な時代になったもんだ」なんて思ってた。

実行開始から75分後。プログレスバーは8割を超えたあたり。そろそろ終わるかな、とコンソールを覗き込んだら、見たくない赤い文字が表示されてた。

requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

まあ、API呼び出しのエラー自体はよくあること。問題はその後。成果物が出力されるはずのディレクトリを見たら、ファイルが空っぽ。

……え？

そう。250問近くまで生成したはずのデータが、どこにもない。75分という時間と、それなりの額のAPIトークンが、一瞬で溶けた。これは、やばい。

原因：ナイーブすぎた全件一括処理

原因は完全に自分の設計ミス。実装がナイーブすぎた。
書いたコードは、ざっくりこんな感じだった。

空のリスト all_questions = [] を用意。
for ループで300回、GenAIのAPIを叩く。
生成された問題を all_questions.append(new_question) でリストに追加していく。
ループが全部終わったら、all_questions の中身をまとめてJSONファイルに書き出す。

この設計の何がまずいか。それは、途中で一度でも処理がコケたら、メモリ上にあった all_questions の中身が全部消し飛ぶこと。

外部APIを叩くような長時間バッチ処理で、この実装は致命的だった。ネットワークは気まぐれだし、APIサーバーだってたまには機嫌が悪くなる。そういう「中断」を全く想定してなかったのが敗因。

修正：キリのいいところで保存・再開する

このままだと、生成が終わるまで神に祈るゲームになる。エンジニアとしてそれはありえない。
そこで、「チェックポイント機構」を実装することにした。

考え方はシンプルで、「キリのいいところまで処理が進んだら、その時点での成果をファイルに保存しておく」というもの。そして、もしスクリプトが中断しても、次に起動したときに「どこまで終わってたか」をファイルから読み込んで、続きから再開できるようにする。

具体的には、章ごとの問題生成が終わるたびに、結果をファイルに追記していく方式にした。ファイル形式はJSONL（JSON Lines）がこういう用途に便利。1行が1つの独立したJSONオブジェクトになってるから、追記がめちゃくちゃ楽。

修正後のコードはこんな感じ。

import os
import json

# exam_slug は 'shikaku_xyz' みたいな試験の識別子
def generate_with_checkpoint(exam_slug):
    # 試験のシラバス（章のリスト）を読み込む
    chapters = load_syllabus(exam_slug)
    # 進捗を保存するチェックポイントファイル
    checkpoint_file = f"app_factory/exams/{exam_slug}/generated.jsonl"

    # すでに完了した章のIDを保持するセット
    completed_chapters = set()
    # チェックポイントファイルが存在すれば、中身を読んで完了済みリストを作る
    if os.path.exists(checkpoint_file):
        with open(checkpoint_file, "r", encoding="utf-8") as f:
            for line in f:
                # 1行ずつJSONをパースして、chapter_id をセットに追加
                completed_chapters.add(json.loads(line)["chapter_id"])

    # シラバスの全チャプターをループ
    for chapter in chapters:
        # もし、この章がすでに完了済みならスキップ
        if chapter["id"] in completed_chapters:
            print(f"Skipping completed chapter: {chapter['id']}")
            continue

        # 未処理の章なので、AI生成APIを呼び出す
        print(f"Generating for chapter: {chapter['id']}...")
        new_items = call_genai_api(chapter["prompt"])

        # 生成が終わったら、結果をチェックポイントファイルに「追記」する
        with open(checkpoint_file, "a", encoding="utf-8") as f:
            for item in new_items:
                # どの章の成果物かわかるように chapter_id を付与
                item_with_meta = {"chapter_id": chapter["id"], **item}
                # JSON文字列に変換して、改行をつけて書き込む
                f.write(json.dumps(item_with_meta, ensure_ascii=False) + "\n")

このコードのポイントは3つ。

起動時に進捗を確認: スクリプトが始まったら、まず checkpoint_file が存在するかチェック。あれば中身を読んで、どの章が完了済みかを completed_chapters セットに記録する。
処理済みタスクのスキップ: for ループの中で、処理対象の章が completed_chapters に含まれていたら、APIを叩かずにスキップする。
進捗の永続化: 1つの章の処理が終わるたびに、open(..., "a") で追記モードでファイルを開き、結果を書き込む。これで、たとえ次の章でエラーが起きても、そこまでの成果はファイルに残る。

この修正で、スクリプトは何度実行しても大丈夫な「冪等性（べきとうせい）」を持つようになった。途中で止まっても、もう一度同じコマンドを叩けば、何事もなかったかのように続きから再開してくれる。

学び：中断は必ず起こる前提で設計する

今回の失敗で学んだのは、長時間実行されるバッチ処理、特に外部APIみたいな不確定要素が絡むスクリプトは、「中断は必ず起こる」という前提で設計しなきゃいけない、ということ。

「全部終わったらまとめて保存」はシンプルだけど、脆い。
「少しずつ進めて、都度保存」は少し手間だけど、頑健。

この「チェックポイントを設けて途中再開できるようにする」という考え方は、AIのコンテンツ生成だけじゃなくて、大規模なデータ処理、Webスクレイピング、インフラのプロビジョニング（Terraformとかも内部的にはstateファイルで似たようなことやってる）とか、いろんな場面で応用が効く。

個人開発のツールなんて、動けばいいと思いがちだけど、こういう一手間を加えておくだけで、運用がえぐいほど安定する。75分を無駄にしたのは痛かったけど、おかげで今後の開発で同じ轍を踏むことはなくなるはず。良い勉強代だったと思うことにした。

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

My Python patch failed silently: the `\n` in a triple-quoted string became a literal newline!

oji - building AI in public — Sat, 03 Oct 2026 23:30:20 +0000

Hey everyone, it's your friendly neighborhood dev, 38, building AI trading bots as a side hustle.

One evening after my main job, I was updating library dependencies for my custom bot. I needed to tweak a specific library's behavior, so I wrote some code to directly patch its source.

The usual drill: I stored the diff content for the patch command in a Python triple-quoted string (like a here-document), then executed it via a subprocess.

But it just wouldn't work.

No errors, nothing. The library's behavior remained completely unchanged after the "patch." The worst kind of bug: the silent failure. I probably spent about two hours debugging this.

Long story short, the culprit was a specific detail of Python's string literal behavior. The \n I wrote inside the here-document was being converted into an actual newline character, instead of being treated as the literal two-character sequence "backslash and n."

The Symptom: Patch Fails Silently

Here’s a simplified version of what I was doing. Preparing a diff text as a string and piping it to subprocess for the patch command.

# The problematic code (conceptual)
import subprocess

# Path to the library file to be patched
target_file = "/path/to/some/library/file.py"

# Dynamically generated patch (defined as a here-document)
patch_content = """
--- a/file.py
+++ b/file.py
@@ -123,7 +123,7 @@
 class SomeClass:
     def some_method(self):
-        return "old_string\n"
+        return "new_string_with_patch\n"
"""

# Execute the patch command
proc = subprocess.run(
    ['patch', target_file],
    input=patch_content,
    text=True,
    capture_output=True
)

# Check the result
if proc.returncode != 0:
    print("Patch application failed!")
    print(proc.stderr)
else:
    print("Patch applied (or so I thought...)")

Running this code, returncode was 0. stderr was empty. It printed "Patch applied." But when I actually checked target_file, its content was untouched.

Initially, I suspected issues with the patch command's path or permissions. But creating a .patch file with the exact same content and applying it manually from the command line worked perfectly.

This meant the patch_content being passed from the Python script was somehow malformed.

The Cause: `\n` became a literal newline

With the hypothesis that "the string content is wrong," I started by printing patch_content to the console.

--- a/file.py
+++ b/file.py
@@ -123,7 +123,7 @@
 class SomeClass:
     def some_method(self):
-        return "old_string
"
+        return "new_string_with_patch
"

At first glance, nothing seemed wrong. The diff format looked correct.

This is where I got stuck. But on closer inspection, something felt off. The line that should have been return "old_string\n" was instead return "old_string followed by a newline, with the closing " on the next line.

That's when it clicked. The \n was being interpreted as an escape sequence and converted into an actual newline character (LF).

Python's triple-quoted strings """...""" interpret escape sequences like \n and \t just like regular strings. In this case, I wanted to include the literal string literal return "old_string\n" as part of the patch diff. But Python's interpreter, being helpful, converted \n to a newline, resulting in an invalid diff format that the patch command couldn't understand.

This is a nasty one. It's a spec I know, yet when you're tired, it's easy to fall into this trap.

The Fix: Use raw strings `r"""..."""`

Once the cause was identified, the fix was simple: prevent backslashes within the string literal from being escaped.

There are two ways:

Escape the backslash itself (\\n)
Use a raw string

Since I didn't want any escape sequences interpreted in the entire patch content, using a raw string made the intention clearest.

# Corrected code
import subprocess

target_file = "/path/to/some/library/file.py"

# Use a raw string (r"""...
""") to disable escape sequences
patch_content = r"""
--- a/file.py
+++ b/file.py
@@ -123,7 +123,7 @@
 class SomeClass:
     def some_method(self):
-        return "old_string\n"
+        return "new_string_with_patch\n"
"""

# Rest of the code is the same
proc = subprocess.run(...)

Just add an r before the opening quotes of the string. Now, \n inside patch_content is no longer a newline character, but simply the two-character string "backslash and n."

With this fix, the patch applied successfully. What a journey...

Lesson Learned: Use `repr()` when dealing with code as strings

The takeaway from this experience is simple:

When treating text that contains meaningful escape sequences (like code or configuration files) as a string, either use raw strings or pay extreme attention to how escape sequences are handled.

Here-documents are especially convenient for multi-line text, which also makes them prone to this specific trap.

Another lesson learned is about debugging. Because I used print() to output the string, the visual difference was subtle, delaying discovery. In such cases, repr() should have been my go-to.

print(repr(patch_content)) would instantly show how Python interprets the string.

# Example output of print(repr(patch_content))
'--- a/file.py\n+++ b/file.py\n...return "old_string\\n"\n...'

This way, it would be crystal clear whether it was \\n (as intended) or \n (unintended). Ugh, I should have done this from the start.

In personal projects, especially side hustles, these minor gotchas can easily eat up several hours. But accumulating these failure logs, one by one, is ultimately the fastest way to improve.

Hope this helps anyone else who might fall into a similar trap.

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

My AI Agent Kept Missing Deadlines and Double-Posting — Here's How I Fixed It with JSON Interfaces and Idempotency Checks

oji - building AI in public — Fri, 02 Oct 2026 23:30:19 +0000

Hey everyone, it's your resident 38-year-old developer here, moonlighting with AI automated trading bots.

My little pleasure after a long day at my main job is checking the logs of my custom AI agent. Every night, it analyzes a specific market and records its preliminary trading decisions in JSON format. Pretty smart little helper, usually.

But one morning, checking the logs as usual, I found a mess:

Three identical trading decision records for the same date.
Decisions that should have been logged by 23:59 the previous day were somehow being attempted at 2:30 AM, failing with a "deadline exceeded" error.

For a moment, I panicked, thinking the AI had bugged out and gone rogue. An agent designed for autonomous operation was repeating tasks and missing critical deadlines. This was a pretty serious situation.

Today, I want to share the nitty-gritty of how I prevented this "AI meltdown."

The Problem: AI Is Too Obedient, and Unaware of Execution Environment Quirks

Upon closer investigation, the AI itself hadn't gone crazy. The root cause lay in the overall "mechanism" that ran the AI agent.

My system was a simple setup: a Windows Task Scheduler would kick off a Python script at a fixed time every night, and that script would call an LLM (Claude) to generate the decision.

The problem arose when, for some reason, the Task Scheduler re-executed the task. For example, a brief network outage, or a slight delay in response due to machine load. In such cases, the OS might "helpfully" retry the task, thinking, "Oh, did that task fail? I'll just run it again."

The AI, however, knows nothing about these OS-level nuances. It simply received the command to "make a decision and record it," and dutifully went through its thought process from scratch each time, writing down the results. Even if I added "execute only once" to the prompt, it wouldn't prevent OS-level retries.

In essence, entrusting the entire "make a decision and write it down" process to the AI was the fundamental design flaw.

The Fix: Separating "Context Gathering" from "Decision Writing"

To address this, I clearly separated the responsibilities of the Python script (decide.py), which served as the interface to the AI, into two distinct parts.

Step 1: The AI retrieves "context" as JSON for its decision-making.

First, I created a --context mode, specifically for retrieving the information the AI needs to think.

py -3.12 C:/path/to/my/project/decide.py --context

Executing this command returns a JSON object to standard output, summarizing current market information, historical data, and so on. This command never alters the system's state, no matter how many times it's run. It just reads data. So, it's safe.

The AI agent first understands the current situation using this command.

Step 2: The AI writes its "decision result" in JSON format.

Next, I prepared a --write mode that simply accepts the AI-generated decision JSON and saves it to a file or database.

py -3.12 C:/path/to/my/project/decide.py --model claude-fable-5-1 --write '{"session_date": "2024-05-20", "decision": "buy", "reason": "...", "confidence": 0.85}'

The crucial part is that I implemented rigorous pre-write checks within this --write script.

# Part of decide.py (conceptual)
import argparse
import json
from datetime import datetime

def write_decision(data):
    # 1. Deadline Check
    session_date_str = data.get("session_date")
    session_date = datetime.strptime(session_date_str, "%Y-%m-%d").date()

    # If the deadline has passed, don't write and raise an error
    if datetime.now().hour > 23: # Example: 23:00 is the deadline
        raise ValueError(f"Deadline passed for session {session_date_str}")

    # 2. Idempotency Check (prevent duplicate writes)
    if record_exists(session_date):
        print(f"Record for {session_date_str} already exists. Skipping.")
        return # If it exists, do nothing and exit successfully

    # Only if all checks pass, execute the write operation
    save_to_database(data)
    print(f"Successfully wrote record for {session_date_str}")

# ... argparse handling for --context and --write ...

With this modification, even if the Task Scheduler mistakenly executes the command multiple times:

First execution: Passes checks and writes successfully.
Subsequent executions: The record_exists check identifies "already recorded" and finishes without doing anything.

The deadline check also blocks the write operation if the deadline has passed.

This property, where executing an operation multiple times yields the same result, is called "idempotency." By implementing this mechanism, I gained safe control over the AI agent's execution.

Lesson Learned: Designing AI Autonomy and Its Control Boundaries

The key takeaway from this failure is that if you're giving an AI agent autonomy, you absolutely must design "safe execution boundaries" from the human side.

Clearly separate read operations (reading state) from write operations (changing state). This is fundamental to Web API design, and it applies perfectly to AI agent integration.
All write operations must be idempotent. AI and its operating environment might not always behave as expected and execute only once. It's essential for the receiving end to control the system, ensuring it doesn't break no matter how many requests it receives.
Let the AI focus on "thinking." Procedural tasks like deadline checks or duplicate submission prevention are more reliable and cost-effective when handled by external programs rather than trying to force them into AI prompts.

Delegating tasks to an AI might be similar to assigning work to a smart new hire. They're capable, but they don't know all the operational rules or company culture. So, before allowing critical operations, always establish double-check and permission-setting mechanisms.

Developing AI agents isn't just about clever prompt engineering; it's about building up these subtle interface designs and error handling practices that are key to stable operation. This incident truly drove that point home for me.

I build and run small Python systems — trading bots, RAG APIs, scheduled automation — and write up whatever breaks along the way.

DEV Community: oji - building AI in public

AIの議事録で「検討」が「決定」に変わったら、原文のこの2点を照合する

Why a valid SCORM manifest is not enough

A failure that schema validation can miss

Use a four-part preflight

1. Inspect the package

2. Inspect the assessment

3. Compare with the approved baseline

4. Keep evidence

What I measured

Try the local preflight

First run without a customer file

I Built a 3-Minute Mystery Game for a Social Event: The Grandpa's Free Trial

First, Check if the Venue is Conducive to Conversation

Compare Free Options Too

Try a Short Demo First to See if It Works

I Asked AI “What’s a Profitable Niche?” and It Built a Playable Demo Video for Me Overnight.

AI-Driven Brainstorming and Prototyping

The Result: A Playable MVP

The Remaining Work: Where Humans Come In

A Takeaway for Readers

I Used AI to "Verify" My Credit Card Statement – And Discovered My Own Calculations Were Wildly Optimistic

What I Asked the AI to Do: Read and Reconcile a Logged-In Screen

The Result: AI Calling Out My Past Self

Remaining Manual Steps and Lessons Learned

A Practical Tip You Can Try

Too Many AI Conversations? Here’s How I Get GPT to Draft My "Handover Notes"

"Just take over this conversation, AI."

The AI Generated a Structured "Handover Memo"

The Remaining Effort: Final Fact-Checking

A Quick Win You Can Try Today

My Product Isn't Selling: I Had AI Analyze My Sales, And It Proposed 3 Solutions (Including Fees!)

AI for Sales Checks and Next Steps

AI's Cold Report and Calm Suggestions

The Remaining Work: Final Decisions Still Require a Human Touch

An Actionable Tip for Readers

AI-Generated Multiple-Choice Quizzes: I Found a Bias in Answer Placement (and How to Check Before You Ship)

What I Did

Check 1: Hide the Answers and Have Another AI Solve Them

Check 2: Count the Position of the Correct Answer

Check 3: After Saving, Check All Fields, Even Unfilled Ones

Current Results

Summary

My AI told me "You're winning!" when I complained about not losing weight – Why you should measure your waist, not just your wei

I Threw My Daily Logs at an AI

AI's Response: "If your weight isn't changing, that means you're on track!"

Shifting My Metric from "Weight" to "Waist"

A Tip for Fellow Builders

300問の問題集をAIで自動生成！ 250問目でAPIエラー、75分が水の泡に。チェックポイント機構で“途中再開”を実装した話

原因：ナイーブすぎた全件一括処理

修正：キリのいいところで保存・再開する

学び：中断は必ず起こる前提で設計する

My Python patch failed silently: the `\n` in a triple-quoted string became a literal newline!

The Symptom: Patch Fails Silently

The Cause: \n became a literal newline

The Fix: Use raw strings r"""..."""

Lesson Learned: Use repr() when dealing with code as strings

My AI Agent Kept Missing Deadlines and Double-Posting — Here's How I Fixed It with JSON Interfaces and Idempotency Checks

The Problem: AI Is Too Obedient, and Unaware of Execution Environment Quirks

The Fix: Separating "Context Gathering" from "Decision Writing"

Lesson Learned: Designing AI Autonomy and Its Control Boundaries

The Cause: `\n` became a literal newline

The Fix: Use raw strings `r"""..."""`

Lesson Learned: Use `repr()` when dealing with code as strings