DEV Community: Evan Lin

[AI in Practice] Building a Dynamic LINE Group Buying Bot with Edit and Unsend Webhooks

Evan Lin — Thu, 03 Sep 2026 04:27:12 +0000

When Messages Can Be Taken Back: Building a "Shapeshifting" LINE Group Buying Bot with Edit and Unsend Webhooks

Author: Evan Lin, LINE Taiwan Developer Relations Team Lead

On August 20, 2026, LINE announced the open free trial of the "Edit Message" feature in LINE Labs.

For general users, this is a very intuitive feature: if you find a typo, a wrong date, or want to adjust your tone after sending a message, you no longer need to go through the "Unsend, retype, resend" process. You can simply edit the original text message.

But when I saw this feature, the first thing I thought of was something else:

If a user changes a message that has already been processed by a LINE Bot, does the Bot know?

The answer is: Yes. The LINE Messaging API provides the Edit event; and when a user unsends a message, there is also an Unsend event that can be received.

In this article, I want to share how I turned these two types of webhook events into a LINE Bot Demo with a story, as well as several easily overlooked but very important details during implementation.

The complete sample code is available on GitHub:

https://github.com/kkdai/linebot-edit-unsend

Starting with "Edit Message" in LINE Labs

To try out message editing in LINE Labs currently, you need to meet the following conditions:

Update the mobile version of LINE to version 26.12.0 or higher.
Go to "Home → Settings → LINE Labs".
Enable "Edit Messages".

In standard one-on-one and group chats, text messages can be edited within 15 minutes of being sent; for Keep Notes, it's within 6 days. Photos, videos, voice messages, files, and stickers cannot be edited at this time. After modification, the chat room will display "Edited," but it does not provide a way to view or restore old versions.

There is also a limitation directly related to Bot development: currently, one-on-one chats with LINE Official Accounts do not support message editing. Therefore, to test the Messaging API's Edit event, the Bot must be added to a group chat.

For features and activation methods, please refer to the LINE Taiwan Newsroom Announcement.

From UI Features to Bot Data Consistency

In the past, the common process for a Bot receiving a text message was:

Receive message webhook.
Parse the text.
Write to the database or trigger subsequent processes.
Reply with the processing result.

If a message cannot be modified after being sent, this model is very simple. But when messages can be edited, the original text is not necessarily the user's final intent.

For example, a user sends:

Pearl Milk Tea / Half Sugar / Less Ice / 1

The Bot has already recorded it as one drink costing 60 TWD. A few seconds later, the user directly edits the original message:

Pearl Milk Tea / Micro Sugar / No Ice / 2

If the Bot does not handle the Edit event, the chat room sees two cups, but the backend still records one cup. The world on the screen and the world inside the Bot become disconnected.

This is the most practical value of the Edit event: it's not just a notification that "the text has changed," but an opportunity for the service to synchronize with the user's latest intent.

Group Buying Transformation: Giving the API Demo a Story

To demonstrate both edit and unsend simultaneously, I set the Bot as "Shapeshifting Store Manager - Dan Dan," responsible for saving the office from low energy at 3:00 PM.

The Demo flow is as follows:

The group leader enters Start Group Buy 50 Lan 15:20 Deadline.
Alice enters Pearl Milk Tea / Half Sugar / Less Ice / 1.
Alice edits the same message to Pearl Milk Tea / Micro Sugar / No Ice / 2.
The Bot receives the Edit event and updates the quantity, sweetness, ice level, and total amount.
Bob unsends his message after placing an order.
The Bot receives the Unsend event, deletes Bob's order, and recalculates.
Enter Current Orders to view the latest summary via Flex Message.

This scenario is perfect for a Demo because everyone knows the two most common things in group buying: changing your mind after ordering, and suddenly not wanting to drink after ordering.

More importantly, the audience can directly see from the quantity and amount whether the webhook actually affected the backend state, rather than the Bot just replying "Edit received."

Structure of the Edit Event

The type of the Edit event is messageEdited, which carries the edited text, timestamp, reply token, and message ID:

{
  "type": "messageEdited",
  "replyToken": "950e63e8f46542ab89f645b4c2a1180a",
  "message": {
    "type": "text",
    "id": "610830548529053697",
    "text": "Pearl Milk Tea / Micro Sugar / No Ice / 2"
  },
  "timestamp": 1776914799524,
  "source": {
    "type": "group",
    "groupId": "Ca56f94637c...",
    "userId": "U4af4980629..."
  }
}

The most critical design point is: the message.id in the edit event is the same as the message ID of the original message event.

Therefore, we can directly use the message ID as the order ID:

case webhook.MessageEvent:
    h.handleMessage(e)
case webhook.MessageEditedEvent:
    h.handleEdit(e)
case webhook.UnsendEvent:
    h.handleUnsend(e)

Add an order when the original message event is received; when MessageEditedEvent is received, use the same message ID to find the order and overwrite the content.

Additionally, the Edit event has its own reply token, which is different from the original message event's reply token, so the Bot can directly reply "Order successfully transformed" to this specific edit.

If your Bot also uses the Mark as Read API, note that the Edit event does not carry a markAsReadToken. You cannot simply apply the standard message event processing flow. These field differences are well-suited for a full test using webhook fixtures after upgrading the SDK.

For complete fields, please refer to Messaging API: Edit event.

The First Trap: Edit Events May Not Arrive in Order

This is the most easily overlooked part of this implementation.

A user might edit the same message multiple times in quick succession, and multiple messageEdited webhooks are not guaranteed to arrive in the order they were edited. Official documentation recommends using the largest timestamp to represent the latest state.

Therefore, you cannot simply adopt the "last received webhook"; you should adopt the "webhook with the latest timestamp":

if timestamp <= current.UpdatedAt {
    return summarize(buy), errStaleEdit
}

current.UpdatedAt = timestamp
buy.Orders[messageID] = current

Assuming the second edit arrives first and the first edit arrives later, this logic prevents old content from overwriting new content.

At the same time, the LINE Platform might resend webhooks, so the example also uses webhookEventId for deduplication. These two mechanisms handle different problems:

webhookEventId: Prevents the same event from being processed twice.
timestamp: Prevents different edit events from updating data in the wrong order.

Both need to be handled.

The Second Trap: Edited Content May No Longer Be a Valid Order

Users don't necessarily just change the quantity. They might also change the original order to something like:

I suddenly don't want to drink anymore

In this case, the new message can no longer pass the order format validation. If the backend continues to keep the old order, it will still cause state inconsistency.

The approach in this Demo is: clear the old order content, mark this record as invalid and temporarily remove it from the total, and then ask the user to continue editing to fix it.

parsed, err := parseOrder(text)
if err != nil {
    current.Item = ""
    current.Sugar = ""
    current.Ice = ""
    current.Quantity = 0
    current.UnitPrice = 0
    current.Valid = false
    current.UpdatedAt = timestamp
    buy.Orders[messageID] = current
    return summarize(buy), err
}

Actual products can adopt different strategies, such as putting it into a "waiting for manual confirmation" state; the key is not to silently continue using old content that no longer exists in the chat room.

Unsend Event: Beyond Technology, User Intent

The content of the Unsend event is simpler than the Edit event:

{
  "type": "unsend",
  "source": {
    "type": "group",
    "groupId": "Ca56f94637c...",
    "userId": "U4af4980629..."
  },
  "unsend": {
    "messageId": "610830548529053697"
  }
}

It only tells us which message ID was unsent; it does not re-attach the message content, nor does it have a reply token.

Since we already use the message ID as the order ID, deletion is straightforward:

delete(buy.Orders, messageID)

But what's truly important here is not delete(), but how to respect the user's intent of "I want to take back this content."

Official documentation specifically reminds service providers that after receiving an Unsend event, they should handle it carefully so that the target message cannot be seen or used in the future. Therefore, this Demo will not quote the unsent item, sweetness, or other original text in the Bot's reply, but will only say:

💨 An order has been safely withdrawn, and the saved content has been deleted.

Since the Unsend event has no reply token, if you want to proactively notify the group, you can only use a push message. This means it will count towards message usage, so quota and cost should be considered when designing a formal service.

For full details, please refer to Messaging API: Unsend event.

Making State Changes Visible with Flex Message

While plain text can show results, to make the Demo understandable at a glance, I highly recommend using Flex Message to display the current order.

The card this time includes:

Store name and deadline
Group buy status (In progress or Closed)
Each member's item, sweetness, ice level, quantity, and subtotal
Total quantity and total amount
"Refresh Order" button

After every addition, edit, or unsend, the Flex Message is regenerated from the current state. This way, the quantity and amount before and after an edit change immediately, and the line item disappears after an unsend.

To avoid the Flex Message becoming too large, the example displays a maximum of eight orders, with the rest summarized. Formal products can switch to carousels, LIFF pages, or add pagination queries as needed.

SDK and Execution Environment Versions

This example uses:

github.com/line/line-bot-sdk-go/v8 v8.22.0
Go 1.25

Older versions of the SDK may already have UnsendEvent, but they might not include the new MessageEditedEvent type. After upgrading the SDK, you also need to check the required Go version; when upgrading from the old example this time, the Go toolchain and CI workflow both needed adjustment.

This is a common but easily missed upgrade issue: being able to compile locally doesn't mean Cloud Build or GitHub Actions are still using the same version.

State Issues When Deploying to Cloud Run

This Demo is deployed on Google Cloud Run. The channel secret and channel access token are injected via Secret Manager and are not committed to the repository.

To keep the example simple, current orders are stored in the program's memory. This brings two limitations:

If the instance restarts or scales to zero, orders will disappear.
When multiple instances exist simultaneously, each instance will have a different order state.

Therefore, the Demo environment sets the maximum instances to 1 to avoid webhooks for the same group buy being split. But this is only suitable for demonstration, not a complete solution for a production environment.

Formal services should use Redis, Firestore, or other shared storage, and further handle:

Atomic updates and concurrency control
Webhook idempotency
Data retention periods
Cross-system deletion after unsend
Push Message failure retries
The boundary between audit logs and privacy requirements

Especially for unsend, if message content has already been sent to search indices, analysis platforms, or other downstream services, deleting only the main database is incomplete. Data flow design should know where content went from the start to be able to truly complete a deletion.

How to Run This Demo

First, please enable webhooks in the LINE Developers Console and allow the Bot to join group chats. Point the webhook URL to the /callback of your deployed service.

Then, enter the following in the group:

Start Group Buy 50 Lan 15:20 Deadline
Pearl Milk Tea / Half Sugar / Less Ice / 1
Current Orders

Then directly edit the second message:

Pearl Milk Tea / Micro Sugar / No Ice / 2

You will see the Bot reply "Order successfully transformed," and the quantity and total amount in the Flex Message will also update.

Finally, unsend that order message, and the Bot will delete the content and show a new order summary.

The menu and prices in the example are fixed Demo data to make amount changes clearly visible. For complete startup instructions, environment variables, and test commands, please refer to the README in the repository:

https://github.com/kkdai/linebot-edit-unsend

What Else Can This Be Applied To?

Group buying is just one easy-to-understand story. The same event model can be extended to:

Itinerary Bots: Update the itinerary when the meeting time is edited, and cancel the activity when the message is unsent.
Task Bots: Edit the assignee or deadline, and remove the task when the message is unsent.
Reservation Bots: Modify the time slot or number of people, and cancel the reservation when the message is unsent.
Announcement Bots: Update announcement content, and synchronize the removal of copies on other channels after unsend.
Interactive Story Bots: Rewrite the plot when a choice is edited, and return to the previous node when a message is unsent.

As long as a Bot has ever converted "a message" into some kind of system state, it's worth re-checking: when the message is edited or unsent, should that state also change?

Conclusion

Message editing looks like an improvement to the chat interface, but from a Bot developer's perspective, it actually changes the lifecycle of an event.

A message is no longer just the moment it is "sent." It might be updated, or it might be withdrawn. Backend services need to understand these events, maintain the correct order, and respect the user's latest intent.

This time, I used an afternoon tea group buy story to string together Edit event, Unsend event, Flex Message, and Cloud Run. I hope this small example helps everyone master the new features faster and start thinking about how their own LINE Bots should respond after a message is changed.

Feel free to refer to the code, and I look forward to seeing everyone create more interesting applications:

https://github.com/kkdai/linebot-edit-unsend

References

[AI in Practice] Gemini 3.5 Transcribe: Real-time Transcription and Speaker Diarization in a macOS Meeting Translation App

Evan Lin — Fri, 28 Aug 2026 15:53:34 +0000

Previously

I have a macOS App I use myself, gemini-live-translate-macos. It uses ScreenCaptureKit to directly capture audio from a specified App, eliminating the need for virtual sound cards like BlackHole. It then sends the audio to the Gemini Live API for real-time translation, outputting Traditional Chinese subtitles while playing Chinese audio. I've written two posts about the development process: the first one was about building it from scratch using AGY CLI, and the second one was about using Claude Code to take it from "functional" to "user-friendly."

The starting point for this new addition was simple: I saw a document for "Real-time Transcription" added to the Live API. Since I was already connected to the Live API, I thought adding a pure transcription mode would just be a matter of changing a few parameters.

However, after checking the documentation, I realized that Google released two models with very similar names but very different capabilities at once. The specific feature I actually wanted (speaker diarization) wasn't available at all on the model I originally thought it was.

Two Models with Names Differing by Only Two Words

Let's lay out the differences first; this is the part I spent the most time figuring out:

	`gemini-3.5-transcribe-live`	`gemini-3.5-transcribe`
API Used	Live API (WebSocket streaming)	Interactions API (Standard HTTP request)
Usage Scenario	Transcribe while speaking	Upload the whole file after recording
Speaker Diarization	Not supported	Up to 8 speakers
Word-level Timestamps	Not supported	Supported
Audio Length	10 minutes per session	1 hour (30 mins with diarization)
Smart Mode	`SMART` available	`smart` is mutually exclusive with diarization
Interim Subtitles	Has `interimInputTranscription`	Not applicable

The official documentation on the Live page's limitations section is very blunt:

Speaker diarization is not supported in live streaming sessions. For speaker diarization, use the non-streaming Audio transcription endpoint.

So, "seeing who is saying what in real-time" is currently impossible. For speaker diarization, you must record it and send the whole thing after the meeting. This limitation determined my entire subsequent architecture.

The Real-time One: Interim is the Key

The setup shape is different from the original translation model. responseModalities must be TEXT, and transcription parameters are placed under setup.inputAudioTranscription:

{
  "setup": {
    "model": "models/gemini-3.5-transcribe-live",
    "generationConfig": { "responseModalities": ["TEXT"] },
    "inputAudioTranscription": {
      "languageCodes": [],
      "mode": "SMART"
    },
    "realtimeInputConfig": {
      "automaticActivityDetection": { "disabled": false }
    }
  }
}

Leaving languageCodes empty enables automatic language detection; filling it with BCP-47 codes like ["en-US"] gives it a language preference. mode has two values: VERBATIM keeps everything word-for-word, while SMART removes filler words like "uh" and "um" and automatically adds formatting. For meeting minutes, I chose SMART, which is much cleaner to read.

There is also a customVocabulary where you can stuff technical terms, with a limit of 1000 items, though the documentation suggests staying under 100 for best results. I didn't implement this yet; I'll wait until I encounter names that are constantly misheard.

The response side adds a field that the original translation model didn't have:

interimInputTranscription: Tentative results while speaking, which will be overwritten by subsequent content.
inputTranscription: The finalized text when the speaker pauses or the turn ends.

This distinction affects how the UI is written, which I'll discuss later.

The Batch One: Uses a Completely Different API

This is the easiest place to trip up. gemini-3.5-transcribe doesn't use generateContent; it uses the Interactions API:

POST https://generativelanguage.googleapis.com/v1beta/interactions

{
  "model": "gemini-3.5-transcribe",
  "input": [
    { "type": "audio", "uri": "YOUR_FILE_URI", "mime_type": "audio/wav" }
  ],
  "generation_config": {
    "transcription_config": {
      "language_codes": [],
      "mode": {
        "type": "verbatim",
        "diarization_mode": "speaker",
        "timestamp_granularities": ["word"]
      }
    }
  }
}

diarization_mode: "speaker" is the switch for speaker diarization, but it can only be paired with verbatim. In other words: if you want speaker diarization, you have to give up the benefits of SMART mode's filler word removal; you can't have both.

I didn't include timestamp_granularities at first because it wasn't in the short example in the documentation. Without it, the entire feature silently fails—a process I'll describe in the final section.

Audio must first be uploaded to the Files API to get a file URI, which is then included in the request. The official documentation doesn't provide a base64 embedding example. The response looks like this:

{
  "steps": [{
    "type": "model_output",
    "content": [{
      "type": "text",
      "text": "Hello world",
      "annotations": [
        { "type": "word_info", "text": "Hello", "speaker": "spk_1",
          "start_offset": "0.100s", "end_offset": "0.450s" }
      ]
    }]
  }]
}

Speaker labels are in the word-level annotations as IDs like spk_1, spk_2. The model doesn't know who is who. The documentation also notes a maximum of 8 speakers, and "attribution for 3 or more people is experimental."

Phase 1: Adding Pure Transcription Mode to the App

This part was simpler than I expected because the App was already parsing inputTranscription and outputTranscription (the translation model returns both for bilingual subtitles). The real change was in the mode branching.

The original connection layer determined it like this:

let isTranslateModel = modelName.contains("live-translate")

A boolean split into two paths. Now needing three, I switched to an enum:

enum LiveMode {
    case translate // gemini-*-live-translate-*: output translated audio + bilingual subtitles
    case transcribe // gemini-*-transcribe-live: text only
    case general // other Live models: rely on systemInstruction for interpretation

    static func from(modelName: String) -> LiveMode {
        if modelName.contains("transcribe") { return .transcribe }
        if modelName.contains("live-translate") { return .translate }
        return .general
    }
}

I also took the opportunity to extract the setup config generation and server response parsing from GeminiLiveConnection into pure functions. That file was getting a bit bloated; extracting these reduced it by over sixty lines, and the extracted parts can be tested directly.

A Preventive Trade-off: Duplicate Words

When parsing responses, I made a decision worth noting. Besides inputTranscription, the parts in modelTurn might also carry the same text. If both are collected, the same sentence will appear twice in the subtitles and export file.

To be honest, I haven't actually seen this happen, but while reading the documentation and old code, I noticed both paths would lead to didReceiveOutputTranscription. I blocked it to avoid the risk:

// Content for pure transcription mode is already provided by inputTranscription.
// Receiving text from modelTurn again would cause the same sentence to appear twice.
if mode != .transcribe,
   let text = part["text"] as? String, !text.isEmpty {
    events.append(.outputTranscription(text))
}

The downside of this preventive defense is that if it never actually happens, this line is dead code that no one understands. So I wrote it as a test ("transcribe mode ignores modelTurn text"), ensuring the behavior is locked in and the intent is documented.

How to Run Tests in a Project Without a Testing Framework

This project doesn't use an Xcode project file; it uses a build_app.sh that calls swiftc to compile all .swift files into a .app. No SwiftPM means no swift test and no XCTest.

My solution was to use the same trick again: since the extracted parts are pure functions, I wrote a main.swift as an assertion runner. I compile it with those pure functions into an executable and use the exit code to determine success or failure.

swiftc -sdk "$SDK_PATH" -target "${ARCH}-apple-macos13.0" \
  -o "${BUILD_DIR}/run_tests" \
  LiveSetupConfig.swift TranscriptFormatter.swift \
  WAVRecorder.swift AudioChunker.swift \
  GeminiTranscribeService.swift GeminiSummaryService.swift \
  Tests/main.swift

The assertion function itself is less than ten lines:

func checkEqual<T: Equatable>(_ actual: T?, _ expected: T, _ name: String) {
    if actual == expected {
        passedCount += 1
    } else {
        failures.append("\(name) (Actual: \(String(describing: actual)), Expected: \(expected))")
    }
}

This obviously can't compete with a real testing framework—no setup/teardown, no parallel execution, and it won't tell you which line failed. But it runs, it prevents regressions, and it doesn't require converting the whole project to SwiftPM just for testing. This phase ended with 37 assertions; after adding speaker diarization, it grew to 98.

Phase 2: Speaker Diarization Requires More Changes Than Expected

"Adding speaker diarization" sounds like calling one more API, but it actually involves three more tasks: saving audio to a file, uploading it, and then connecting to a completely different API. And because batch transcription might take several minutes, it can't block the original stop process.

The final data flow looks like this:

graph TD
    A[ScreenCaptureKit captures PCM] --> B[Gemini Live API<br/>Real-time Subtitles]
    A --> C[WAVRecorder<br/>Synchronous writing]

    D[Press Stop] --> E[Phase 1: Immediate Output]
    E --> E1[Transcript md]
    E --> E2[AI Meeting Summary]
    E --> E3[Meeting Record Webpage]

    D --> F[Phase 2: Background Execution]
    F --> F1[Collect file → Split if > 30 mins]
    F1 --> F2[Upload to Files API]
    F2 --> F3[Interactions API Batch Transcription]
    F3 --> F4[Delete cloud copy]
    F4 --> F5[Write diarized.md]
    F5 --> F6[Rerun summary with speaker info + infer names]
    F6 --> F7[Regenerate webpage]
    F7 --> F8[Delete local recording]

Splitting it into two phases was intentional. Every step in Phase 2 could fail (upload timeout, quota exhausted, response format mismatch). If it were hooked into the middle of the original process, a failure would mean losing even the basic transcript and summary. Now, Phase 1 remains untouched, and Phase 2 is just appended; if it fails, you just lose one output.

The recording format was also conveniently available: the audio sent to the Live API is already resampled to 16kHz mono 16-bit PCM. Writing that same data to disk results in valid WAV content—just add a 44-byte header, no re-encoding required.

The same defensive mindset was applied to parsing: parseDiarized assumes the response structure based on documentation, but if the actual format differs, I left a fallback—if word annotations aren't found, it falls back to the entire output_text so the transcript isn't completely empty. This fallback actually came in handy, though not in the way I expected, as I'll explain in the last section.

Major Pitfalls and Solutions

Pitfall 1: Chunking cannot be "fill 30 minutes and leave a remainder"

The audio limit for speaker diarization is 30 minutes, but meetings often last over an hour, so chunking is necessary.

My first idea was the most intuitive: fill 30 minutes for one segment and leave the rest as the last segment. While writing tests, I realized this approach had two holes.

A 60-minute and 5-second meeting would be split into 30 mins, 30 mins, and 5 seconds. Sending that 5-second tail for transcription is meaningless and costs an extra upload and API call. What if I merge the tail into the previous segment? Then that segment becomes 30 mins and 5 seconds, exceeding the API limit, and the whole segment gets rejected.

Cause & Solution: Switch to even splitting. Calculate how many segments are needed (round up), then distribute the total length evenly:

let count = (usable + maxChunkBytes - 1) / maxChunkBytes
guard count > 1 else { return [0..<usable] }

var ranges: [Range<Int>] = []
var start = 0
for index in 1...count {
    var end = usable * index / count
    end -= end % blockAlign // Align to 16-bit sample boundary
    if index == count { end = usable }
    ranges.append(start..<end)
    start = end
}

60 mins and 5 seconds becomes two segments of 30 mins and 2.5 seconds? No, that would exceed the limit. Actually, it's ceil(3605 / 1800) = 3, split into three segments of 20 minutes each. There will never be a tiny remainder segment, and it will never exceed the limit.

The end -= end % blockAlign is also necessary. 16-bit mono uses 2 bytes per sample. If you cut on an odd byte, the entire subsequent audio stream's bytes will be shifted by one, resulting in noise when played.

Pitfall 2: spk_1 in Segment 2 is not the same as spk_1 in Segment 1

A problem I thought of only after chunking. Each segment is an independent API call. The model doesn't know what happened in the previous segment, so spk_1 in Segment 2 has no relation to spk_1 in Segment 1; they could be different people.

If you just concatenate the three segments, the reader will naturally assume spk_1 is the same person throughout. This is worse than having no speaker labels: it gives a false sense of certainty.

Cause & Solution: Whenever chunking occurs, append the segment number to the ID to create a namespace:

static func qualifiedLabel(chunkIndex: Int, chunkCount: Int, speaker: String) -> String {
    guard chunkCount > 1 else { return speaker }
    return "Seg\(chunkIndex + 1)-\(speaker)"
}

Insert an explanatory line at the segment boundaries during export:

---

> Segment 2 (Speaker IDs are not continuous with the previous segment)

These IDs are also used for the text sent to the AI for name inference. By sharing the same vocabulary, the model has a chance to link the same person across segments—if someone is called "Evan" in both Segment 1 and Segment 3, it can map them individually rather than being forced to assume the IDs are identical.

Pitfall 3: Joining words results in "H e l l o W o r l d"

The batch API returns word-level annotations, which you have to join into sentences yourself. For English, it's intuitive: join with spaces.

The problem is Chinese. Gemini's Chinese tokens joined with spaces look like "Ni hao shi jie wo men jin tian" (Hello world we today), looking like a word segmentation exercise.

Cause & Solution: When joining characters, check the properties of the characters on both sides. If either side is CJK (Chinese, Japanese, Korean), don't add a space; also, don't add a space before punctuation:

private static func needsSpace(after previous: Character, before next: Character) -> Bool {
    if isCJK(previous) || isCJK(next) { return false }
    if next.isPunctuation { return false }
    return true
}

isCJK checks Unicode blocks, covering CJK Unified Ideographs, Kana, Hangul, and full-width characters.

The punctuation rule was added later. Originally I only blocked CJK, but testing ["Hello", ",", "world"] revealed it became Hello , world. You'd never notice this without writing tests because it doesn't "break"—it's just a bit ugly.

Pitfall 4: Two asynchronous processes racing to write the same summary

Phase 1 generates an AI meeting summary after stopping. Phase 2 generates another one after getting the diarized transcript (this time with speaker info, so the "Assignee" field in action items can be filled).

Normally, Phase 2 is definitely slower—it has to upload over 100MB of audio and wait for transcription. But "normally slower" isn't a guarantee. If the Phase 1 summary API happens to hang and retry, while the Phase 2 audio is only one minute long and finishes quickly, the order will reverse. The late-returning old summary from Phase 1 would overwrite the diarized result.

Cause & Solution: Phase 2 waits for Phase 1 to finish before writing:

// Wait for Phase 1 summary to land, otherwise it might return after us and overwrite the diarized result
await minutesTask?.value
guard !Task.isCancelled else { return }

A one-line fix, but you have to first realize that "these two things actually have no guaranteed order." This kind of race almost never appears in testing; it only happens on a day with particularly bad network, leaving the user with a confusingly reverted meeting record.

Pitfall 5: During the TDD RED phase, the test itself crashed

While writing tests for the WAV header, I followed TDD rules: write the test first, create a stub returning an empty Data(), and run it to see it fail. Instead of failing, the entire test program crashed:

Swift/arm64e-apple-macos.swiftinterface:41299: Fatal error:
UnsafeRawBufferPointer.load out of bounds
Trace/BPT trap: 5

My test was trying to read the 24th byte to check if the sample rate was 16000, but the stub returned empty Data, causing an out-of-bounds read.

Cause & Solution: Zero-pad before reading:

let produced = WAVRecorder.header(dataByteCount: 64000)
checkEqual(produced.count, 44, "WAV header is 44 bytes")

// Pad with zeros if length is insufficient, so subsequent fields report failure instead of crashing the test
let header = produced + Data(repeating: 0, count: max(0, 44 - produced.count))

This is a small thing, but it highlighted something I usually ignore: the test program itself must be resilient to the object under test being completely broken. If a test ends in a crash during the RED phase instead of reporting a failure, you only know "something broke," not which of the twelve fields were wrong. After padding with zeros, a single run lists all twelve expected values, allowing me to implement them by following a list rather than guessing.

Pitfall 6: Scroll anchor fails in the new mode

There's a structural difference between real-time transcription and translation modes: translation mode accumulates words into the "current sentence" and pushes to history only when punctuation is reached; transcription mode's inputTranscription is a finalized sentence that goes straight to history.

The original auto-scroll was written like this:

proxy.scrollTo("currentLine", anchor: .bottom)

The ID currentLine was attached to the display block for the "current sentence." In transcription mode, the sentence goes to history as soon as it's finalized, and that block immediately disappears—the scroll target no longer exists, so the screen stays put.

Cause & Solution: Use a bottom anchor that always exists:

Color.clear
    .frame(height: 1)
    .id("bottomAnchor")

This bug had no error message or crash; the "new feature just wouldn't scroll," and it only became apparent when there was enough content to exceed the screen. I found it by reading the view's conditional branches, not by running it.

Pitfall 7: Letting AI guess names—the key is stopping it from hallucinating

Once you have spk_1 and spk_2, the natural desire is to replace the IDs with real names. This is actually feasible—meetings often contain clues like "Evan, how's the progress on your end?" or "I'm Sarah, in charge of frontend." If you give the tagged transcript to the model, it can make the connection.

But this is also where hallucinations are most likely. Models are happy to "infer" a name from tone, job content, or speaking frequency and present it with the same confidence as a fact in the meeting minutes.

Cause & Solution: Three methods combined.

First, the schema explicitly allows null and requires evidence:

properties["speakers"] = [
    "type": "ARRAY",
    "items": [
        "type": "OBJECT",
        "properties": [
            "label": ["type": "STRING"],
            // Must return null if no clues are found, rather than forcing a name
            "name": ["type": "STRING", "nullable": true],
            "evidence": ["type": "STRING", "nullable": true]
        ],
        "required": ["label"]
    ]
]

Second, the prompt sets strict rules:

Only fill in the name if there is a clear address, roll call, or self-introduction in the transcript or meeting notes. Explain in 'evidence' which sentence led to this conclusion. Do not speculate on names based on tone, job content, or speaking frequency. If no clear clues are found, both name and evidence must return null.

Third, remove the confidence score. I originally designed a confidence field but later removed it. The reason is that a model's self-assessment of confidence is inherently unreliable, and evidence already fully serves this role: if there's evidence, the inference succeeded; if not, it didn't. An extra confidence score just makes people think it's more credible than it is—"It says 0.7 here, so it's probably 70% accurate"—but that 0.7 doesn't come from any real probability distribution.

The output looks like this, with evidence for successful inferences and honest admissions for failures:

## Participants
- **Evan** (spk_1) — Evidence: Addressed as "Evan, how's the progress on your end?" in Segment 3
- spk_2 — Insufficient clues in transcript and notes to identify name

Keeping the ID in parentheses is also intentional. Seeing **Evan (spk_1)** tells you it was inferred; if it's wrong, you can verify it yourself. If it just said **Evan**, it would look like an absolute fact.

Another source of clues: I also send the meeting notes the user wrote on the spot. Notes often already contain a list of participants, which significantly increases the success rate. But a safeguard is needed here—the prompt must clearly state "Notes can only be used to map speaker names; do not treat note content as something someone said in the summary or action items," otherwise your own memos might turn into someone else's speech.

Privacy and Cleanup: Don't leave behind what you don't want to keep

Batch transcription inherently creates two more pieces of data than real-time streaming: the local recording file and the copy uploaded to Google. Both must have a clear disposal plan.

Recording files are only created if the option is checked. If speaker diarization isn't enabled, no file is written, and nothing stays on the disk.
Local recordings are deleted upon successful transcription. A one-hour meeting is about 115MB, which adds up quickly.
Files are kept if transcription fails. This is an intentional exception: if it fails, the recording is kept, and the status bar tells you the path so you can retry or handle it yourself. Deleting it then would be actual data loss.
Cloud copies are deleted immediately after use. Files API files expire automatically after 48 hours, but "it will disappear eventually" and "I'm sure it's gone now" are two different things.

// Delete the cloud copy immediately after use, don't wait 48 hours for auto-expiration
await GeminiFilesUploader.delete(apiKey: apiKey, name: uploaded.name)

Results and Benefits

The numbers for both phases combined:

	Pure Transcription Mode	Speaker Diarization	Post-testing Fixes
New Files	2	4	0
Total Assertions	37	98	108
Commit	`b0e12c5`	`486ba8f`	`686899a`, `1cc41c3`

GeminiLiveConnection.swift, the file that originally did everything, was reduced by over sixty lines after setup generation and response parsing were extracted. Those two pure function modules are now guarded by over 20 tests. This was an unexpected benefit: the decoupling done to make things testable was itself a refactoring I had put off for a long time.

First, confirm which model has the feature you want. I initially assumed "Real-time Transcription" would have speaker diarization—they're both transcription models, right? The only difference should be real-time vs. batch. I only found out otherwise after checking the docs, and the difference isn't just a parameter switch; it's architectural: for diarization, you must record, upload, use a different API, accept a 30-minute limit, and give up SMART mode. If I hadn't checked first, I would have hit a whole new subsystem while expecting to just "change a few parameters."

Limitations often dictate the architecture. Almost every design decision this time was forced by limitations: the 30-minute limit forced chunking, chunking forced ID namespaces, the slowness of batching forced two-phase export, and the mutual exclusivity of smart and diarization forced me to choose between a clean transcript and speaker labels. Checking limitations before designing is much easier than designing and then hitting a wall.

The real-time transcription half has been tested in the field. The screenshot above is the result of feeding it a Japanese video; gemini-3.5-transcribe-live automatically detected Japanese and output a Japanese transcript directly without translation, and the meeting record webpage was generated as usual after stopping. Leaving languageCodes empty for auto-detection actually works.

The speaker diarization half had issues during field testing, and in a way I didn't expect: the file was generated, the transcript was correct, and the program reported no errors, but there was no speaker separation. This part deserves its own section.

Postscript: Documentation examples might give you an empty result without errors

After posting, I ran speaker diarization on a real conversation. meeting-2026-08-28-11-54-diarized.md was generated, the content was complete, not a word was missing, but there were no speaker labels from beginning to end—just one continuous block of text.

No error messages, the status bar showed success, and the file had everything it should. This kind of failure is the hardest to debug because it looks like success.

Create a Minimal Reproducible Example First

The first problem was that I had no evidence—the App didn't save the raw response, and the recording of that meeting was automatically deleted because "transcription succeeded." To re-run it, I'd have to start another meeting with no guarantee of reproduction.

So instead of guessing what broke, I first found a way to get a raw response. macOS's built-in say command can use different voices, so I used it to synthesize a two-person conversation:

say -v Alex -o a1.aiff "Hi Samantha, did you finish the quarterly report yesterday?"
say -v Samantha -o a2.aiff "Yes Alex, I sent it to the whole team this morning."
say -v Alex -o a3.aiff "Sure, I will look at the budget section this afternoon."

for f in a1 a2 a3; do afconvert -f WAVE -d LEI16@16000 -c 1 $f.aiff $f.wav; done

The three segments joined together are 17 seconds long with two speakers, in the exact same format as the App's recordings (16kHz mono 16-bit). Then I used curl to go through the entire upload and transcription process, dumping the full JSON.

This step took less than five minutes, but it turned "starting another meeting, running for ten minutes, and not being sure of reproduction" into "changing one field, running for ten seconds, and seeing the difference immediately." A minimal reproducible example is worth the time, especially when the original reproduction path is expensive.

The dumped response looked like this:

{
  "steps": [{
    "content": [{ "text": "Hi Samantha, did you finish...", "type": "text" }],
    "type": "model_output"
  }]
}

811 bytes, complete transcript, zero annotations. So the problem wasn't my parsing; it was the request.

Root Cause 1: Word-level annotations must be explicitly enabled

Using the same audio and the same uploaded file, I changed just one field in the request:

Request Content	Objects with `speaker` in response
Only `diarization_mode: "speaker"`	0
Removed `language_codes: []`	0
Added `timestamp_granularities: ["word"]`	41

Speaker IDs are attached to word_info annotations, and word-level annotations must be explicitly requested in the request to be returned. No request, no annotations; no annotations, no speakers.

I didn't include this field originally because I copied the shortest Python example from the documentation—that example only had type and diarization_mode. The full REST example elsewhere in the documentation actually has timestamp_granularities, but by then I already "knew" how to write it and didn't look back.

The most frustrating part is that it doesn't report an error. The API returns 200, gives you the full transcript, and the status is completed. If it had returned an error like "You requested speaker diarization but didn't enable word annotations," I would have fixed it in five minutes.

Root Cause 2: The actual label format differs from the documentation

After adding the field, annotations appeared, but they looked different from what I expected:

{"text":"Hi,","start_offset":"0.100s","end_offset":"0.500s","speaker":"spk:0","type":"word_info"}

spk:0. A colon, and starting from 0. The documentation consistently uses spk_1, spk_2.

My display logic was written like this:

if let range = speaker.range(of: "spk_"), let number = Int(speaker[range.upperBound...]) {
    return "Speaker \(number)"
}
return speaker // ← Fallback prints the raw ID if no match

spk:0 doesn't match spk_, so it fell straight into the fallback, displaying **spk:0**: on the screen.

My tests completely missed this bug for a simple reason: the test data was written according to the documentation. The documentation was wrong, so the tests were wrong, and the green light told me everything was fine. This is the main thing I want to record: for external API tests, you are actually testing "my understanding of this API," not the API itself. If your understanding is wrong, the test will faithfully protect that error.

I didn't fix it by changing spk_ to spk:; that would just be betting in a different direction. Instead, I stopped parsing the number in the ID and used the order in which the speaker first appeared in that segment:

/// The actual format of the ID is determined by the API (actual returns spk:0, spk:1, while official docs say spk_1),
/// so we don't parse the number in the ID. Instead, we number them based on their first appearance in this segment.
static func speakerOrder(in chunk: [DiarizedSegment]) -> [String: Int] {
    var order: [String: Int] = [:]
    for segment in chunk where !segment.speaker.isEmpty {
        if order[segment.speaker] == nil {
            order[segment.speaker] = order.count + 1
        }
    }
    return order
}

It won't break even if the format changes again because it doesn't look at the format at all.

My own fallback made failure look like success

I wrote this earlier and was quite proud of it:

parseDiarized assumes the response structure based on documentation, but if the actual format differs, I left a fallback—if word annotations aren't found, it falls back to the entire output_text so the transcript isn't completely empty.

This fallback did work, and the effect was as expected: the user got a complete transcript with nothing lost.

But it also hid the failure. If I hadn't had that fallback, -diarized.md would have been empty or not generated at all, and I would have known immediately that something was wrong. With it, I got a file that looked perfectly normal, just missing the feature I wanted—the very feature that was the sole reason for enabling it.

I haven't fully figured out the balance here. Graceful degradation isn't wrong; the mistake is not speaking up after degrading. My current approach is to keep the fallback but change the status bar message when it's triggered, explicitly stating "Speaker info not obtained this time" instead of the usual "Diarized transcript saved."

After the Fix

Running the full parsing with a real response, both segments spoken by Alex correctly returned to Speaker 1:

**Speaker 1**: Hi, Samantha. Did you finish the quarterly report yesterday?
**Speaker 2**: Yes, Alex. I sent it to the whole team this morning. Could you review the budget section?
**Speaker 1**: Sure. I will look at the budget section this afternoon and get back to you.

The quality of the speaker diarization itself is good. Tests increased from 98 to 106, with the extra eight being regression tests for these two root causes—this time, the test data wasn't copied from the docs but clipped from real responses.

Three more details learned from field testing:

The REST endpoint only accepts snake_case; sending generationConfig is explicitly rejected with Unknown parameter 'generationConfig'. Did you mean 'generation_config'?. This is a great error message, a hundred times more useful than the silent failure above.
Annotations include start_index and end_index, mapping directly to positions in content.text. Using these to split strings is much more accurate than my heuristic for joining CJK/English words, meaning the entire joinWords could be removed. I haven't done this yet.
The state of uploaded files is immediately ACTIVE; 17 seconds of audio didn't go through a PROCESSING phase. My polling logic seems redundant for short audio, but I don't know if it's needed for long audio, so I'll keep it for now.

This article thus has two endings, and I've decided to keep both. The first part said "Speaker diarization hasn't been verified yet; I'll update after I run it," and then I actually ran it, and it broke. If I had edited the first part and only kept the fixed version, this would have been a smooth "I did X, and it worked" post—but what actually happened was "I did X according to the docs, it failed silently, and it took me half an hour to find out why." The latter is much more useful to readers.

The code is at kkdai/gemini-live-translate-macos. The two official documents are Live transcription and Audio transcription. I recommend reading the limitations section on the speaker diarization page thoroughly before starting.

[Go in Practice] Writing Modern Go with AI: Testing JetBrains go-modern-guidelines and Refactoring a 1,039-line main.go

Evan Lin — Thu, 27 Aug 2026 04:13:56 +0000

Background

In the AI era, even I delegate most of my code optimization or writing tasks to AI. However, due to factors in model training data, too many writing styles are outdated. This results in code that cannot utilize features of the latest Go versions, which is quite a pity.

Fortunately, JetBrains released go-modern-guidelines, a very useful plugin. It makes your AI Agent smarter and teaches it how to use the latest syntax to optimize your Golang code.

What is go-modern-guidelines?

The problem it aims to solve: Models have a knowledge cutoff, Go doesn't

The positioning of this project is very straightforward: Provide contemporary Go writing specifications for AI agents so they don't write outdated Go due to knowledge cutoffs.

The problem has two layers. The first layer is easy to understand: training data has a cutoff. Anything added to the standard library after that cutoff won't be used because the model hasn't seen it. The project's own example is errors.AsType[T] (Go 1.26); if the model hasn't seen it, it naturally won't write it.

The second layer is more subtle, which the project calls frequency bias: even if the model "knows" the new way, the old way appears overwhelmingly more often in the training data. In ten years of Go code on the internet, interface{} appears far more than any, and sort.Slice far more than slices.SortFunc. Models perform probabilistic prediction; the one that wins by majority vote is usually the old one.

I really saw this second point in this refactoring. The original project had this snippet:

// The oauth2 library can return an error containing "invalid_grant"
// when the refresh token is expired, revoked, or otherwise invalid.
if err != nil {
    errorStr := err.Error()
    // Basic substring check to avoid importing "strings"
    for i := 0; i <= len(errorStr)-13; i++ {
        if errorStr[i:i+13] == "invalid_grant" {
            return true
        }
    }
}

A hand-rolled string search, with a comment specifically explaining "to avoid importing strings". strings is in the standard library; the cost of importing it is zero. What this code actually needed was just one line: strings.Contains(err.Error(), "invalid_grant").

How it works: Two commands, a list that grows with Go versions

The tool itself is a CLI with only two subcommands:

list [--go-version <version> | --file-path <path>]
    Returns a list of guidelines supported by this Go version, sorted from newest to oldest.

explain <id>...
    Returns detailed explanations and before/after examples for specific guidelines.

The key point of list is that it provides different answers based on the Go version. You can pass a file path directly, and it will look up for go.mod, go.work, or fall back to the local Go toolchain:

$ go-modern-guidelines list --file-path ~/Documents/linebot-file/main.go

My project's go.mod specifies go 1.24.0, so it returned 45 guidelines. Change the version number, and the count changes:

Go Version	Guideline Count
1.21	32
1.22	37
1.23	41
1.24	45
1.25	46
1.26	48
1.27	54

This design is intentional: it only suggests syntax that your project version can actually use. This is crucial for AI agents; otherwise, it might happily suggest errors.AsType[T], and your CI would fail because it's running on Go 1.24.

Looking at the differences between versions, it's essentially a condensed list of Go's recent features:

$ diff <(list --go-version 1.21) <(list --go-version 1.22)
> range_over_int: Use for i := range n when iterating from 0 to n-1.
> loopvar_capture: Do not add redundant loop-variable copies before closures or
  taking addresses; Go 1.22 gives each iteration its own variables.
> cmp_or: Use cmp.Or to pick the first non-zero value from a fallback chain.
> reflect_type_for: Use reflect.TypeFor[T]() instead of reflect.TypeOf((*T)(nil)).Elem().
> http_servemux_patterns: Use method-aware ServeMux patterns and r.PathValue for
  path parameters.

list provides a one-line summary; use explain for detailed instructions when you're ready to work. Output looks like this:

$ go-modern-guidelines explain cmp_or

cmp_or:
  Since: Go 1.22

  Summary:
    Use cmp.Or to pick the first non-zero value from a fallback chain.

  Details:
    cmp.Or returns the first non-zero value from its arguments. It is concise
    for simple fallback chains, but remember that all arguments are evaluated
    before the call.

  Examples:

  Before:
    name := os.Getenv("NAME")
    if name == "" {
      name = "default"
    }

  After:
    name := cmp.Or(os.Getenv("NAME"), "default")

Note the last sentence in the Details section: "all arguments are evaluated before the call." This is the real trap of cmp.Or—if your fallback source is an expensive function call, writing cmp.Or(a(), b()) will execute both. This kind of "you can use it, but know the cost" reminder is much more useful than just telling you to change the syntax.

The two-layer information design is actually to save context

This list/explain layering might look like just interface design, but it's actually for the AI agent's context window. 45 guidelines, each with a one-line summary, take about 1000 tokens; but if every guideline included full explanations and before/after examples, just stuffing this list would burn tens of thousands of tokens.

So the workflow is: first list to scan everything, determine which guidelines are relevant to the current code, and then only call explain for those. In this case, I actually only explained six guidelines.

graph TD
    A[Prepare to modify Go code] --> B[list --file-path main.go]
    B --> C[Parse Go version from go.mod]
    C --> D[Return 45 guidelines available for that version<br/>one-line summary each]
    D --> E{Which ones are relevant to this code?}
    E -->|Pick candidates| F[explain cmp_or min_max ...]
    F --> G[Get detailed explanations and before/after]
    G --> H[Actually apply to code]
    E -->|None relevant| I[Write in original way]

There's one rule in the skill documentation written with particular emphasis: Do not pipe the output of list to head, tail, or grep, as you might miss important guidelines. I violated this rule on my first try, which I'll discuss in the "Pitfalls" section.

Installation

For Claude Code, it's two lines:

/plugin marketplace add JetBrains/go-modern-guidelines
/plugin install modern-go-guidelines@goland-claude-marketplace

Once installed, it triggers automatically for Go-related tasks, or you can call it manually: /modern-go-guidelines:use-modern-go. Cursor, Junie, and Codex have their own installation methods; other agents can use npx skills add JetBrains/go-modern-guidelines. The project is licensed under Apache 2.0.

On the first run, the wrapper script will automatically install the CLI to the local cache directory:

go-modern-guidelines: installing github.com/JetBrains/go-modern-guidelines@v0.1.1
  into /Users/xxx/.cache/go-modern-guidelines/v0.1.1

This project: a main.go that grew to 1039 lines

First, some background. The architecture of linebot-file isn't complex:

graph LR
    A[LINE App] -->|Send file| B[LINE Platform]
    B -->|webhook| C[Cloud Run]
    C -->|Read token| D[(Firestore)]
    C -->|Upload/Query| E[Google Drive API]

Users authorize with /connect_drive, tokens are stored in Firestore, and files sent to the chatroom are automatically uploaded to a folder structure like LINE Bot Uploads/YYYY-MM/. Features were added incrementally, and everything was piled into main.go, where the main() function itself took up 564 lines.

What the health check found

This section isn't directly related to go-modern-guidelines—that tool manages whether the "writing style is contemporary," not whether the "logic is correct." But these are the things that actually bite users, so I'll record them anyway.

1. Code that compiles but will never execute

This is the most interesting one. The original event handling looked like this:

switch e := event.(type) {
case webhook.MessageEvent:
    switch message := e.Message.(type) {
    case webhook.TextMessageContent:
        // ...
    case webhook.FileMessageContent:
        // ...
    case webhook.FollowEvent: // ← Note the indentation level here
        if s, ok := e.Source.(*webhook.UserSource); ok {
            bot.LinkRichMenuIdToUser(s.UserId, richMenuConnect)
        }
    }
}

webhook.FollowEvent was written inside the inner switch. The inner switch evaluates e.Message, which is of type MessageContentInterface—a follow event can never be the content of a message.

Why did it compile? Go does check type switches; if a case type cannot possibly implement that interface, the compiler reports impossible type switch case. The problem lies in the SDK's interface definition:

type MessageContentInterface interface {
    GetType() string
}

It only requires a GetType() string. And FollowEvent happens to have this method (all event types do), so in the type system, it "can" be a MessageContentInterface. The compiler allows it, but it never matches at runtime.

Actual consequence: When new users add the bot as a friend, the Rich Menu for guiding authorization was never bound. This feature had probably been broken for a long time because it doesn't report an error; it just quietly does nothing.

2. Group messages cause a panic

userID := e.Source.(webhook.UserSource).UserId

Unchecked type assertions appeared six times. As long as the bot is pulled into a group and someone sends an image, this line panics.

By the way, several other places in the same file used e.Source.(*webhook.GroupSource) (pointer). Checking the SDK's UnmarshalSource, it returns a value, not a pointer, so those assertions with , ok were always false—also dead code. Two different wrong ways in the same file, in opposite directions.

3. /recent_files returns folders

// When uploading: file is placed in LINE Bot Uploads/YYYY-MM/
monthFolderID, _ := findOrCreateFolder(srv, "2026-08", mainFolderID)
srv.Files.Create(&drive.File{Parents: []string{monthFolderID}})

// When querying: only look under LINE Bot Uploads
query := fmt.Sprintf("'%s' in parents and trashed=false", mainFolderID)

Files are stored in monthly subfolders, but the query only looks at the root folder. In the Google Drive data model, a folder is also a type of file, so this query does return things—it returns the 2026-08, 2026-07 folders themselves.

4. User input directly concatenated into the Drive query

query := fmt.Sprintf("... and name contains '%s'", searchQuery)

No escaping. If a user searches for it's, that single quote breaks the query syntax; thinking further, extra query conditions could be injected. The fix is to properly write an escaping function, noting that the order cannot be reversed:

// Backslashes must be escaped first, otherwise the backslash added 
// to escape a quote will be escaped again in the second round of processing.
func escapeDriveQuery(s string) string {
    s = strings.ReplaceAll(s, `\`, `\\`)
    return strings.ReplaceAll(s, `'`, `\'`)
}

5. /quit treated as a search command

} else if (len(message.Text) > 13 && message.Text[:13] == "/search_files") ||
          (len(message.Text) > 2 && message.Text[:2] == "/q") {
    commandPrefixLen := 0
    if ... {
        commandPrefixLen = 14 // Length of "/search_files "
    } else if ... {
        commandPrefixLen = 3 // Length of "/q "
    }
    searchQuery = message.Text[commandPrefixLen:]

Manual string slicing, and it hardcoded "must be followed by a space." If a user types /quit, the first two characters are /q, so it becomes a search for it.

What go-modern-guidelines actually changed

Back to the topic. Out of those 45 guidelines from list, these were the ones actually applied in this change:

`http_servemux_patterns`: Also removed a piece of manual path checking

The original way was to have all requests go to the same handler and then determine the path manually:

http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
    // LINE Platform must POST to the webhook URL
    if req.URL.Path != "/" {
        http.NotFound(w, req)
        return
    }
    // ...
})

After Go 1.22, ServeMux patterns support methods and exact paths:

mux := http.NewServeMux()
// "/{$}" only matches the root path, not all paths beneath it
mux.HandleFunc("POST /{$}", webhookHandler)
mux.HandleFunc("GET /oauth/callback", oauthCallbackHandler)
// Cannot be called /healthz, see Pitfall 5 for the reason
mux.HandleFunc("GET /health", healthHandler)

The {$} syntax is key: "/" in ServeMux is a subtree pattern that consumes all paths beneath it, which is why the original code needed that manual check. "/{$}" only matches the root path itself, so the check is no longer needed. I also added method restrictions and a health check endpoint while I was at it.

The health check endpoint issue came up later, but that was discovered after deployment; I'll save that for Pitfall 5.

`cmp_or`: Three fallback segments turned into three lines

// Before
port := os.Getenv("PORT")
if port == "" {
    port = "5000"
}

// After (also changed default to 8080 to align with Dockerfile EXPOSE and Cloud Run convention)
port := cmp.Or(os.Getenv("PORT"), "8080")
richMenuConnect = cmp.Or(os.Getenv("RICH_MENU_CONNECT"), defaultRichMenuConnect)
richMenuMain = cmp.Or(os.Getenv("RICH_MENU_MAIN"), defaultRichMenuMain)

This exactly fits the usage conditions reminded by explain: all three parameters are os.Getenv and constants, and evaluating them all has no side effects.

`strings_cut_prefix_suffix`: Replacing manual string slicing

The command parsing from point five earlier, message.Text[:13], was replaced with a proper parsing function:

func parseCommand(text string) (name, arg string, ok bool) {
    text = strings.TrimSpace(text)
    if !strings.HasPrefix(text, "/") {
        return "", "", false
    }

    name, arg, _ = strings.Cut(text, " ")
    switch name {
    case cmdConnect, cmdReconnect, cmdDisconnect, cmdRecent, cmdSearch, cmdSearchShort:
        return name, strings.TrimSpace(arg), true
    }
    return "", "", false
}

Changing to strings.Cut to first slice out the full command name and then using a switch for comparison structurally eliminated the /quit bug—the sliced name is /quit, which isn't in the allowed list, so it returns false directly.

`slices_sort_func` + `min`: Fixing that fake sorting

The original search results looked like this after deduplication:

// Remove duplicates and sort by creation time (newest first)
uniqueFiles := make(map[string]*drive.File)
for _, file := range files {
    if _, exists := uniqueFiles[file.Id]; !exists {
        uniqueFiles[file.Id] = file
    }
}

result := make([]*drive.File, 0, len(uniqueFiles))
for _, file := range uniqueFiles {
    result = append(result, file)
}

if len(result) > 10 {
    result = result[:10]
}

The comment says "sort by creation time (newest first)", but in reality, there was no sorting action at all—map iteration order is random, and then it just truncated the first 10 items. So users got 10 random items, not the 10 newest ones.

// createdTime returned by Drive is an RFC 3339 UTC string; direct string comparison is the correct chronological order
func sortAndTrimFiles(files []*drive.File, limit int) []*drive.File {
    slices.SortStableFunc(files, func(a, b *drive.File) int {
        return cmp.Compare(b.CreatedTime, a.CreatedTime)
    })
    return files[:min(len(files), limit)]
}

The min built-in function (Go 1.21) saves an if here.

`any`, `errors_is`: Small details

Changing map[string]interface{} to map[string]any; no need to say more about such one-liners.

`crypto/rand.Text()`: A suggestion that requires a version upgrade first

Original way to generate OAuth state:

func generateState() string {
    b := make([]byte, 16)
    rand.Read(b) // Error ignored
    return base64.URLEncoding.EncodeToString(b)
}

crypto/rand.Text(), added in Go 1.24, returns a random string directly, won't fail, and the output is base32 (A-Z, 2-7), which is naturally URL-safe—perfect for states and Firestore document IDs:

func generateState() string {
    return rand.Text()
}

But there's a prerequisite for this one, which I'll discuss below.

Major Pitfalls and Solutions

Pitfall 1: The skill explicitly tells you not to grep, and I did exactly the opposite the first time

The skill documentation is clear:

Do not pipe the output through head, tail, grep, sed, or any other truncating/filtering command. Important guidelines may otherwise be missed.

The first time I called it, I typed:

$ go-modern-guidelines list --file-path main.go 2>&1 | tail -60

Purely a reflex to avoid flooding the screen with long output. In hindsight, I realized this was dangerous on two levels: first, list explicitly states it is sorted from newest to oldest, so tail gets exactly the oldest batch; second, I only got away with it this time because the go.mod specified 1.23, totaling 41 lines, which is less than 60, so tail -60 printed everything.

Reason and Solution: Pure luck. If the project had been Go 1.27 (54 guidelines), tail -60 still wouldn't have truncated; but if I had typed head -20 or grep slices, I would have missed entire batches of items with absolutely no hint that I missed anything. This kind of "output truncated but looks normal" failure is the hardest to detect. Just read the full output honestly; it's only 45 lines.

Pitfall 2: Your go.mod might not allow the syntax suggested by the tool

rand.Text() appeared in the suggestion list, but the project's go.mod at the time was:

module github.com/kkdai/linebot-file

// +heroku goVersion go1.21
go 1.23.0

toolchain go1.24.3

The go 1.23.0 line determines the language version, which is different from the toolchain. The tool answers based on the version it can parse, but to actually use rand.Text(), you have to modify go.mod.

It's not just about brainlessly changing one line; you have to ensure everything in the chain aligns: the toolchain was already go1.24.3, the Dockerfile used golang:1.24-alpine, both were fine. However, the CI had an issue—.github/workflows/go.yml hardcoded go-version: '1.22', which is older than what go.mod required; it was currently only not breaking because of Go's automatic toolchain download mechanism.

Reason and Solution: Updated go.mod to go 1.24.0, cleared out that outdated // +heroku goVersion go1.21 line (this project has long been running on Cloud Run), and changed the CI to use go.mod as the single source of truth:

- uses: actions/setup-go@v5
  with:
    # Use go.mod as the single source of truth to avoid CI and project version inconsistency
    go-version-file: go.mod

Pitfall 3: After changing go.mod, the tool's answers changed

This was the most interesting discovery this time. After upgrading go.mod, I ran list again before writing tests and found four new items at the top of the list:

testing_t_context: Use t.Context() when a test function needs a context tied to
                   the test lifetime.
json_omitzero: Use omitzero on JSON-tagged bool, numeric, struct, and time
                   fields whose zero value should be omitted...
testing_b_loop: Use b.Loop() for the main loop in benchmark functions.
strings_split_seq: Use strings or bytes SplitSeq and FieldsSeq helpers...

These four are exactly what was added in Go 1.24. And testing_t_context directly changed the test I was currently writing:

// Before
srv, err := drive.NewService(context.Background(),
    option.WithEndpoint(server.URL), option.WithoutAuthentication())

// After — context bound to test lifetime, automatically canceled when test ends
srv, err := drive.NewService(t.Context(),
    option.WithEndpoint(server.URL), option.WithoutAuthentication())

Reason and Solution: The output of this tool changes according to the project state; it's not a static document. Upgrade the version or switch projects, and the answers change. So the correct usage isn't to check once before starting and be done, but to rerun it when the nature of the changes shifts—in my case, I reran it at the junction of "finished main program, starting tests," and happened to catch testing_t_context. If I had only checked at the very beginning, I would have missed it.

Pitfall 4: The tool manages syntax, not architecture—and architectural pitfalls are deeper

This is the reverse: where go-modern-guidelines doesn't and shouldn't have an opinion.

Originally, file uploads were done synchronously in the webhook handler: downloading a video from LINE and then uploading to Drive could take dozens of seconds. LINE expects a response within a certain time; if it times out, it retries, and retries would cause duplicate uploads of the same file.

The standard advice for this is almost reflexive: return 200 first, and throw the rest into a goroutine. I thought the same at first, but halfway through writing, I remembered something—this service runs on Cloud Run, which by default only allocates CPU during request processing. Once the response is sent, that goroutine will be throttled by the CPU, becoming a black hole that looks like it's working but actually has no idea when it will finish. This is worse than synchronous processing; at least synchronous processing fails honestly.

Reason and Solution: Changed to use the webhook's event ID for deduplication so that retries don't cause duplicate uploads, while keeping synchronous processing:

// handledEvents remembers recently processed webhook event IDs. LINE will resend
// requests it considers failed; without this protection, resending would upload the same file again.
type handledEvents struct {
    mu sync.Mutex
    seen map[string]time.Time
}

func (h *handledEvents) markHandled(id string) bool {
    if id == "" {
        return true // Without an ID, there's no way to deduplicate, so treat as a new event
    }
    // ... clear expired ones, then check for duplicates
}

Added a fallback of "if the reply token expires, use push message" so that the user is still notified after a large file finishes uploading.

This is a compromise; the real solution is to use Cloud Tasks or Pub/Sub. I've added it to the project roadmap, including the reason "can't just use goroutines"—otherwise, the next person taking over (likely me in three months) will probably hit the same pitfall again.

Pitfall 5: ServeMux was written correctly, but Cloud Run won't let you use it

After the PR was merged, I checked Cloud Build with gcloud; the status was SUCCESS, the new revision was Ready, and all traffic was switched over. It looked like a job well done.

Poked the endpoints:

GET / 405 ← method-aware ServeMux in effect
POST / No signature 400 ← signature verification in effect
GET /nope 404 ← {$} exact match in effect
GET /healthz 404 ← ?

The first three were correct, but the health check returned 404.

At first, I thought I wrote the route wrong, but I only realized something was off after printing the response content—it was a Google-branded HTML error page (Error 404 (Not Found)!!1, with the Google robot image), not Go's 404 page not found plain text. This meant the request never even reached my program.

Checking the Cloud Run request logs confirmed this:

15:44:54 GET 400 /oauth/callback
15:44:36 GET 404 /nope
15:44:36 POST 400 /
15:44:36 GET 405 /

I sent five requests, but there were only four in the log. The two /healthz requests didn't even have a record.

Scanning various common health check paths narrowed it down significantly:

/healthz 404 GFE(Google) ← Intercepted
/healthz/ 404 app(Go) ← Only one slash difference
/health 404 app(Go)
/readyz 404 app(Go)
/livez 404 app(Go)
/_ah/health 404 app(Go)
/status 404 app(Go)
/ping 404 app(Go)
/healthcheck 404 app(Go)

Only the exact path /healthz was intercepted by Google Frontend; even adding a slash allowed it to reach the app normally. I looked it up and found this is a known behavior of Cloud Run, which Streamlit and n8n have also encountered.

Reason and Solution: Changed the endpoint to /health, a one-line fix. The annoying part is that this pitfall doesn't make a sound—go vet doesn't speak, tests don't speak, CI is all green, build succeeds, Cloud Run shows Ready, and even request logs leave no trace. The only way to find it is to actually poke the endpoint and notice that the returned 404 looks different from the one your program returns.

So when fixing it, I left a comment in the code and wrote a section in the README:

// Not "/healthz": Cloud Run's frontend reserves that exact path and
// answers it with its own 404, so the request never reaches us.
mux.HandleFunc("GET /health", func(w http.ResponseWriter, _ *http.Request) {

Without this line, the next person who sees /health and thinks "this isn't the convention, it should be healthz" (very likely myself) will change it back.

Pitfall 6: I thought I added context to all external calls, but I missed an entire path

While fixing /healthz, I scanned the code again and found something even more embarrassing.

One change I was very satisfied with earlier was "using context throughout the process, with timeouts for all external calls." Firestore had it, Drive had it. Then I grepped all external calls:

webhook.go:312 blob.GetMessageContent(messageID)
line.go:73 bot.ReplyMessage(...)
line.go:94 bot.PushMessage(...)
line.go:118 bot.LinkRichMenuIdToUser(...)

Four LINE calls, none of which took a context. I had to dig into the SDK to find out why:

c := &MessagingApiAPI{
    channelToken: channelToken,
    httpClient: http.DefaultClient, // ← Timeout is zero, meaning no timeout
}

And the method signatures generated by the SDK don't accept context, so the timeout I wrapped in the outer handler had absolutely no effect on these four calls. If the LINE side hangs, the goroutine just hangs indefinitely.

Reason and Solution: The problem wasn't that I didn't know to add a timeout, but that the memory of "I've added context" overrode the fact of "whether this SDK actually accepts context." When modifying Drive and Firestore, I added .Context(ctx) all the way down very smoothly—so smoothly that I didn't stop to think about which other external calls didn't look like that.

The SDK provides injection points:

bot, err = messaging_api.NewMessagingApiAPI(accessToken,
    messaging_api.WithHTTPClient(&http.Client{Timeout: lineAPITimeout})) // 10 seconds

blob, err = messaging_api.NewMessagingApiBlobAPI(accessToken,
    messaging_api.WithBlobHTTPClient(&http.Client{Timeout: lineBlobTimeout})) // 5 minutes

I gave the blob side 5 minutes because it needs to download videos sent by users.

Along with a more hidden trap. The SDK provides something that looks exactly like what I wanted:

func (call *MessagingApiAPI) WithContext(ctx context.Context) *MessagingApiAPI {
    call.ctx = ctx
    return call
}

It directly overwrites the field of a shared structure and then returns the same pointer. My bot is a package-level shared variable; if multiple requests come in simultaneously and each calls WithContext, it's a standard data race. The name sounds like a functional option, but the behavior is mutation. I also left a comment for this line in the code to prevent someone from changing it later thinking it's more precise than WithHTTPClient.

Pitfall 7: Bugs that don't make a sound need to be actively hit by tests

There was another one found in the same round. The uploadParents function is responsible for listing all YYYY-MM month folders, originally written like this:

r, err := srv.Files.List().Q(query).Fields("files(id)").Context(ctx).Do()

No PageSize set. The Drive API defaults to 100 items per page; anything beyond that requires asking again with a nextPageToken. One folder is generated per month, so after 100 months—about 8 years and 4 months—the oldest folders would disappear from the scope of searches and /recent_files. There would be no error, no warning, just fewer results.

Reason and Solution: Changed to use Pages() to iterate through all pages. What I really want to talk about is the next step. I didn't quite trust the pagination logic I just wrote, so I wrote a mock server that returns a nextPageToken and then temporarily changed the implementation back to only fetch the first page to see if the test would fail:

--- FAIL: TestUploadParentsPagesThroughAllSubfolders
    uploadParents() = [root_id month_1 month_2], want [root_id month_1 month_2 month_3]

Confirmed it would fail before restoring the implementation. This step took less than two minutes, but without it, I would only have a "ran and passed" test without knowing if it was actually testing anything. Bugs like silent truncation don't reveal themselves; if the test is also a green illusion, you have nothing.

Results and Benefits

First, the most direct numbers. Originally, main.go was 1039 lines and main() was 564 lines. After splitting into six files:

File	Lines	Responsibility
`main.go`	94	Startup, environment variable checks, routing
`config.go`	74	Constants and shared state
`webhook.go`	344	Event dispatching, command parsing, command handling
`line.go`	164	LINE message assembly
`drive.go`	183	Drive query/upload
`auth.go`	264	OAuth, token, revocation

main() went from 564 lines to 62 lines. Interestingly, the total lines of main code barely changed (1039 → 1123); what significantly increased was the tests: from 88 lines to 468 lines, and the number of tests from 1 to 16, all passing under -race.

In terms of performance, the search function was originally "find the root folder, then check each monthly subfolder one by one," which is 1 + N Drive API calls; after changing to concatenate all parents with or into a single query, it's fixed at 2 calls.

The value of go-modern-guidelines isn't that it taught me syntax I'd never seen. I generally knew about cmp.Or, min, and slices.SortFunc; the problem is that I don't actively think of them while coding—especially when modifying an existing file, where the surrounding old syntax creates a kind of gravity, making it natural to continue writing in the same style. A sentence in the skill documentation hits the mark:

If a guideline applies, follow it even when nearby code or repository convention uses an older pattern.

This sentence is fighting the frequency bias mentioned earlier, and it applies to humans too.

Its boundaries are also very clear. Those five bugs that actually bite users—the wrong-level switch, the panicking type assertion, the query returning folders, the unescaped query, the misjudged /quit—none of them were caught by go-modern-guidelines; that's not its defensive scope. It manages whether "this Go code is contemporary enough," not whether "this logic is correct." Treat it as a supplement to a linter, not a replacement for code review.

The first half of this article was written before deployment. The /healthz pitfall was discovered after the article was finished and the PR merged, when I checked the build status on gcloud. The situation was: all 45 guidelines checked, everything applicable applied, 17 tests passed under -race, all three CI checks green, Cloud Build SUCCESS, and Cloud Run showing Ready with 100% traffic switched. In this entire row of green lights, not a single one told you an endpoint was dead.

I wrote in the previous post about handling Cloudflare that "a successful build cannot be taken as verification," and I thought I remembered it, but I still paid tuition in the same place this time, just on a different layer—last time the build succeeded but the container crashed on startup; this time the container ran fine but was eaten by the outer infrastructure layer. Tools manage syntax, tests manage logic, CI manages whether these two have regressed, but none of them manage "what happens when this thing is deployed to that specific environment." That part you have to poke yourself.

Keep in mind that the output changes with the project state. The phenomenon in Pitfall 3 where "four more suggestions appeared after upgrading go.mod" was the most practical realization this time. It's not a static document to be checked once, but a query interface that answers based on the project's current state. When the nature of the changes shifts (from main program to tests, language version upgrade, project switch), it's worth running again.

Finally, this change is in PR #4, the two fixes added after deployment are in #5 and #6, and the code is at kkdai/linebot-file. The source code for go-modern-guidelines is at JetBrains/go-modern-guidelines, licensed under Apache 2.0.

[LINE Bot Debugging] A 403 Error and Three Underlying Issues: npm Version Drift, Node.js Requirements, and Cloudflare Verificati

Evan Lin — Thu, 27 Aug 2026 02:46:14 +0000

Previously

"URL retrieval failed, help me check the logs."

I receive this kind of report about once every week or two. Usually, it's because a website has added another layer of anti-crawling protection. A quick check and adding a fallback method to bypass it usually settles it. This time started the same way: a user sent an acm.org URL, and the summary feature spat out an error.

Upon investigation, I realized it wasn't that simple. The URL retrieval feature had been silently broken for three days. The problem wasn't the target website; it was an unlocked npm package version in my own Docker image. One day, it quietly upgraded to a new version that required a newer Node.js to run, while the Node.js in the container was still stuck on the version installed six months ago.

First, let's look at the logs:

gcloud logging read 'resource.type="cloud_run_revision" AND \
  resource.labels.service_name="linebot-helper-python" AND \
  textPayload:"acm.org"' --limit=50 --freshness=2d \
  --format="table(timestamp,severity,textPayload)"


ERROR:loader.url:All methods failed for URL: https://www.acm.org/articles/people-of-acm/2026/russ-cox
WARNING:loader.url:cloudscraper failed ...: 403 Client Error: Forbidden for url: ...
WARNING:loader.url:httpx failed ...: Client error '403 Forbidden' for url: ...
WARNING:loader.url:singlefile failed ...: SingleFile exited with code 1
ERROR:loader.singlefile:SingleFile loading failed ...: SingleFile exited with code 1

This Bot has a fallback chain for grabbing webpage content: first try singlefile (full rendering with headless Chromium), if that fails switch to httpx, and if that fails switch to cloudscraper (a tool specifically for bypassing Cloudflare). It only reports an error to the user if all three fail. It's understandable for httpx and cloudscraper to get a 403; acm.org has protection. But singlefile shouldn't fail; it's designed as the last line of defense to bypass such protections.

Scrolling down, I found the actual error for singlefile:

Error: WebSocket is not available, Node.js 22.4.0 or later is required
    at file:///usr/local/lib/node_modules/single-file-cli/lib/deno-polyfill.js:132:8
Node.js v18.20.4

The container was running Node 18, but single-file-cli required 22.4 or later. Checking the logs, this error first appeared three days ago, not today. This means for the past three days, if any of the first two methods were blocked for a URL, the whole process would fail. Users would only receive a message saying "Unable to read content from the URL," with no indication of the underlying cause.

Why Run Cloud Build Three Times Instead of Deploying Immediately After Changes

The rest of this article will feature three cycles of "modify Dockerfile → build with Cloud Build → run and see." Initially, I didn't plan on doing this; I thought changing a version number and seeing the build pass would be enough to deploy. I later found this assumption was wrong every single time.

The reason is that for commands like npm install -g single-file-cli, a successful build only means "npm found the package and could install it," not "the package works when it runs." npm's check for the engines field is just a warning by default and won't fail the build. Whether the package actually crashes won't be known until it tries to launch a headless Chromium and establish a WebSocket connection. There's a gap in between that the build log doesn't show at all.

So this time, I didn't stop at "build passed." Every time I changed a version, I ran a shell script directly in Cloud Build to use the installed single-file in the container to actually fetch the acm.org URL that was failing. This habit later saved me twice: if I had deployed just because the build succeeded, the Dockerfile changes would have eventually re-enacted the exact same crash in the production environment, just with a different error message.

System Architecture

After the fix, the URL retrieval failure handling looks like this:

graph TD
    A[User sends URL] --> B[loader/url.py fallback chain]
    B --> C1[singlefile: headless Chromium]
    C1 -->|Content fetched| D{is_challenge_page check}
    D -->|Is Cloudflare challenge page| E[Treat as failure, move to next method]
    D -->|Is real content| F[Return to Gemini]
    C1 -->|Crash or timeout| E
    E --> C2[httpx]
    C2 -->|403 or challenge page| C3[cloudscraper]
    C2 -->|Real content| F
    C3 -->|403 or challenge page| G[All three failed]
    C3 -->|Real content| F
    G --> H[Throw Chinese error message to user]

Originally, only the "crash or timeout" path was considered a failure; the is_challenge_page branch was newly added this time, and I'll explain why later.

Core Implementation

1. Dockerfile: Locking Versions for Both Node.js and single-file-cli

Before the fix, this RUN command looked like this:

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        nodejs \
        npm \
        git \
        chromium \
        ffmpeg \
    && npm install -g single-file-cli \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

Both problems were in these lines. apt-get install nodejs pulls the version from the built-in Debian apt source, which is stuck on v18. npm install -g single-file-cli had no version lock, so every image rebuild would pull the latest version currently on npm. Separately, these aren't severe, but together they are a ticking time bomb: I have no control over when single-file-cli increases its Node.js requirement, while my container version was fixed.

After the fix:

# Lock both Node.js and single-file-cli versions to avoid silently pulling incompatible new versions during future rebuilds:
# - Node.js: Switched to NodeSource 24.x (Debian's built-in v18 is too old), locking the exact version.
# The ws/simple-cdp dependency of single-file-cli requires a global CloseEvent, which is only available in Node 24 
# (not in Node 22, even with --experimental-websocket, as verified by testing). This was the root cause of the acm.org scraping failure.
# - single-file-cli: Locked at 2.0.83, verified to work correctly under Node 24.
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        ca-certificates \
        curl \
        gnupg \
        git \
        chromium \
        ffmpeg \
    && mkdir -p /etc/apt/keyrings \
    && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \
    && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_24.x nodistro main" > /etc/apt/sources.list.d/nodesource.list \
    && apt-get update && apt-get install -y --no-install-recommends nodejs=24.19.0-1nodesource1 \
    && npm install -g single-file-cli@2.0.83 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

The exact version numbers nodejs=24.19.0-1nodesource1 and single-file-cli@2.0.83 weren't chosen at random; they were determined through three rounds of Cloud Build testing. I'll explain how I arrived at these numbers in the "Major Pitfalls" section below.

2. Detecting Cloudflare Challenge Pages

After singlefile stopped crashing, I thought it was over until I actually used it to fetch acm.org:

== single-file fetch acm.org (headless chromium) ==
OK: 18400 bytes
<title>Attention Required! | Cloudflare</title>

It "succeeded," but what it fetched was the Cloudflare human verification page, not the article itself. The problem was that this feature had no detection for this situation. While httpx and cloudscraper would call raise_for_status() on 4xx/5xx and fail immediately, the page rendered by headless Chromium is always an HTTP 200 (the browser displays whatever it gets). This layer of check didn't exist for singlefile. The verification page text would be sent to Gemini, and the user would receive a nonsensical summary completely unrelated to the original article, appearing as a normal response without any error prompts.

I added the detection logic in loader/html.py:

# Common characteristic strings for Cloudflare/WAF human verification pages. 
# singlefile uses headless Chromium to render pages, and unlike httpx/cloudscraper, 
# it doesn't call raise_for_status() for 4xx/5xx. It treats fetching a challenge page 
# as a "successful fetch"—this specifically intercepts that case. If detected, it must 
# be treated as a failure so the fallback chain in loader/url.py moves to the next method, 
# preventing challenge page content from being sent to Gemini.
CHALLENGE_PAGE_MARKERS = (
    "Attention Required! | Cloudflare",
    "Just a moment...",
    "Checking your browser before accessing",
    "cf-browser-verification",
    "Enable JavaScript and cookies to continue",
    "DDoS protection by Cloudflare",
)

def is_challenge_page(text: str) -> bool:
    """Check if the fetched content is a Cloudflare/WAF challenge page rather than real content."""
    return any(marker in text for marker in CHALLENGE_PAGE_MARKERS)

All three loaders (httpx, cloudscraper, singlefile) now use this check. If a challenge page is caught, it's treated as a failure, allowing the fallback chain to try the next method:

if is_challenge_page(resp.text):
    raise RuntimeError(f"httpx got a bot-challenge page instead of real content: {url}")
return parse_html(resp.text, markdown=markdown)

Major Pitfalls and Solutions

Pitfall 1: Don't Take Version Numbers in Error Messages at Face Value

Initially, seeing Node.js 22.4.0 or later is required, I naturally upgraded the container to Node 22, and the build passed smoothly. I thought I was done until I actually ran single-file to fetch acm.org and got a completely different error:

ReferenceError: CloseEvent is not defined
    at #onClose (.../single-file-cli/node_modules/simple-cdp/mod.js:334:36)
Node.js v22.23.2

The ws package that single-file-cli depends on tries to new CloseEvent(...) when handling connection close events, which requires a global CloseEvent class in the execution environment. I ran a probe directly in the container:

node -e "console.log('WebSocket', typeof WebSocket, 'CloseEvent', typeof CloseEvent)"
# WebSocket function CloseEvent undefined

node --experimental-websocket -e "console.log('CloseEvent', typeof CloseEvent)"
# CloseEvent undefined

Node 22.23.2 has a global WebSocket (which is what the "22.4.0 or later" error message was actually checking), but it doesn't have CloseEvent, even with the experimental flag. Testing again with Node 24.19.0:

node -e "console.log('CloseEvent', typeof CloseEvent)"
# CloseEvent function

Cause and Solution: The internal version check in single-file-cli (the "Node 22.4 or later" in deno-polyfill.js) only checks the first API it uses; it doesn't mean all APIs used by its dependencies are present in that version. The real blocker was the downstream CloseEvent, which only became globally available in Node 24. Treat version numbers in error messages as a starting point, not the final answer. You only know where it will actually get stuck by running it.

Pitfall 2: The "Latest Version" on npm is a Moving Target

After upgrading to Node 22, I thought it was over until a user asked to "lock the versions while you're at it to avoid future issues." Checking the current single-file-cli info on npm:

npm view single-file-cli engines
# { deno: '>=2.2', bun: '>=1.2', node: '>=24.0.0' }

The latest version requires Node 24 or higher, not the 22 I had just fixed. Checking the release times, I realized there were two consecutive version jumps within the same week:

npm view single-file-cli time --json | grep -E "2\.0\.83|2\.1\.0"
# "2.0.83": "2025-11-29T23:34:16.671Z"
# "2.1.0": "2026-08-16T15:56:28.819Z"

Version 2.0.83 was from last November, with engines set to node >= 20, and it worked stably for over six months. Version 2.1.0, released on August 16th, suddenly raised the bar to node >= 24. I first observed the crash on August 20th, which aligned perfectly with an image rebuild: npm install -g single-file-cli without a specified version had silently pulled the 2.1.0 version released just days prior.

Cause and Solution: npm install -g <package> (without a version) in CI/CD or a Dockerfile essentially means "every rebuild might install something different." This becomes a problem the moment a package maintainer raises the Node version requirement. Locking versions isn't about being conservative; it's about changing "when this container breaks" from "whenever the package author feels like it" to "whenever I decide to upgrade." I eventually chose 2.0.83—not the latest, but the last stable version explicitly compatible with Node 20—rather than trying to meet the new Node 24 requirement of 2.1.3.

Pitfall 3: Build Success Does Not Equal Runtime Success

I mentioned this earlier in "Why Run Cloud Build Three Times," but here is how it actually looked. After locking Node 24 + single-file-cli 2.0.83, gcloud builds submit finished successfully, and Successfully built was printed. If I had only looked at the build success message, I would have mistakenly thought the problem was solved. But the version that first passed the build was actually the Node 22 version, which would have exploded at runtime due to the CloseEvent issue in Pitfall 1. Build success itself does not prove the content of the container actually works.

I added an extra step to the Cloud Build configuration to run the newly built image immediately:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'nodefix-verify', '.']
- name: 'nodefix-verify'
  entrypoint: 'bash'
  args:
    - '-c'
    - |
      node -v
      single-file --version
      single-file --browser-executable-path=/usr/bin/chromium \
        --browser-args='["--no-sandbox","--disable-dev-shm-usage"]' \
        "https://www.acm.org/articles/people-of-acm/2026/russ-cox" /tmp/out.html \
        && echo "OK: $(wc -c < /tmp/out.html) bytes"

The first time I ran this, it caught the CloseEvent is not defined error from Pitfall 1. Only on the second run (Node 24 + 2.0.83) did I finally see OK: 18400 bytes.

Cause and Solution: npm install only prints a warning for engines mismatches by default; it doesn't fail the command. docker build success only means the exit code of every command was 0. Together, this means there is no necessary correlation between "build passed" and "this thing works correctly." Including "actually run the target scenario" in the verification process is the only way to bridge this gap.

Pitfall 4: Fetching Content Doesn't Mean Fetching the Right Content

When the verification in Pitfall 3 printed OK: 18400 bytes, I initially thought the problem was solved until I saw that within those 18,400 bytes, the <title> said Attention Required! | Cloudflare. I realized that even if single-file-cli works perfectly and renders the page with headless Chromium, Cloudflare might still block the request because it detects a headless browser, returning a challenge page. To single-file-cli, this challenge page looks exactly like a normal article: it's HTML, it can be saved, and it counts as a "success."

After adding the is_challenge_page check, the first round of tests didn't all pass:

FAILED test_load_html_with_httpx_raises_on_challenge_page
FAILED test_load_html_with_cloudscraper_raises_on_challenge_page

| The logic seemed correct, yet the httpx/cloudscraper tests failed to catch the challenge page. Upon investigation, I found that load_html_with_httpx in loader/html.py converts HTML to markdown (using markdownify) before returning it. markdownify does not preserve tags like <title> which are in the <head> and not displayed as body text. My detection string "Attention Required! | Cloudflare" happened to be in that discarded <title>, so it wasn't in the text after the markdown conversion. Even worse was cf-browser-verification, which in a real Cloudflare page is a CSS ID (<div id="cf-browser-verification">). It only exists in HTML attributes, so neither markdown conversion nor plain text extraction (get_text()) would include the attribute value as text content. |

Cause and Solution: Move the checkpoint from "processed text" to "raw HTML":

# Some challenge page features (like cf-browser-verification) only appear in HTML attributes, 
# which markdownify/get_text won't catch. So, check the raw HTML before conversion.
if is_challenge_page(resp.text):
    raise RuntimeError(f"httpx got a bot-challenge page instead of real content: {url}")
return parse_html(resp.text, markdown=markdown)

I made the same adjustment for singlefile. Originally, it used BeautifulSoup to convert the file to plain text before checking. I changed it to read the raw bytes and check them first; only if it passes is it handed over to BeautifulSoup:

with open(f, "rb") as fp:
    raw_html = fp.read()

if is_challenge_page(raw_html.decode("utf-8", errors="ignore")):
    raise RuntimeError(
        f"SingleFile got a bot-challenge page instead of real content: {url}")

soup = BeautifulSoup(raw_html, "html.parser")
text = soup.get_text(strip=True)

The lesson here is: when writing protection logic, you must be clear about which layer of data you are checking. The same content, after different conversions (HTML → markdown, HTML → plain text), will lose information asymmetrically: some words are visible, while others only exist in attributes, and both extraction methods might miss them.

Results and Benefits

After the changes, I ran a real end-to-end test, calling the load_url() function actually used in Python within the container against the same acm.org URL:

INFO:loader.url:Trying singlefile for URL: ...
ERROR:loader.singlefile:SingleFile loading failed ...: SingleFile got a bot-challenge page instead of real content: ...
WARNING:loader.url:singlefile failed ...
INFO:loader.url:Trying httpx for URL: ...
WARNING:loader.url:httpx failed ...: 403 Forbidden
INFO:loader.url:Trying cloudscraper for URL: ...
WARNING:loader.url:cloudscraper failed ...: 403 Forbidden
ERROR:loader.url:All methods failed for URL: ...
Raised as expected: Exception 無法從網址讀取內容，請確認網址是否正確或稍後再試 (Unable to read content from the URL, please verify the URL or try again later)

All three methods still failed; the Cloudflare protection behind acm.org was not bypassed. That hasn't changed. What changed is how it failed: previously, the singlefile crash dragged down the entire fallback chain, leaving the user unaware that three methods had even been attempted. Now, all three methods fail cleanly on their own, and the user receives a clear error message instead of a nonsensical summary that is actually a Cloudflare challenge page.

Locking versions solves "will it break again in the future," not "can the acm.org URL be fetched." It's easy to conflate these two at first: once versions are locked and single-file-cli no longer crashes, it's tempting to think the problem is solved. But the Cloudflare hurdle was never the scope of this fix, nor is there a simple solution (bypassing it would likely require browser fingerprint spoofing, which has a poor ROI). What was achieved was the complete separation of "crawler blocked" and "my own container is broken." The latter should no longer recur due to npm version drift.

"Build success" cannot be treated as verification passed. This was the biggest takeaway. If I had only looked for red text in the build log after every Dockerfile change, I would have missed at least two issues: the CloseEvent crash with Node 22, and the fact that Cloudflare challenge pages were being sent as body text even when singlefile worked perfectly. Both are "build okay, crash at runtime" types of problems that only become visible when running against a target URL.

Protection logic must align with the actual data layer received. The issue in Pitfall 4 where markdownify swallowed the <title> would have likely gone unnoticed until a user reported the exact same problem again, had I not written a test and seen it DID NOT RAISE.

In total, four files were modified: Dockerfile, loader/html.py, loader/singlefile.py, plus a new test tests/test_challenge_page_detection.py. All 150 tests in the project passed. The code is available at kkdai/linebot-helper-python.

[Gemini API in Action] Adding a "Detailed Research Report" Button to a LINE Bot: Using Google Search Grounding to Turn Summaries

Evan Lin — Thu, 20 Aug 2026 14:00:07 +0000

Background

My LINE Bot has always had a summary feature: you drop a URL in, it crawls the content, generates a summary, and attaches a social media post draft along with a button to save it as a bookmark. This feature has been around since 2024, but it has always only solved the "what is this about" problem. I often find myself wanting to know three other things:

What is the background context of the things discussed in this article? Have there been counter-arguments from others? Are the numbers mentioned sourced, or are they just the author's own claims?

Summaries can't answer these because the input for a summary is only the article itself. The model has no other materials; if you ask it for a "critical analysis," it can only circle around the original text or start hallucinating.

Google Search Grounding fills exactly this gap. I used it as a search assistant in a previous article; back then, the purpose was to answer questions. This time, I wanted to try another approach: give an existing article to the model, let it search for information outside the article on its own, and then look back to review the article.

The result is a new "📄 Detailed Research Report" button on the summary card. About one to two minutes after clicking it, the Bot pushes a web link to you.

Main Repo: https://github.com/kkdai/linebot-helper-python

Why Grounding instead of building your own search pipeline

Before Grounding, to let a model read real-time information from the web, you had to build a pipeline yourself: first, ask the model to extract keywords from the article, use those keywords to call a search API, crawl the search result pages one by one, stuff them into the prompt, and then ask the model to summarize. This involves three or more API calls, any of which could fail, and the quality of the extracted keywords directly determines whether the retrieved information is useful.

Grounding integrates this entire process into the model. You simply attach a google_search tool in the GenerateContentConfig, and the model handles the rest: it decides whether to search, what to search for, how many times to search, and judges which results are worth using.

For the "Research Report" topic, the model deciding what to search for is particularly valuable. When writing the prompt, I don't know what article the user will provide, so I naturally can't write the specific keywords to search. But after the model reads the article, it knows; it will look for the context of the topic and check if there are opposing views.

Another advantage I care about is that citations are returned. The grounding_metadata in the model's response contains the actual web pages it referenced, including titles and URLs. This means phrases like "according to other reports" in the report aren't just the model speaking from memory; there are corresponding web pages you can click to verify. For information-based products, this makes a huge difference.

The code to extract sources is in loader/langtools.py, written defensively because these fields don't exist at all if no search was triggered:

def _extract_grounding_sources(response) -> list:
    """Extract citations from grounding metadata (same approach as chat_session)."""
    sources = []
    try:
        if getattr(response, 'candidates', None):
            candidate = response.candidates[0]
            metadata = getattr(candidate, 'grounding_metadata', None)
            chunks = getattr(metadata, 'grounding_chunks', None) if metadata else None
            for chunk in chunks or []:
                web = getattr(chunk, 'web', None)
                if web:
                    sources.append({
                        'title': getattr(web, 'title', '') or '',
                        'uri': getattr(web, 'uri', '') or '',
                    })
    except Exception as e:
        logging.warning(f"Failed to extract grounding sources: {e}")
    return sources

System Architecture

The entire flow starts from the button on the summary card, goes through a re-crawl and a grounding call, and ends with a temporary webpage.

graph TD
    A[User sends URL] -->|Summary Flex Bubble| B[📄 Detailed Research Report Button]
    B -->|Postback with bookmark doc id| C[Verify bookmark ownership]
    C -->|Immediate Reply: Researching| D[LINE Chatroom]
    C -->|Background Task| E[load_url: Re-crawl original text]
    E --> F[Gemini + Google Search Grounding]
    F -->|Markdown + Citations| G[render_report_page to HTML]
    G -->|Store in memory ReportStore| H[Get uuid report_id]
    H -->|Push Link| I[GET /reports/:id Temporary Webpage]

The button carries the bookmark's document ID, not the URL itself. This follows the existing "Save Bookmark" mechanism. The benefit is that using the doc ID allows verifying that the bookmark actually belongs to the user before generating the report. Conversely, if Firestore isn't connected or the doc ID can't be retrieved, this button won't appear.

Core Implementation

1. Generating Research Reports with Grounding

The key to generate_research_report() isn't the code, but the prompt. I explicitly ask the model to search proactively and require it to label which information comes from the search and which comes from the original text:

    prompt = f"""You are a rigorous research analyst. Please write a detailed research report based on the following article content,
in Traditional Chinese (Taiwan usage), Markdown format (starting from ## level, do not include the main article title).

Required Structure:
## Executive Summary (3-5 sentences explaining what this is about and why it matters)
## Background Context (The history and context of this topic, combined with relevant information you searched for)
## Core Arguments & Evidence (Organize the article's claims and supporting evidence point by point, labeling the strength of evidence)
## Data & Fact Summary (Key numbers, dates, people, and organizations from the text, using tables or lists)
## Counter-perspectives & Critique (Search for related reports, compare other viewpoints; point out blind spots, assumptions, or controversies in the article)
## Further Questions (3-5 questions worth investigating further)

Requirements:
- Please proactively search for supplementary background and comparative information outside the article, and label in the text whether the information comes from search or the original text.
- Be specific rather than abstract; clearly label unsupported inferences as "speculation".
- Use full-width punctuation, avoid AI-sounding clichés.

Original URL: {url}

Article Content:
{text}"""

The phrases "label the strength of evidence" and "clearly label unsupported inferences as speculation" are the parts of the prompt I care about most. Without them, every sentence in the report would sound equally confident, and the reader wouldn't be able to distinguish what the article said, what the model added from search results, and what it inferred itself.

The part for attaching the tool is very short; whether tools is provided or not is the difference between having grounding or not:

    def _call(with_grounding: bool):
        client = _get_vertex_client()
        tools = [types.Tool(google_search=types.GoogleSearch())] if with_grounding else None
        return client.models.generate_content(
            model="gemini-3.1-flash-lite",
            contents=prompt,
            config=types.GenerateContentConfig(
                temperature=0.4,
                tools=tools,
                max_output_tokens=16384,
                labels={"client_id": "info_helper"},
            )
        )

    try:
        try:
            response = _call(with_grounding=True)
        except Exception as e:
            logging.warning(
                f"Grounded research call failed, retrying without tools: {e}")
            response = _call(with_grounding=False)

I used two layers of try because grounding involves external searches, so the failure rate is naturally higher than pure text generation. When the tool call fails, instead of returning "Generation failed," it's better to retry once with the same prompt but without the tool. In this case, the user gets a pure article analysis without comparative views or sources, but at least they have something. This degradation is intentional, not an accident.

2. Where to put the report

The report is a full Markdown document, often thousands of words long, which can't fit into a LINE message. Making it a Flex Message isn't suitable either because it contains tables and multi-level headings. So, I turned it into a webpage.

But then I had to decide: should these reports be stored in a database?

I chose not to. The reports are only stored in memory and disappear as soon as the Cloud Run instance is recycled:

class ReportStore:
    def __init__ (self, ttl_seconds: float = DEFAULT_REPORT_TTL_SECONDS):
        self.ttl = ttl_seconds
        self._reports: Dict[str, dict] = {}
        self._lock = Lock()

    def put(self, html: str) -> str:
        report_id = uuid.uuid4().hex
        with self._lock:
            self._purge_expired()
            self._reports[report_id] = {
                "html": html,
                "created_at": time.time(),
            }
        return report_id

The report_id uses uuid.uuid4().hex because this URL has no login protection; anyone with the link can open it, so the ID must be unguessable. The page itself also includes <meta name="robots" content="noindex"> to prevent search engines from indexing people's reading history.

3. Expired Pages

Since reports disappear, "link expiration" is not an exception but the normal end for every report. So the route is written like this:

@app.get("/reports/{report_id}")
def serve_research_report(report_id: str):
    """Temporary research report page: returns expired page (404) after expiration or instance restart."""
    html = report_store.get(report_id)
    if html:
        return HTMLResponse(html)
    return HTMLResponse(render_expired_page(), status_code=404)

I also made it clear in the message pushed to the user, not pretending it's a permanent link:

⏳ This is a temporary page, kept for about 24 hours (invalidated after the service sleeps). Please copy the content if you need to save it.

Major Pitfalls and Solutions

Pitfall 1: Grounding tools and response_schema cannot be used together

I encountered this pitfall earlier when building a map restaurant search. At that time, I naturally thought: since I want to get a restaurant list from the model, I'll use structured output, attach response_mime_type="application/json" and response_schema, and get correctly typed data directly to save myself from parsing it.

The result was an immediate API error. The fix then was to remove the schema (commit a2c8745) and instead ask the model to output JSON text, which I then parsed myself.

Reason and Solution

Google Search Grounding and structured output are mutually exclusive. When the model is performing grounding, it needs to freely intersperse searching, thinking, and citing; this process cannot be simultaneously constrained to a fixed JSON schema.

So when making the research report, I abandoned the idea of "returning a structured object" from the start and let the model output Markdown plain text directly, writing the structure into the "Required Structure" section of the prompt instead of the schema.

Looking back, this limitation actually made things simpler. The report is meant to be a long-form text for humans to read; Markdown is its most natural form. If forced into JSON fields, it would just have to be stitched back into an article during rendering anyway. The only fields that truly need a strict structure are the citations, and those can be taken from grounding_metadata, which never needed a schema to begin with.

Pitfall 2: Three-second timeout, plus Gemini is synchronous and blocking

LINE Webhook requires an HTTP 200 response within three seconds, but this feature needs to re-crawl the original text and then wait for the grounding call to finish, which takes one to two minutes in total.

Let's talk about the part fewer people notice. client.models.generate_content() is a synchronous blocking call. Even if you wrap it in an async def, it will still block the entire event loop. While one user is generating a report for those two minutes, messages from other users will also be blocked.

Reason and Solution

Split it into two parts: reply and push, each with its own responsibility:

    url = doc.get("url", "")
    await line_bot_api.reply_message(
        event.reply_token,
        [TextSendMessage(text="🔬 Starting in-depth research on this article (approx. 1-2 mins). I'll send you the report link once finished.")])

    try:
        crawled_text = await load_url(url)
        # Gemini call is synchronous and blocking; offload to a thread to avoid blocking other tasks on the event loop
        result = await asyncio.to_thread(generate_research_report, crawled_text, url)

First, use reply_message to say "Starting research," finishing the webhook request within three seconds. The heavy lifting is moved to the background, and the synchronous Gemini call is offloaded to a thread using asyncio.to_thread. Once finished, use push_message to proactively send the link.

The phrase "approx. 1-2 mins" is also intentional. If a user clicks a button and nothing happens for thirty seconds, they'll start to suspect it's broken and click it again. Explaining how long to wait upfront is cheaper than explaining it afterward.

Pitfall 3: Who can see whose report

The button carries the bookmark's doc ID. If I simply used this ID to query data and generate a report, anyone who could construct a postback could read the content of bookmarks saved by others.

Reason and Solution

Verify ownership during the query. Both parameters for get_bookmark(user_id, doc_id) are required. If not found, treat it as expired without telling the user "this exists but doesn't belong to you":

    doc = svc.get_bookmark(user_id, doc_id) if (doc_id and svc.available) else None
    if not doc:
        await line_bot_api.reply_message(
            event.reply_token,
            [TextSendMessage(text="⚠️ Data has expired. Please send the URL again and try once more.")])
        return

Results and Benefits

In practice, this button changes more than just "making the summary longer."

Summaries and research reports answer different questions. A summary tells you what the piece is about, suitable for quickly deciding whether to read it. A research report tells you if the piece is correct, how others view it, and which numbers are sourced. That's why I made it two layers instead of making the summary longer: the cheap layer runs every time, and the expensive layer is there for when you really want to dive deep.

Citations make the report verifiable. At the bottom of the report is a "📚 References" list, all from grounding_metadata, which are the actual web pages the model read. If you see something in the "Counter-perspectives & Critique" section that differs from the original text, you can click directly to the original report. This is why I think grounding is more worthwhile than building your own search API pipeline: what you save isn't just code, but the traceability of "where did this sentence come from."

There's still something to see even if search fails. After degrading to pure article analysis, the report will lack background context and comparative views, but the executive summary, core arguments, and data organization sections will still be there.

Temporary webpages save more than expected. No need to set up a database, no need to write cleanup schedules, and no need to design a report list page. A dict plus a lock is all it takes, with security maintained by unguessable UUIDs. The trade-off is that links will expire, which I've made clear in the push message. Reading behavior is usually concentrated in the few minutes after receiving a link; I don't think it's worth the overhead of a full persistence system for the few cases where someone wants to save it long-term.

If long-term storage is really needed later, the current architecture isn't hard to change: the ReportStore interface only has put and get methods. Replacing it with a Firestore implementation wouldn't require any changes to the upper layers.

The entire feature adds up to about five hundred lines, a third of which are tests. The code is at kkdai/linebot-helper-python, and the design document is in docs/superpowers/specs/2026-08-15-research-report-design.md. Feel free to check it out if you're interested.

[Claude Code in Practice] Rethinking Terminal Workflows: From zsh Autocomplete to Search and Diff Toolchains

Evan Lin — Fri, 14 Aug 2026 16:56:08 +0000

Pain Point: Starting Over with Every Command

When using Claude Code for tasks, there have always been two minor frictions that I hadn't seriously addressed until now.

The first is that the shell itself is too bare-bones: there's nothing in ~/.zshrc except for two lines of PATH. No auto-completion, no command history. Navigating history requires pressing the up arrow repeatedly, and even then, similar commands must be manually edited. The second is when collaborating with Claude Code, some commands that are clearly read-only and have no side effects (listing files, checking versions, curling a README) require a manual "allow" click every time. This back-and-forth breaks the flow.

This post records the process of tackling both issues at once: how tools were chosen, what pitfalls were encountered, and how it was eventually integrated with Claude Code's permission system.

Solution 1: Let Zsh Remember What You've Typed

I haven't installed oh-my-zsh and didn't want to carry a whole framework for just two features, so I picked the two smallest, sufficient packages:

zsh-autosuggestions: Suggests similar commands in gray text while typing; press Ctrl+Space or → to accept.
zsh-completions: Enhances the coverage of tab auto-completion.

brew install zsh-autosuggestions zsh-completions

Combined with history settings, the up/down keys can filter history based on current input instead of just scrolling through everything:

# --- Command History Settings ---
HISTFILE=~/.zsh_history
HISTSIZE=10000
SAVEHIST=10000
setopt SHARE_HISTORY # Share history across multiple terminal windows
setopt HIST_IGNORE_DUPS
setopt HIST_IGNORE_ALL_DUPS
setopt HIST_FIND_NO_DUPS
setopt INC_APPEND_HISTORY # Write to history file immediately upon command entry

autoload -Uz up-line-or-beginning-search down-line-or-beginning-search
zle -N up-line-or-beginning-search
zle -N down-line-or-beginning-search
bindkey "^[[A" up-line-or-beginning-search
bindkey "^[[B" down-line-or-beginning-search

After installing the packages and adding the settings, theoretically, restarting the terminal should work—but it wasn't that smooth.

Solution 2: Add Color to Read the Terminal Faster

Once auto-completion was set up, I added colors as well:

zsh-syntax-highlighting: Real-time coloring while typing; green for valid commands, red for invalid ones.
ls -G: Different colors for folders, executables, and links.
grep --color=auto: Highlights matched keywords in red.

export CLICOLOR=1
export LSCOLORS=GxFxCxDxBxegedabagaced
alias grep='grep --color=auto'

source /opt/homebrew/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

Handled fonts too: Ghostty's config file (~/Library/Application Support/com.mitchellh.ghostty/config.ghostty) didn't specify a font size and defaulted to the system's 13. Adding font-size = 16 solved it.

Solution 3: Replace `ls`, `cat`, and `cd` with Smarter Versions

Color and auto-completion are infrastructure; next, I added modern alternatives for three common commands. The selection criterion was simple: single executable, no background daemon, and no impact on startup speed.

Command	Alternative	Benefit
`ls`	`eza --icons`	Color, icons, tree structure (`lt`)
`cat`	`bat --paging=never`	Syntax highlighting, line numbers
`cd` (helper)	`zoxide`	Remembers frequent folders; `z proj` jumps directly

I evaluated fzf but didn't install it this time—my current habits don't require fuzzy searching history/files yet. I'll add it if I feel a bottleneck later.

Solution 4: Make File Searching and Comparison Faster for Claude Code

The previous items were for "user experience"; this one is for "Claude Code speed." Claude Code's built-in search tool already uses ripgrep (rg), and jq is already installed, so these three were missing:

fd: Replaces find; simple syntax, fast, especially noticeable when listing files.
ast-grep: Structural code search. Not just text matching, but looking at the Abstract Syntax Tree (AST). It can search for "all calls to a function where the first argument is a string," which is much more accurate than regex for large-scale refactoring or precise searches.
difftastic (difft): Syntax-aware diff. It recognizes when a function has been moved rather than deleted and rewritten.

brew install fd ast-grep difftastic git-delta

git-delta is mainly for human-readable git diff. It's unrelated to Claude Code, but I installed it anyway.

Solution 5: Add Common Read-Only Commands to Claude Code's Permission Whitelist

A new problem emerged after installing the tools: the first time Claude Code calls these new commands, it still asks for permission. Using the fewer-permission-prompts skill to scan recent session transcripts, I identified frequently run, truly read-only commands and compiled a whitelist:

{
  "permissions": {
    "allow": [
      "Bash(curl -s https://raw.githubusercontent.com/*)",
      "Bash(curl -s \"https://api.github.com/*)",
      "Bash(brew list*)",
      "Bash(xcodes list*)",
      "Bash(xcodebuild -version)",
      "Bash(curl -sI *)",
      "Bash(difft *)",
      "Bash(delta *)"
    ]
  }
}

fd, rg, and jq are not on the list, not because they were missed, but because Claude Code already includes them as built-in, auto-allowed read-only commands.

Three Easily Overlooked Pitfalls

Pitfall 1: `compinit` complaining about "insecure directories"

After installing packages and restarting the terminal, the first launch showed:

zsh compinit: insecure directories, run compaudit for list.

compaudit revealed the issue was that the /opt/homebrew/share directory permissions were too open (group write access). compinit checks permissions before loading completion scripts; if any directory is "writable by others," it refuses to load to prevent malicious scripts from being injected into the completion path. The fix is the official Homebrew recommendation:

chmod go-w /opt/homebrew/share
chmod -R go-w /opt/homebrew/share/zsh
rm -f ~/.zcompdump*

Clearing the cache to let it rebuild solved the problem.

Pitfall 2: `zsh-syntax-highlighting` must be the last line

The official documentation is clear: this line must be the last thing executed in .zshrc. If placed before zsh-autosuggestions or other bindkey settings, syntax highlighting and auto-suggestions may interfere, and key bindings might fail. I specifically moved it to the very end of the file.

Pitfall 3: `ast-grep` was intentionally left out of the whitelist

ast-grep is missing from the whitelist on purpose. By default, it's a read-only search, but adding the -U / --update-all flag allows it to rewrite files. Permission rules use prefix matching, so I can't allow only the "no -U" usage. Opening a broad rule like Bash(ast-grep *) theoretically allows file-modifying usage as well.

This aligns with the original logic for sed: only "read-only expressions" are auto-allowed; any usage with in-place editing still prompts for permission. Rather than re-evaluating the risk, I followed the same standard.

Summary and Benefits

This terminal environment overhaul was essentially about optimizing "human typing" and "Claude Code execution" separately:

Less typing: zsh-autosuggestions + history filtering means almost no re-typing repetitive commands.
Faster reading: Syntax highlighting, eza, and bat make outputs instantly readable.
Faster navigation: zoxide replaces memorizing paths, and fd replaces the slow find.
More accurate Claude Code searches: ast-grep adds structural search that plain text matching can't do, and difftastic makes diff results closer to actual changes.
Fewer "allow" clicks: Whitelisting truly read-only, risk-controlled commands while still prompting for things that should be asked (like ast-grep -U).

The core principle for tool selection remained the same: single executable, no background daemons, and avoiding frameworks where possible. The real time-consumer was deciding "whether to whitelist this"—speed is secondary; the priority is ensuring a command that can modify files isn't accidentally wrapped in a seemingly safe, generic rule.

[Dev Log][Python] Create short videos from photos and clips with Gemini 3.7 Flash: ReelCraft

Evan Lin — Fri, 14 Aug 2026 16:55:48 +0000

Preface:

It all started with a misunderstanding.

I noticed a new page in the Gemini API documentation called Omni, introducing a model named Gemini Omni Flash, described as "natively multimodal, processing text, images, audio, and video simultaneously." My first thought was straightforward: if I throw a whole folder of videos and photos from my phone into it, let it understand what each asset is about, and then tell it in one sentence to edit them into a short video—isn't that a video editing app?

After reading the documentation, I realized I had misunderstood, and the misunderstanding happened to be at the most critical point. However, after bypassing that limitation, the rest was actually feasible. The result is ReelCraft: a Python CLI where you feed in a bunch of videos and photos, Gemini 3.7 Flash understands the assets one by one and provides editing suggestions. Once I confirm the edit list, ffmpeg cuts it into a 9:16 vertical short video, background music is generated using Lyria 3, and subtitles are automatically burned in.

Along the way, there were three issues where both ffmpeg and Gemini reported success, but the output was wrong—the kind of errors you only discover by actually playing the video.

TL;DR

This article will cover:

Omni Flash is not what I thought it was
Bypassing limitations: Per-file understanding, then text aggregation
Using edl.yaml as a human confirmation point
The difference after switching to Gemini 3.7 Flash
Background music: Lyria 3 uses a different API
ffmpeg will silently fail your edits
Subtitles: Two issues only visible after burning them in
Other pitfalls
Conclusion
Reference links

Omni Flash is not what I thought it was

Gemini Omni Flash (gemini-omni-flash-preview) is a video generation and editing model that uses the Interactions API. It allows you to use natural language to apply effects to a single video, such as "when the person touches the mirror, make the mirror ripple beautifully like liquid." It is not a tool for "understanding a bunch of videos."

The limitation section states clearly:

Referencing or reasoning across multiple videos is not supported. Attempting multi-video prompting may result in degraded model performance or unexpected outputs.

Additionally:

Video references up to 3 seconds in duration are accepted by the API schema but are not correctly processed by the model at this time.

So the path of "throwing a bunch of videos in and letting it understand and edit them" was blocked for Omni Flash. The models that can actually perform multi-video understanding are the standard Gemini models: starting from version 2.5, a single request can include up to 10 videos. With a 1M context window, it can handle about an hour of footage at default resolution, tokenize it second-by-second, and output scene descriptions with timestamps.

The time spent on this misunderstanding wasn't wasted. The verification process helped clarify "which task should be handled by which model," and the architecture followed naturally.

Bypassing limitations: Per-file understanding, then text aggregation

The entire pipeline is split into five stages, with states stored in files:

[Asset Folder]
     │ poc ingest: Scan videos/photos → catalog.json
     ▼
     │ poc analyze: Call Gemini for each file individually → analysis/*.json
     ▼
     │ poc plan: Aggregate all analysis results, call once for editing suggestions
     ▼ → summary.md (for humans) + edl.yaml (for machine execution)
     ⏸ Human inspection and editing of edl.yaml
     ▼
     │ poc render: ffmpeg editing, 9:16 cropping, xfade transitions
     ▼
output/final.mp4

The key design decision is in the second and third steps: call Gemini once for each video to get precise internal timestamps and descriptions; then feed these text results (not the raw videos) into a second call for cross-asset aggregation, sequencing, and editing suggestions.

This approach has two benefits. First, it completely avoids the "multi-video reasoning not supported" issue because the second call only sees text, not ten videos. Second, it isn't limited by the 10 videos/request cap; no matter how many assets there are, it just means more independent calls in the analyze phase. Those calls can be retried or fail individually without affecting each other.

Testing also proved that timestamps are more reliable when processed separately. When asking about ten videos in a single prompt ("which seconds are the highlights?"), the model easily confuses the timelines of different videos.

Failure handling in the analyze phase is recorded separately: if a file fails after three retries, it's logged in analysis/_errors.json, while other files continue. This later revealed a loophole during review, which I'll discuss later.

Using edl.yaml as a human confirmation point

I decided from the start not to make it "one-click fully automatic." Between inputting assets and outputting the final product, there must be a place where I can manually intervene, because LLM-provided edit points will inevitably have some irrationalities, and re-running the entire pipeline incurs API costs again.

That interface is a YAML file:

target_duration_sec: 23
aspect_ratio: '9:16'
clips:
- source: /abs/path/808327978.mp4
  note: Opening shot: Showing the COSCUP x UbuCon Asia main visual backdrop.
  in: '00:00.000'
  out: '00:02.500'
- source: /abs/path/S__1908753.jpg
  note: Fun venue easter egg: Creative semiconductor chip snacks distributed on-site.
  duration_sec: 4.0
transitions: crossfade 0.3s
mood_tags: [Professional, Joyful, Community Cohesion]

Videos use in/out to mark the range, photos use duration_sec for duration, and note is the reason for selection written by Gemini (this field was later used for subtitles, see below). To change an edit point, just change the numbers; to change the order, move the clip; after saving, run poc render.

The outputs of each stage remain in the project directory, so any step can be re-run individually. analyze also skips files that already have analysis results, so re-running doesn't incur double charges—this is very helpful when iterating on prompts.

poc plan --theme was added later: you can provide a sentence as the editing theme, e.g., --theme "Participating in the COSCUP open source community". This affects the narrative angle of the summary, the priority of clip selection, and the wording of each clip's note. Since it only affects the plan stage, changing the theme doesn't require re-analyzing assets, making it very cheap to try different narratives on the same set of materials.

The difference after switching to Gemini 3.7 Flash

The understanding and aggregation stages initially used gemini-2.5-flash, then switched to gemini-3.7-flash. This is the GA stable version, not a preview:

Item	Specification
Model ID	`gemini-3.7-flash`
Input	1,048,576 tokens
Output	65,536 tokens
Input Types	Text, Image, Video, Audio, PDF
Capabilities	structured outputs, function calling, caching, thinking (low/medium/high)
Not Supported	Video/Image/Audio generation, Live API

For this project, the most important features are structured outputs and video input, as the analyze stage involves feeding in a video and requesting a JSON with a fixed schema.

After switching, I didn't just change the string and call it a day; I verified it with actual API calls, running analyze_file on real assets. For the same lecture video, the difference in descriptions between the two models was quite noticeable.

gemini-2.5-flash version:

At the start of the video, a woman on stage uses a microphone to introduce herself to the audience. The large screen behind her shows her name "Zona Wang" and her job description.

gemini-3.7-flash version:

In the video, a female speaker (Zona Wang, LINE Technology Evangelist) is giving a self-introduction and presentation on a stage in a lecture hall, followed by a camera pan across the audience listening intently.

The difference lies in "job description" vs. "LINE Technology Evangelist." The latter actually read the small text on the slide, while the former only knew there was some job information there.

The gap in the aggregation stage was even larger. For the same set of COSCUP assets and the same --theme, 2.5's summary was: "This short video aims to showcase the vitality and diversity of the COSCUP open source community. From professional knowledge sharing and deep technical exchange to warm interaction and inclusion among community members"—the whole thing stayed at an abstract level. 3.7 recognized the full event name "COSCUP x UbuCon Asia," booth names like "FOSS for All" and "Kubernetes," and even described a photo as "Creative semiconductor chip snacks distributed on-site." These details weren't in my prompt; they all came from the text and objects in the photos.

For an application where "asset understanding quality directly determines editing quality," the benefit of switching models was greater than I expected. The editing suggestions improved because it actually understood more, not because the prompt was written better.

By the way, 3.7's note style also changed to a "Short Label: Detailed Description" format. This change later broke all my subtitles, as discussed below.

Background music: Lyria 3 uses a different API

Background music is generated using Lyria 3. There are two models: lyria-3-clip-preview for 30-second clips, and lyria-3-pro-preview for full songs. My output is about 20 seconds, so the clip version is perfect.

It doesn't require a separate Vertex AI application or allowlisting; the same Gemini API key works. However, the calling method is completely different from generate_content, using client.interactions.create():

interaction = client.interactions.create(
    model="lyria-3-clip-preview",
    input="An instrumental background music track for a short social-media video, "
          "about 20 seconds long. Mood: Professional, Joyful, Community Cohesion, Happy. "
          "No vocals, no lyrics, loopable.",
)
audio_bytes = base64.b64decode(interaction.output_audio.data)

Several things were different from what I imagined.

It has no structured parameters. Length, BPM, genre, and mood must all be written in the natural language prompt, rather than passing a field like bpm=120. So the generate_score(mood_tags, duration_sec) function's job is actually to concatenate mood tags and seconds into an English sentence. Mood tags are aggregated from asset analysis results during the plan stage, and poc render --mood "Happy, Joyful, Celebration" can further overlay desired directions.

It is single-turn generation and cannot be iteratively modified. Unlike Omni Flash's video editing, once the music is generated, it's set; if you're not satisfied, you have to submit a new prompt. All generated audio includes a SynthID watermark.

When the music is shorter than the video, you have to handle it yourself. The clip version is max 30 seconds, but the video might be longer. So during mixing, I use -stream_loop -1 to loop the audio infinitely and -shortest to trim it to the video length:

cmd.extend(["-stream_loop", "-1", "-i", str(audio_path)])
# ... filter_complex, map video ...
cmd.extend(["-map", f"{audio_index}:a", "-c:a", "aac", "-b:a", "128k", "-shortest"])

Music generation failure (quota, network, safety filters) won't crash the entire render; it prints a warning and falls back to silent output. This principle was later added to the project's CLAUDE.md: any value-added feature calling an external generative API must degrade gracefully and not let the main process die because of a secondary feature.

ffmpeg will silently fail your edits

The render stage uses ffmpeg's xfade filter to connect clips. Each xfade requires an offset parameter, which is "at which second in the output timeline to start this transition." The logic for accumulation is: the sum of all previous clip lengths minus the seconds overlapped by each transition.

After writing the first version, unit tests were all green, and real assets produced normal videos. Then review identified two scenarios where ffmpeg returns exit code 0, but the output file is wrong.

Scenario one: The transition is longer than the clip, causing the clip to be silently swallowed. For two 1-second clips with transitions: "crossfade 2s", the calculated offset is -1.000. ffmpeg accepts this negative number, doesn't report an error, and finishes normally. The output is a 1-second video containing only the first clip; the second one disappears entirely. Since EDL.transitions is a free-text field, it's entirely possible for me to type 3s instead of 0.3s when manually editing the YAML, and it won't tell me in any way.

Scenario two: out exceeds the actual asset length, causing everything following it to be truncated. For a 10-second video, if the EDL says in: 8.0 / out: 15.0, only 2 seconds can actually be taken. If a 1.5-second photo follows, the offset is calculated as 6.700, which falls after the end of the first stream. The result is a 2-second output where the photo is completely missing, and the exit code is still 0. This scenario is even more important to prevent because the EDL is generated by an LLM, and hallucinating an out-of-bounds end time is quite natural.

I added explicit checks for both: if a negative offset is calculated, a ValueError is thrown specifying which clip and transition length; before rendering, ffprobe is used to read the actual length of each video asset, and if out exceeds it, an error is reported clearly stating the requested vs. actual duration.

I care so much because a "successful" but incorrect output is much worse than a crash. If it crashes, I know to fix it immediately. With exit code 0 and a seemingly normal mp4, I might not notice until I watch the whole video and think "wait, a segment is missing," and then have no idea where to start investigating.

Subtitles: Two issues only visible after burning them in

The source for subtitles is the note for each clip in the EDL—the editing reason written by Gemini. Since it already wrote a description for each segment, using it as an on-screen title is perfect.

The implementation doesn't use drawtext; instead, it generates an SRT file and burns it in using libass's subtitles filter. The reason is that drawtext requires manual handling of Chinese font paths and escaping characters; colons, commas, and single quotes all clash with filtergraph syntax. SRT with force_style is much cleaner, and specifying FontName=Noto Sans TC lets fontconfig find the Chinese font.

The first issue was two subtitles appearing on screen simultaneously. In the first version, each subtitle's display interval was just the clip's own start and end times. But with a 0.3s crossfade overlap between adjacent clips, those 0.3 seconds would have two lines of white text on a black background stacked together, which looked ugly. The fix was to change each subtitle's end time to "when the next clip starts" rather than its own end time, ensuring at most one subtitle is visible at any moment. Unit tests couldn't catch this because the SRT was perfectly valid and ffmpeg burned it successfully; I only found it by looking at the frames.

The second issue was subtitles all trailing with an ellipsis. The note is a full sentence description, which would fill the screen if burned directly, so it's truncated into a short title: cut at the first comma or period, or use a character limit if no punctuation is found, adding "..." if truncated.

After switching to Gemini 3.7 Flash, this rule fell apart. 3.7 tends to write notes in a "Opening shot: Showing the 2024 COSCUP x UbuCon Asia main visual backdrop" format—a "Short Label: Detailed Description" style. Since colons weren't in my sentence-breaking character list, the whole sentence fell into the character-limit truncation path, and all eight subtitles ended with "...".

Hard truncation had a second flaw: it ignored word boundaries. "Presenting the female speaker sharing presentation content about ChatGPT and Antigravity" cut at the 20th character resulted in "...and An...", a halved English word.

I fixed both: colons are now treated as label separators, and the label itself is used as the full title without an ellipsis; when hard truncation is necessary, if the cut point falls in the middle of a continuous string of English letters/numbers, it backtracks to before that string started, discarding the whole segment rather than cutting it in half. I also relaxed the character limit from 20 to 24.

After re-burning, the eight subtitles became clean short titles like "Opening Shot," "Session Hall Live," "Technical Sharing Close-up," "Venue Easter Egg," and "Community Booth Interaction," without a single ellipsis.

Other pitfalls

files.upload() returning doesn't mean the file is ready. This was caught by digging into the SDK source code during review, and it would crash on the real API while never showing up in tests. client.files.upload() returns as soon as the bytes are transferred, without waiting for server-side processing. After a video is uploaded, it stays in a PROCESSING state for several seconds; trying to use it for generate_content during this time results in a 400 FAILED_PRECONDITION.

Worse, my original retry loop made things worse: analyze_file was wrapped in a retry, so each retry re-uploaded the entire video and immediately failed again, with only about 3 seconds of backoff across three tries. After three tries, the asset went into _errors.json, and the plan stage didn't read that file at the time, so the asset silently disappeared from the final product. The fix was adding a wait_for_active(), polling client.files.get() after upload until the state is ACTIVE before proceeding, and moving the upload out of the retry loop.

_errors.json was written but not read. As mentioned, analyze diligently recorded failed assets, but plan didn't read them, and summary.md wouldn't mention them. The only way a user could notice was by counting the segments in the final product. Now plan attaches the failure list to the end of the summary, explicitly stating which assets were not included.

Re-running ingest can bite you with old analysis results. This was encountered during actual use, not review. I changed the contents of the asset folder, adding new photos and deleting old ones, then re-ran poc ingest. catalog.json was updated, but analysis results for deleted files were still sitting in analysis/. When plan read the analysis results, it didn't cross-reference them with the current catalog, so it fed outdated assets to Gemini. The model reasonably picked a segment from them, but since the file no longer existed, the whole plan failed. Now load_analyses() filters by the catalog and prints which outdated records are ignored.

Timestamp precision. format_timestamp initially used :04.1f, keeping only one decimal place. Every time an EDL went in and out of YAML, it lost up to 0.05 seconds, which is about 1.5 frames at 30fps, causing edit points to drift. I changed it to :06.3f to keep millisecond precision.

Looking back, these problems fall into two categories. files.upload and ffmpeg silent errors were caught by reading the code line-by-line during review. Subtitle overlapping, ellipsis issues, and stale analysis results only surfaced by actually running the code, playing the videos, and trying different sets of assets. When the tests were all green, those three issues were still lurking in the code.

Conclusion

What ReelCraft does now is simple: a folder of videos and photos goes in, and a 9:16 short video with music and subtitles comes out, with a YAML file in the middle that I can manually edit.

Architecturally, what really makes this work is the "per-file understanding, text aggregation" split. It was conceived to bypass Omni Flash's lack of multi-video reasoning, but it ended up solving timestamp precision and asset count limits as well. After switching to Gemini 3.7 Flash, the granularity of asset understanding significantly increased, and the editing suggestions improved accordingly—the gains here were greater than what I got from tuning prompts.

Two areas remain untouched: Omni Flash's single-clip generative touch-up has an empty touch_up_clip interface, and subtitles are currently derived automatically from note, with the text_overlays field still empty. Neither music nor subtitles are cached; they are re-generated every time render is run.

Reference Links:

[Learning Notes][Golang] Authorization Challenges in the AI Agent Era: What is ID-JAG and Why I Re-implemented It in Go

Evan Lin — Tue, 28 Jul 2026 05:12:52 +0000

Preface:

In the past six months or so, connecting AI Agents directly to internal systems to help get things done is no longer news. However, if you think one step further: in "whose identity" should the Agent call those APIs? If the permissions it receives are as broad as a human's, once it is tricked into performing an operation it shouldn't, the consequences could be more severe than a human's accidental slip.

This is exactly the problem that ID-JAG (Identity Assertion JWT Authorization Grant) aims to solve. I recently organized the principles of this mechanism and re-implemented the MCP Server from the tutorial repo athenz-community/id-jag-the-hard-way using Go: kkdai/id-jag-mcp. This article aims to clarify the technical principles of ID-JAG: which RFC standards it is built upon, how it differs from OAuth2 / PKCE that I've written about before, what the actual token exchange flow looks like, and finally, a demonstration of how to run and test this Go project.

TL;DR

This article will introduce the following in order:

What is ID-JAG? Why is it needed?
From OAuth2 and PKCE to new problems in the Agent era
Two RFC cornerstones: Token Exchange and JWT Bearer
The complete ID-JAG token exchange flow
Downscoping permissions at every hop: How the Principle of Least Privilege is implemented
Why re-implement this MCP Server in Go?
Hands-on: Installation, Execution, and Testing
Conclusion
References

What is ID-JAG? Why is it needed?

ID-JAG is an authorization mechanism that allows AI Agents to access protected resources on behalf of users, with the keyword being "on behalf of." Currently, it is still an IETF Internet-Draft and has not yet become an official RFC, but organizations like LY Corporation (on the Athenz authorization system) and Okta have already begun implementation, and the MCP (Model Context Protocol) specification has already cited this draft.

Traditional service-to-service authorization usually falls into two extremes: either the entire service shares a universal master key (API Key, Service Account), or the user's session or long-lived token is directly lent to the program. The former has too much power, while the latter lacks an audit trail; if leaked, an attacker can almost completely impersonate the user, and it is very difficult to detect. When AI Agents start deciding which tools to call and which internal APIs to connect to on their own, the risks of both approaches are amplified: an Agent might perform operations the user never intended due to prompt injection or hallucinations. If the Agent holds a master key, the consequence is that all company data is at its mercy. In real-world scenarios, there is often a multi-layered architecture where an "Orchestrator Agent calls a Sub-Agent," and the risk is passed all the way down.

What ID-JAG wants to achieve is: for every action an Agent takes, it must be able to prove that "this is a specific user, at a specific moment, authorizing me to do this specific thing," and this authorization scope should be as narrow as possible, with a validity period as short as possible. It is built on top of OAuth2 token exchange (RFC 8693), adding a layer of proof that "this token is derived from a human's identity assertion." This is also why it is listed on OAuth.net's Cross-App Access (XAA) page—this is precisely a new problem emerging in the Agent era.

Incidentally, ID-JAG also solves a very practical user experience problem: if an Agent had to pop up a browser window for the user to manually click "Agree" every time it needed to access a new service, the experience would quickly become exhausting, leading users to just agree to everything. ID-JAG consolidates the authorization action to the moment the user logs in via SSO. After that, when the Agent needs new access permissions, it uses the already issued identity assertion to exchange for a token with the authorization server, without requiring the user to pop up and click again.

From OAuth2 and PKCE to new problems in the Agent era

I previously wrote about How to develop OAuth2 PKCE via Golang, which covered the implementation experience of LINE Login adopting PKCE. The problem solved in that article and the one ID-JAG solves are actually on two different levels. Comparing them makes it clearer what is new about ID-JAG.

PKCE solves the problem of "whether the client identity is trustworthy": for public clients like mobile apps that cannot safely store a client secret, the authorization code might be intercepted by a malicious app on the same phone during transmission. PKCE uses a code_verifier / code_challenge one-time pair to ensure that even if the code is stolen, it cannot be exchanged for a token without the correct verifier. The entire problem occurs in a "single hop" between the user and the app in their hand.

ID-JAG solves the problem of "whether this non-human service identity is qualified to act on behalf of this person for this task," and it often spans several hops: User logs into IdP → AI Client Gateway → MCP Server → Final Resource Server. The caller at each hop is not the user themselves, yet each must prove they are "acting under authority." The original design of OAuth 2.0 was for "human user ↔ application" scenarios and does not directly support this multi-layer Agent chain delegation scenario. PKCE protects the integrity of a single authorization exchange; ID-JAG protects the minimum necessary permissions for every link in an entire authorization chain. The two do not conflict; they are mechanisms solving problems at different stages under the same broad architecture.

Two RFC cornerstones: Token Exchange and JWT Bearer

ID-JAG did not invent a new protocol out of thin air; instead, it combines two existing IETF standards. Understanding these two cornerstones is necessary to understand what the subsequent complete exchange flow is doing.

The first is RFC 8693 — OAuth 2.0 Token Exchange, which defines a general protocol for "exchanging one type of token for another." Conceptually, it's like going to a currency exchange to swap Yen for Taiwan Dollars, except here you are swapping security tokens. A token exchange request looks roughly like this:

POST /token
grant_type=urn:ietf:params:oauth:grant-type:token-exchange
subject_token=<Identity Assertion JWT>
subject_token_type=urn:ietf:params:oauth:token-type:jwt
requested_token_type=urn:ietf:params:oauth:token-type:access_token
scope=read:orders

The subject_token contains the token representing the delegated identity, requested_token_type specifies what type of token you want to exchange it for, and scope can narrow down the permission range at the moment of exchange.

The second is RFC 7523 — JWT Bearer Grant, which allows a JWT itself to be used directly as an OAuth 2.0 authorization credential without having to go through an authorization code exchange first:

POST /token
grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer
assertion=<Signed Identity Assertion JWT>

After receiving it, the authorization server verifies the signature of this JWT using the issuer's (IdP) public key. If fields like audience, scope, and exp are reasonable, it can directly issue an Access Token without secondary user consent—because the IdP has already endorsed this identity assertion.

The "Identity Assertion JWT" to be exchanged usually contains the following fields:

{
  "iss": "https://enterprise.idp.example.com/v1",
  "sub": "alice@company.com",
  "aud": "https://api.service.example.com",
  "exp": 1773839486,
  "iat": 1773825086,
  "jti": "abc123-7dc6-42ab-b326-uniqueid",
  "scope": "read:orders write:tickets"
}

sub is the user being proxied, aud is which service this assertion is intended for, scope is the allowed permission range, and jti is a unique ID used to prevent the same assertion from being replayed. ID-JAG uses this short-lived identity assertion and, through the exchange mechanisms defined in the two RFCs above, gradually exchanges it for the Access Token that the Agent can actually use to call the API.

The complete ID-JAG token exchange flow

Using the architecture of the id-jag-the-hard-way tutorial repo as an example (using Athenz as the authorization server), the complete chain connects the previous two RFCs:

User logs into IdP (Keycloak)
   │ Obtains OIDC ID Token
   ▼
AI Client Gateway
   │ Uses RFC 8693 Token Exchange to swap ID Token for ID-JAG
   │ (grant_type=token-exchange, subject_token_type=id_token,
   │ requested_token_type=id-jag)
   ▼
Then uses RFC 7523 JWT Bearer to swap ID-JAG for a usable Athenz Access Token
   │ (grant_type=jwt-bearer, assertion=<ID-JAG>)
   ▼
AI Client calls MCP Server with this Access Token
   │
   ▼
After receiving the request, the MCP Server performs an RFC 8693 Token Exchange "itself"
   │ Swaps the received Access Token for a new Access Token with the "minimum scope needed for this tool"
   │ (This step uses the MCP Server's own mTLS service identity, not the user's credentials)
   ▼
Calls the final Resource Server with the downscoped Access Token

Throughout the chain, the token is exchanged more than once; it is exchanged every time it crosses a trust boundary, and the scope becomes narrower with each exchange. This design intentionally ensures that the "credential" for each segment of the path is different—the token held by the AI Client Gateway cannot be used directly to trick the final Resource Server because the MCP Server stage forces re-verification and re-issuance. The issued Access Token usually records both sub (the proxied user) and act (the identity of the Agent actually performing the operation), so downstream services can clearly see that "Alice performed this operation through a certain Agent," and the audit trail is not broken midway.

Downscoping permissions at every hop: How the Principle of Least Privilege is implemented

This is what I find to be the most beautiful part of the entire architectural design: least privilege is not just a principle written in a document; it is physically enforced by the token exchange mechanism.

Taking my re-implemented id-jag-mcp as an example, it provides three tools:

Tool	Corresponding Athenz Scope
`get_k8s_docs`	`api:role.docs-getter`
`delete_k8s_doc`	`api:role.docs-deleter`
`post_k8s_doc`	`api:role.docs-poster`

When the MCP Server receives a request, it does not directly forward the Access Token sent by the AI Client to the upstream API—it will always use its own mTLS certificate to perform a token exchange with Athenz ZTS for the "scope actually needed by this tool." Only the newly exchanged token will be used to call the upstream. Even if the token scope at the AI Client Gateway is broader (e.g., possessing both read and delete permissions), the MCP Server will only request the specific small piece of permission actually needed for each tool before forwarding.

In other words, no part of the entire system "happens" to have more power than its current task requires—this is not checked via code review or internal regulations, but is architecturally impossible to bypass.

Why re-implement this MCP Server in Go?

The original MCP Server in id-jag-the-hard-way (api_server/mcp/) was written in TypeScript + Express and used a hand-coded JSON-RPC 2.0 protocol (without the official SDK). I wanted to confirm two things: first, if this token exchange architecture is implemented in a different language with a different MCP SDK, can the logic truly be replicated; second, how the official modelcontextprotocol/go-sdk actually performs.

The final implementation maintains the original core logic (same scope mapping, same mTLS token exchange flow) but replaces the protocol layer entirely with the official Go SDK. The mTLS client was custom-built (without depending on Athenz's official Go client library).

Hands-on: Installation, Execution, and Testing

The code is at kkdai/id-jag-mcp (Apache 2.0 license). The project structure and separation of responsibilities are roughly:

cmd/id-jag-mcp/ Entry point: reads settings, assembles all components, starts HTTP server
internal/config/ Environment variable configuration reading
internal/athenz/ mTLS client + Athenz ZTS RFC 8693 token exchange
internal/tools/ Tool input types + shared upstream forwarding logic
internal/server/ MCP tool registration (official SDK) + REST shortcut routes + logging

First, clone the project and build the binary:

git clone https://github.com/kkdai/id-jag-mcp.git
cd id-jag-mcp
go build -o id-jag-mcp ./cmd/id-jag-mcp

To actually run it, you need to prepare mTLS certificates, a reachable Athenz ZTS, and an upstream API server. Configuration is done entirely through environment variables:

mkdir -p certs
cp /path/to/api-mcp.crt /path/to/api-mcp.key /path/to/ca.crt certs/

export UPSTREAM_BASE_URL=http://localhost:14443
export AUTHORIZATION_SERVER_URL=https://athenz-zts-server.athenz:4443/zts/v1

go run ./cmd/id-jag-mcp

After startup, in addition to the /mcp endpoint for MCP client connections, it also provides REST shortcut routes corresponding to the three tools, making it easy to test directly with curl without an MCP client:

curl -H "Authorization: Bearer $AT" http://localhost:8101/api/docs

curl -X DELETE -H "Authorization: Bearer $AT" http://localhost:8101/api/docs/5

curl -X POST -H "Authorization: Bearer $AT" -H "Content-Type: application/json" \
  -d '{"name":"doc1","content":"hello"}' \
  http://localhost:8101/api/docs

If you just want to confirm if the logic itself is written correctly, you don't need to actually set up an Athenz/Keycloak environment—testing uses httptest to simulate ZTS and the upstream API throughout:

go build ./...
go vet ./...
go test ./...

For example, the tests for internal/athenz will start a fake ZTS server to verify if the outgoing grant_type, subject_token, scope, and audience parameters are correct; the tests for internal/tools verify that when each tool forwards to the upstream, it carries the "downscoped token after exchange" rather than the original one received. This way, you can confirm the entire token exchange logic is correct without actually connecting to Athenz.

The README (available in both Chinese and English) contains a complete list of environment variables and more details.

Conclusion

ID-JAG does not solve "whether this client is who it says it is" (that is a PKCE problem), but rather "whether this non-human service identity is qualified to represent a specific person to do this specific thing right now." What supports this architecture is not documentation constraints, but the chaining of RFC 8693 and RFC 7523 standards, ensuring every hop is forced to re-verify and re-downscope permissions.

If your AI Agents have started interacting with internal systems, this is an architecture worth taking the time to understand—and you don't necessarily have to copy Athenz's implementation. The key is to understand the core principle that "every hop must re-issue, and the scope must get narrower," and apply it to your own authorization server.

[Dev Log][Node.js] Feedly Classic is gone, so I built my own FeedFlow (Part 1)

Evan Lin — Tue, 28 Jul 2026 05:12:29 +0000

Introduction:

Since Feedly's redesign, I haven't been able to get used to the interface. I especially miss the high-density, single-column, no-nonsense reading rhythm of the old Feedly Classic. My subscription sources include many English, Japanese, and Korean technical blogs. Every time I encountered them, I either had to open a new tab for translation or simply skip them. Over time, these sources became "read but ignored."

Instead of continuing to settle, I spent a weekend building my own RSS reader: FeedFlow. It features a mobile-first design, dark theme, and multi-column view modes—a tribute to Feedly Classic. Data is stored in Google Firestore, and accounts are tied to LINE Login. For non-Chinese articles, the background automatically uses Gemini 2.5 Flash to translate them into Traditional Chinese. It is currently deployed on Cloud Run, and I use it every day.

This repo will continue to be developed. This post is the first in a series, documenting the skeleton of the entire project and the context behind several key decisions.

TL;DR

This article will introduce:

Why build an RSS reader from scratch
The significance of LINE Login in this project
The RSS reader development process: From standalone MVP to multi-user cloud sync
Why Gemini was chosen as the translation engine and the pitfalls encountered
Frontend interface: Four view modes and mobile-first design
An unexpected interlude: Cleaning leaked keys from git history
Current progress and the direction of the next post
Conclusion
Reference links

Why build an RSS reader from scratch

There is no shortage of RSS readers on the market, but what I wanted was very specific: it needs to be fast on mobile, have high information density (no big images taking up the whole screen for every article), and allow for in-place translation of foreign sources without switching tabs. These three conditions combined are not fully met by any existing service, especially "automatic translation of non-Chinese articles," which is almost never treated as a first-class citizen in other readers.

The project is named FeedFlow, and its core consists of three parts: an Express backend (server.js) responsible for fetching RSS, parsing content, calling Gemini, and reading/writing to Firestore; a frontend using pure Vanilla JS ES Modules (app.js, store.js, api.js, i18n.js) without any frameworks; and Firestore for the database with LINE Login for accounts. The first MVP version produced subscription management, folder categorization, four view modes, and a dark theme. Every subsequent version has been built upon this skeleton.

The significance of LINE Login in this project

I previously wrote about How to develop OAuth2 PKCE via Golang, which discussed the implementation details of introducing PKCE to LINE Login. At that time, LINE Login was the subject of research for deconstructing the protocol. In FeedFlow, LINE Login takes on a different role—it is the key to making the multi-user architecture viable.

FeedFlow initially didn't have an account system; data was stored in the browser's localStorage. If you switched devices, your subscriptions would disappear. To achieve cloud sync, the first step was to have a stable user identity that could serve as a key for document paths in Firestore (users/{userId}/...). Instead of building a custom username/password system, using the LINE User ID (sub claim) obtained via LINE Login as this key saved me from building the entire password, verification email, and "forgot password" flow. For a personal project, this was much more cost-effective.

This decision wasn't made all at once. The earliest version was a "lazy" version: letting users paste a LINE UID string, which the backend just used as a Firestore key without any authentication. This kind of "login" meant anyone could impersonate any UID. The next version switched to formal LINE OpenID Connect: a standard OAuth 2.1 authorization code flow. After obtaining the id_token, the signature is verified using LINE's /oauth2/v2.1/verify, and the session is stored in an HTTP-only cookie rather than being passed around in URL parameters. Later, state and nonce checks were added to block CSRF. This "get it working first, then get it right" sequence reflects the typical rhythm of personal projects: get the features working to confirm the direction, and fix security holes once the need is visible.

Since the primary use case is sharing links within LINE and opening them in the LINE in-app browser, I also integrated the LIFF SDK. This allows users opening the app within LINE to log in directly using LIFF's SSO without being redirected to an external browser.

The RSS reader development process: From standalone MVP to multi-user cloud sync

The features during the MVP stage weren't much different from a standard RSS reader: paste a URL, the backend parses the feed using rss-parser, and if no feed is found, it uses cheerio to scan the webpage's <link> tags for auto-discovery. Subscribed sources can be categorized into folders; articles can be marked as read or "mark all as read"; and a refresh function pulls the latest articles. In this version, all data was still stored in localStorage.

What truly made the project "feel like a product" were the versions after connecting to Firestore. Each user's subscription list, folders, reading progress, and preferences are written to independent paths under users/{LINE_UID}/..., ensuring multi-tenant data isolation. Reading progress is tracked in detail: not just a list of "read article" IDs (readArticleIds), but also which article was last read for each feed (lastReadArticleId), allowing users to pick up where they left off across devices.

An interesting small mechanism is "auto-hydration": when Cloud Run redeploys or a user logs in on a new device, there is no article data in memory, only the subscription list in Firestore. At this point, the backend re-fetches and parses each feed in the subscription list in the background to populate the screen. Users don't see an empty "please subscribe first" screen; the transition is quite natural.

Later, a "rich preview" version was added: when an RSS URL is pasted, in addition to fetching the feed title and description, it also grabs the latest three articles as samples. Non-Chinese content is sent to Gemini for translation, so users can understand what the source is about before actually subscribing.

Why Gemini was chosen as the translation engine and the pitfalls encountered

Translation was necessary because the subscription list contains many non-Chinese sources. The logic is simple: the backend detects the article language; if it's not Traditional Chinese, it's sent to Gemini 2.5 Flash, which returns a JSON structure with translatedTitle / translatedContent. The frontend adds a "✨ Trad-Ch" translation badge to article cards and the reader, and the reader includes a button to toggle between the original text and the translation. Choosing Gemini was straightforward: the latency and cost of 2.5 Flash are suitable for this "translate several articles upon entering the screen" usage. Other APIs could do it, but I was already using this GCP project, so it was easy to integrate.

The first version was the most direct: using GEMINI_API_KEY to call the Generative Language API. It worked, but after deploying to Cloud Run, it meant managing an extra set of API Key environment variables, increasing the risk of key leakage. Later, I changed the translation part to use Vertex AI, authenticating with the Cloud Run service's own identity (ADC, Application Default Credentials). This eliminated the need for a separate API Key—the Cloud Run service account itself has permission to call Vertex AI; you just need to set the IAM permissions. When the ADC environment is unavailable during local development, it falls back to GEMINI_API_KEY.

After this version went live, a whole batch of translation requests failed in the Cloud Run logs with the message: "Neither Vertex AI ADC self-identity nor GEMINI_API_KEY is available," even though both paths were configured. Upon investigation, I found I had misused the @google/genai SDK: it requires a boolean vertexai: true along with project and location as parallel parameters. I had initially written it as a nested vertexai: { project, location }. The SDK's logic for "is Vertex mode on" and "did it read project/location" were separate; consequently, the SDK thought Vertex mode was on but couldn't find the project or region, causing every call to fail at the start.

Instead of wrestling with the SDK's parameter rules, I eventually removed the @google/genai package entirely. I used google-auth-library's GoogleAuth to directly request an ADC token and constructed the HTTP request to the Vertex AI generateContent REST endpoint myself. With one less layer of SDK abstraction, the behavior became much more predictable. In cases like this, bypassing the SDK and hitting the REST API directly is often easier than digging through documentation to find which parameters should be nested or parallel.

Frontend interface: Four view modes and mobile-first design

The reference point for the interface design was Feedly Classic: a view mode toggle button in the top right, with four modes applying different layouts to the same article data without re-fetching.

Mode	Features
Magazine	Default mode, summary cards with images and text, suitable for quickly scanning titles and snippets.
List	High-density text-only list, showing the maximum number of articles at once.
Title Only	Only titles are kept, allowing for the fastest scrolling speed.
Cards	Visual cards focused on large images, suitable for sources with rich visual content.

The sidebar uses a folder structure, listing each folder and uncategorized subscriptions with unread count badges. The article list and the reader are two separate panels; clicking an article slides it from the list into the reader, and a swipe-up gesture slides it back to the list. The operation logic follows mobile native app habits rather than the typical web behavior of full-page jumps. The overall theme is dark. The top bar contains icons for the menu, view modes, mark all as read, refresh, and settings. After logging in, the LINE display name appears where the "LINE Login" button used to be.

In the settings page, "Interface Language" and "Translation Target Language" are split into two independent options. The interface currently supports zh-TW / en / ja. The translation target language determines which language Gemini translates foreign text into. There's no reason to tie these together; a user might want an English interface but still want to translate Japanese articles into Traditional Chinese.

Current progress and the direction of the next post

As of writing this, two new documents have been added to the repo (docs/superpowers/specs/ and docs/superpowers/plans/) to plan for paginated browsing of the article list—five articles per page, supporting swipe gestures, mouse wheels, and buttons for navigation. This part is still under development and will be the subject of the next post in this series.

Conclusion

FeedFlow currently solves a simple problem: quickly scanning a bunch of foreign technical articles on mobile without switching tabs for translation or enduring a bloated interface. Three decisions support this architecture: LINE Login provides user identity without a custom account system, Firestore enables multi-user cloud sync, and Gemini 2.5 Flash (via Vertex AI ADC, without storing API Keys locally) handles translation. None of these were perfect on the first try—LINE Login evolved from string pasting to formal OAuth, and translation moved from raw API Keys to service identity authentication. It was all about getting it running first and then strengthening it.

Reference Links:

[Book Sharing] Tsaisang’s Tales of the Strange: Japanese Mythology, Ghost Stories, and Sometimes Taiwan

Evan Lin — Thu, 23 Jul 2026 13:25:47 +0000

Tsai-sang Talks About the Strange
Japanese Myths and Spiritual Ghost Stories, and Sometimes Taiwan
Rated by 73 people
Author: Tsai Yi-chu  Publisher: Eurasian Publishing Group

Foreword:

This is the first book I finished reading in 2026. I didn't write any book reviews in the first half of this year because I only read a little bit of many books. I spent quite a long time reading this one as well; I happened to see it while looking for books, and it turned out the second half of the book was quite captivating, so I finished it all in one go.

Synopsis

Not just crazy, but crazier! Tsai Yi-chu a.k.a. the Chuunibyou Professor of Folklore unleashes a flood of ghost stories!

◆ How do Japan's first generation of gods play out soap opera dramas every day?
◆ How does Japanese mythology hide various "sexual metaphors" in its stories?
◆ Yokai aren't all troublemakers; which ones can make you rich or send you to heaven?
◆ Is there bullying in the world of Yokai? Can just getting old and ugly make you a Yokai?
◆ Is the Jade Emperor actually not a CEO? Is Guanyin Bodhisattva actually a foreigner?
◆ Can Taiwan also have "Shigong (Priest) Watches" and "Yokai Pokémon"?

Taiwanese people fear ghosts, Japanese people fear ghosts, people all over the world fear ghosts...
It doesn't matter if you haven't seen "Ghost Stories," come here to listen to Tsai-sang talk nonsense and discuss gods and ghosts, so you won't feel "creepy" anymore!

Most people's impression of Japan is—endless temples to visit, super cute and otaku cosplayers, AV actresses... Hey! There must also be supernatural stories, Sadako, Yokai, and various urban legends!

Listen to how Tsai-sang combines hair-raising ghost encounters with historical stories passed down through the ages, and see how he uses super "grounded" slang to reveal the cultural meanings behind Japanese mythology!

Japanese Folklore PhD Tsai Yi-chu has gathered years of research on folklore, using myths and ghost stories as a medium and easy-to-understand "netizen" language to lead readers into Japan's "Gods and Monsters." This includes the genealogy of Japanese gods, the connection between Yokai and culture, and the hidden meanings within. At the same time, he allows Taiwan's various gods to actively participate in the text through a Taiwan-Japan friendly crossover, making you understand ghost talk and become obsessed with gods and ghosts! After reading, I guarantee your mom will ask why you are reading this book on your knees!

Because "Tsai-sang Talks About the Strange" will make you shout on your knees: "What on earth was Japanese mythology on? I want some of that too!"

This book isn't one of those stiff academic papers; it's a cultural analysis book where Tsai Yi-chu (Tsai-sang), a PhD in Folklore from the University of Tsukuba, uses super "grounded" netizen slang and a hilarious style to strip down Japanese mythology and spiritual ghost stories for you! The most brilliant part is that he doesn't just talk about Japan; he occasionally pulls back to Taiwan's folk perspective for comparison.

Here are the three core sections and key points of this book refined for you:

Summary of the Three Core Sections

Section Category	Core Research Focus	Tsai-sang's "Taiwanese-style Plain Language Interpretation" and Highlights
1. Japanese Mythological Prototypes	The birth of Japan's first generation of gods (Izanagi, Izanami, Amaterasu, Susanoo) and the belief in vengeful spirits in history.	Uses "Sex and violence, gore and SOD collections" to roast the absurd plots of Japanese mythology. Introduces the "Vengeful Spirit Fan Club" of ancient Japanese history and endemic species like Tengu.
2. Yokai and Urban Legends	How rural ghosts and monsters evolved into modern urban legends over time (e.g., Slit-Mouthed Woman, Super High-Speed Granny).	Yokai are "new pets of urbanization," reflecting the collective anxiety and loneliness of modern people, and the media's role in fueling the supernatural trend.
3. And Sometimes Taiwan	Cross-sea interactions and cultural comparisons of beliefs between Taiwan and Japan (e.g., Nagasaki Mazu and Tainan's General Flying Tiger).	Demonstrates a "Taiwan-Japan Friendly Crossover." Reflects on Taiwanese people's own cultural roots and subjectivity through the lens of Japanese ghost stories.

I. Japanese Mythology: A First Family More Dramatic Than a Soap Opera

Love and Hate of the First Family: The fallout process of Japan's creator gods (Izanagi and Izanami) is absurd and horrifying (the wife turns into a rotting corpse in Yomi, the husband is so scared he flees and divorces). Their descendants, the Sun Goddess Amaterasu and her brother Susanoo, also have a love-hate relationship. Tsai-sang jokingly says that from a modern perspective, these plots are simply a collection of various horror and gore scenarios.
Vengeful Spirit Fan Club: Many high-ranking gods worshipped in Japanese history (such as Sugawara no Michizane, the God of Learning, and Emperor Sutoku) were actually "victims of political struggles who died miserable deaths." Because later generations feared they would become vengeful spirits and seek revenge, they quickly built shrines to worship them as gods, forming Japan's unique culture of vengeful spirit worship.

II. Yokai and Urban Legends: Collective Anxiety of Modern People

Yokai are the City's New Pets: Former Yokai (like Kappa and Yama-uba) lived deep in the mountains and forests, representing human awe of nature. After urbanization, Yokai also "moved into the city," evolving into urban legends like the Slit-Mouthed Woman and the Super High-Speed Granny.
Reflecting the Loneliness of Real Society: The birth of these modern legends appears to be horror stories on the surface, but at their core, they reflect the alienation and collective anxiety of urbanites. At the same time, the book reviews the "rise and fall of the supernatural craze" fueled by Japanese mass media (TV supernatural programs) for ratings in the 80s and 90s.

III. And Sometimes Taiwan: The Mysterious Connection Between Taiwanese and Japanese Gods and Ghosts

Japanese-speaking Mazu and Japanese Gods: The book specifically mentions the intertwining of Taiwanese and Japanese beliefs. For example, there are several Mazu temples in Nagasaki, Japan, where Mazu "speaks Japanese" due to localization. Meanwhile, in Tainan, Taiwan, there is the "General Flying Tiger Temple," which enshrines Shigeo Sugiura, a Japanese pilot who sacrificed himself to protect Taiwanese villagers during WWII.
Folklore is the "Study of People": Tsai-sang emphasizes that whether studying Japanese mythology or Taiwanese supernatural phenomena, what is ultimately terrifying or absurd is not the ghosts and monsters, but the human society behind them. Belief can soothe the soul because it reflects the logic of contemporary thinking.

Core Quote of the Book:

"To understand people is to understand ghosts; Yokai and spirits are all imaginations based on reality."

We must discover the main reasons why each phenomenon forms. When we use Japanese ghost stories as a mirror to deeply understand the workings of folklore and legends, we can then look back with clearer eyes to discover and identify the cultural identity that belongs to "Taiwan itself."

Reflections

This book is full of many rural legends, and the final section, which includes one of his research reports and actual events, will leave you astonished. First, the book begins by sharing stories of Japanese ghosts and monsters, reflecting on the origins of many ghost stories behind Japanese mythology. It also discusses the connection between Japanese sex and violence and their many ghost stories.

The second part shares some Taiwan-related stories and also discusses the origin of the "Turbo Granny" in Dandadan and the stories related to the Slit-Mouthed Woman. These will make you want to read it all in one breath. As the Ghost Month seems to be approaching again recently, it seems this series of books will become popular again. Everyone should check it out.

[Book Sharing] Taiwan's AI Future: Analyzing Trends, Local Landscape, Corporate Strategy, and Personal Development

Evan Lin — Thu, 23 Jul 2026 13:25:21 +0000

Taiwan's AI Future
Analyzing the latest AI trends, Taiwan's situation, corporate strategies, and personal development
Author: Chien Lee-feng, Hsiao Yu-pin  
Publisher: Business Weekly

Preface:

This is the second book I finished reading in 2026. It is a fairly new book, released at the end of 2025. I bought it because my company invited Chien Lee-feng to give a speech in 2024. Later, I happened to see his book on my e-book shelf and decided to take a look.

Outline

When AI rewrites the world, what is Taiwan's next step?
Former Managing Director of Google Taiwan, computer scientist, and AI scholar—Chien Lee-feng
Writes the first instruction manual for the AI era for Taiwan,
Helping Taiwanese people understand the opportunities and challenges of the AI age!

The world undergoes a digital revolution every ten years:
● 1990: Personal computers start the computer generation;
● 2000: The Internet creates the web generation;
● 2010: Mobile devices and social media lead the mobile generation;
● 2020: Generative AI like ChatGPT makes a shocking debut...
Now is the AI generation, where the rules of the game are completely rewritten, and the gap is rapidly widening between 1:99.
Will you fall behind and be eliminated, or seize the opportunity and become a 1% winner?
This book takes you deep into the AI landscape to master the key to transformation!

【AI Development under Geopolitics】
As the world undergoes an AI-driven paradigm shift, the US sees AI as the key to its return to hegemony. This not only predicts that AI productization will completely subvert the world's rules of operation but also opens up infinite possibilities for the future evolution of AI. From the "Manhattan Project" to the "Stargate" layout, this book will deeply analyze the global situation under the US-China tariff war and provide insights into how AI reshapes the international order.
● The 1:99 challenge: Countries, companies, and individuals who seize the opportunity may become the unique "1" that far surpasses others, while others become the "99" lagging far behind.
● The emergence of DeepSeek has subverted the US monopoly, bringing a "rebalancing" to the AI world, equivalent to inventing the "poor man's atomic bomb."
● If the pace of domestic chip production is not accelerated, an unproductive America will have no tomorrow and will directly lose competitiveness in the AI battle. TSMC has thus become the X-factor in the US-China confrontation.

【Taiwan Looking at the World】
The AI wave is sweeping the globe; this is not just a technological innovation, but a key turning point for national development. In this giant wave, Taiwan not only has the uniquely endowed "Silicon Shield" TSMC but also sees a golden decade to become "the world's Taiwan" as AI challenges and opportunities coexist. This book gives you a glimpse into the future potential of Taiwan's manufacturing industry and how old and new enterprises are redefining "Made in Taiwan."
● Facing geopolitical changes, Taiwan's manufacturing-oriented enterprises should go with the flow. Through overseas production and R&D in Taiwan, they can create a "Taiwan + N (foreign)" model, helping Taiwan remove the red supply chain and join the US-led supply chain.
● An island's market is always outside. Flying to Japan or the US for travel or business for a few days does not equal internationalization. Internationalization is daily life being impacted by different cultures.

【AI Practice in All Walks of Life】
The AI era is a key moment for corporate transformation and talent reinvention. Only companies that dare to pivot will have competitive opportunities. This book lists cases of how companies in different industries respond to AI and provides practical strategic directions to guide Taiwan's corporate transformation to seize the AI market and move towards growth and innovation.
● The impact of AI can be compared to "musical chairs." From tech giants to SMEs, whether it's "emptying the cage for new birds" (industrial restructuring) or empowering employees, wherever the wind blows, new opportunities lie there.
● "Old-ventures + New-ventures": Shifting from software integration to hardware-software integration, combining the advantages of both, makes AI applications possible.
● Developing Sovereign AI doesn't end with outsourcing. Whether building your own model or asking tech giants for help, the strategy must be planned clearly, otherwise, it might just be a waste of money.

【Mastering the Golden Key to Personal Learning and Career】
As a member of the AI generation, how to use AI to improve learning efficiency while clearly identifying AI's limits is an important task. This book suggests how to use AI tools while pointing out that human differentiated experience will become an irreplaceable treasure. Therefore, cleverly accumulating personal unique value is the only way to remain invincible in the AI era.
● AI likes to use certain specific sentence patterns because AI is a probabilistic concept, so naturally, there are some patterns. But conversely, precisely because its data volume is large enough, it can try various combinations that humans have never seen.
● AI has raised the "passing line" of many jobs from 60 to 80 points in one fell swoop, forcing all industries to redefine the core competencies and value of human labor.
● In the AI era, senior talents with professional foundations learn AI the fastest because their long-term accumulated knowledge can judge the correctness of AI-generated content. This AI paradigm shift has, in turn, amplified the advantages of the older generation.

This is an AI survival guide tailored for Taiwan, helping you fully grasp the context of the AI revolution and find the path to growth for the nation, enterprises, and individuals amidst the changes.

This book can be described as an "AI Era Survival Manual" tailored for Taiwanese people and businesses by Dr. Chien Lee-feng, former Managing Director of Google Taiwan, and senior media professional Hsiao Yu-pin. Dr. Chien uses a very pragmatic and precise local perspective to analyze how Taiwan should reposition itself, how companies should play the international game of hardware-software integration, and how individuals can avoid falling into the crisis of "brain outsourcing" under this crazy AI wave.

I have organized the four core frameworks of the book for you to help you grasp the overall context through this overview:

Overview of the Book's Four Core Frameworks

Category	Core Pain Points & Trends	Breakthrough Strategies for Taiwan & Individuals
1. Latest AI Trends	AI brings high centralization and unification, potentially evolving into a 1:99 disparity in capabilities and resources; however, the rise of emerging forces like DeepSeek is bringing "rebalancing" opportunities to the world.	Understand the essence of AI's "probability and language architecture," find breakthroughs outside of US monopolies, and raise the lower limit of basic capabilities.
2. Taiwan's Positioning	Although Taiwan is a key X-factor in geopolitics and AI chips, it also faces structural constraints like island involution, a declining birthrate, and the "five shortages."	Completely shift from a "farmer's mindset" to a "navigator's mindset," taking "going global" as the only way to survive, and stepping beyond Taiwan's borders to expand digital territory.
3. Corporate Transformation	Taiwan is "extremely strong in hardware, extremely weak in software." Software startups lacking computing power and business scenarios find it hard to survive independently on the international stage.	Promote "Old-ventures (hardware giants) + New-ventures (software applications)" collaboration, using Edge AI to add "brains" to powerful hardware devices.
4. Personal Development Keys	Facing the invisible crisis of "brain outsourcing," mediocre professional newcomers who only teach by the book and lack practical experience will be the first to be hit.	Shift from a "problem-solving" habit to a "problem-posing" mindset, creating unique value through high-frequency "repeated interaction and correction" with AI.

I. Latest AI Trends: The 1:99 "Superhuman" Challenge

Extreme Concentration of Power: The AI era has brought high centralization, with global tech giants holding a massive advantage. Among thousands of languages worldwide, only about a hundred can be used in mainstream AI, and English and Simplified Chinese are deeply optimized. This means language and cultural frameworks are the primary keys to mastering AI.
The 1:99 Watershed: The cruelest part of this tsunami is not the elimination of ordinary people at the bottom (AI actually raises the floor for ordinary people), but the elimination of "mediocre professionals." The 1% who seize the opportunity will become superhumans through AI, taking the capabilities and opportunities of the 99%, while others become the lagging 99%.
Rebalancing the AI World: The recent emergence of non-US low-cost, high-efficiency models has broken the absolute monopoly of US tech giants. This has been described as inventing the "poor man's atomic bomb," bringing an opportunity for a reshuffle to countries and enterprises with fewer resources.

II. Taiwan's Situation: From "Island Involution" to the "Age of Discovery"

The Geopolitical X-Factor: TSMC and Taiwan's hardware supply chain hold a key position in the US-China tech confrontation because Taiwan possesses the characteristic of "knowing the demand earliest" (e.g., being able to grasp system requirements like server voltage changes first), giving it an important identity in the adjustment of global infrastructure.
Breaking the Farmer's Mindset: Most Taiwanese companies are used to an "island mindset," where the "sea is invisible" in daily life, making it easy to fall into involution within a comfortable echo chamber. Facing the structural crisis of a declining birthrate and plunging newborn numbers over the next 20 years, Dr. Chien urgently calls for a shift to a "navigator's mindset," because "going global" is the only way for all industries in Taiwan to survive.
Extension of Digital Territory: Taking TSMC as an example, AI allows Taiwan to replicate factories overseas and have them operated remotely by Taiwanese engineers. Taiwan should also turn the crises of aging and labor shortages into opportunities by actively developing robotics and its own Sovereign AI to avoid a national-level digital divide.

III. Corporate Layout: Hardware-Software Integration, Letting "Old-ventures + New-ventures" Dance Together

Using Edge AI to Add Brains to Hardware: Edge AI (referring to terminal devices having local computing power without relying entirely on the cloud) is Taiwan's domain. It is hard for pure software startups in Taiwan to compete with international giants, but we can embed and bundle AI services directly into powerful hardware devices used worldwide (such as Giant bicycles or various terminal equipment), significantly increasing added value.
Old-ventures plus New-ventures Playing the International Game: Today's AI startups can hardly succeed without the data, computing power, and real "business scenarios" provided by a "rich father." Therefore, hardware giants (Old-ventures) should join hands with software startups, combining the international channels of the former with the flexible applications of the latter to go global as a team.
Pragmatic Planning for Sovereign AI: Developing Sovereign AI cannot just be about blindly outsourcing business to tech giants. Whether building their own models or cooperating with major manufacturers, enterprises must clearly plan their own strategies and field applications; otherwise, it is just a waste of money.

IV. Personal Development: Refuse "Brain Outsourcing," Be a High-Level "Questioner"

Shift Thinking from "Solving" to "Posing": AI's capabilities are "asked" out. The more professional the question, the more accurate the response. The future workplace will no longer value rote memorization; core competencies will shift to problem definition, critical thinking, and direction control. Only "questioners" who can demonstrate proactive influence will win.
Using "Iterative Refinement" to Deepen Learning: If you just throw a question to AI, get an answer once, and copy it directly, this behavior is equivalent to plagiarism. However, if you can go back and forth with AI to modify it 10 times, that is a "learning" process; if you continue to repeatedly correct and adjust up to 100 times, that is truly approaching the level of "creation."
Accumulating Irreplaceable "Differentiated Experience": Functions like memory and calculation can be outsourced to AI, but your unique personal experience, cross-disciplinary collaboration ability (π-shaped talent), and human critical thinking are the irreplaceable treasures of the AI era. Cleverly using AI tools to amplify your output is the only way to avoid becoming the "lost generation" eliminated by the times.

The Core Soul Quote of the Book:

"Change is humanity's eternal unease, but from a macro perspective, what AI brings is an opportunity to make humans more capable." When calculation and memory are outsourced from the brain, be sure to retain your power of thinking and creation.

This video Interview with Dr. Chien Lee-feng: Who is the Lost Generation of the AI Era deeply explores the "1:99 Superhuman Challenge" and workplace mindset transformation mentioned in the book, helping you more intuitively understand how to retain personal competitiveness in this era of brain outsourcing.

Personal Thoughts

From my own perspective, this book organizes many recent AI development processes both domestically and internationally. It provides many future insights based on Chien Lee-feng's own experience as the former Managing Director of Google Taiwan. It often shares advice on how various industries should face the AI era. This part is frequently mentioned in his speeches and is shared and explained quite clearly.

Personally, I feel that one can just skim through this book. In comparison, I still prefer Dr. Chien Lee-feng's speeches, which provide more impact and inspiration.

[GCP in Action] LINE Business Card Bot Evolution: Dual-Side Recognition and Merging with Gemini

Evan Lin — Thu, 23 Jul 2026 13:24:50 +0000

Pain Point: One Chinese Business Card is Actually Two Business Cards

Business cards in Taiwan often have a common design: Chinese printed on the front and English on the back (or vice versa). Our LINE business card bot's original logic was very simple: receive an image, perform OCR once, and save one record.

This is where the problem lies. When a user sends the front side, the bot saves a record with only the Chinese name. If the user then sends the back side, the bot treats it as "another new business card," resulting in two records for the same person in the database, each missing half the information. Users have to manually compare and delete duplicate data, which is a terrible experience.

This article records how we taught the bot to recognize that "these are two sides of the same business card" and merge the information from both sides into a single complete record.

Solution: First Ask, "Is there a back side?"

Instead of writing rules to guess if two images are the same business card, we chose a more direct approach: Ask the user.

The workflow is designed as follows:

The user sends the front of the business card, and the bot performs OCR as usual.
After OCR is complete, the bot does not save immediately. Instead, it replies with "📇 Front side data recognized. Is there a back side to this card?" and provides two Quick Reply buttons.
User clicks "No, save directly" → Save according to the original process and finish.
User clicks "Yes, there's a back side" → The bot remembers the front image and waits for the next image.
Once the back side image arrives, both images are sent to Gemini together to be merged into a single record before saving.

We manage this waiting state using the user_states memory dictionary already present in the project, and add a 5-minute timeout. If a user clicks "Yes, there's a back side" but then ignores it or does something else, the process is treated as abandoned after 5 minutes, preventing the entire workflow from getting stuck.

user_states[user_id] = {
    'action': 'pending_backside_confirm',
    'card_obj': card_obj,
    'front_image_bytes': image_content,
    'expires_at': time.time() + PENDING_BACKSIDE_TIMEOUT_SECONDS
}

Core: Let Gemini See Two Images at Once and Merge Them

The most critical technical decision was: should we recognize the front and back sides separately and then write code to merge them? We chose another path: wrapping both the front and back images in a single generate_content request and letting Gemini handle the judgment directly.

The reason is simple: merging Chinese and English names into a format like "Wang Daming David Wang" using string rules is prone to being messy and inaccurate. Semantic-level integration is more likely to fail with hardcoded rules, so letting Gemini handle it directly is much easier.

In app/gemini_utils.py, we added generate_json_from_two_images, which reuses the existing NAMECARD_SCHEMA structured output, but this time the contents includes two image Parts:

def generate_json_from_two_images(
        front_img: PIL.Image.Image,
        back_img: PIL.Image.Image,
        prompt: str) -> object:
    model = GenerativeModel(
        "gemini-3-flash-preview",
        generation_config={
            "response_mime_type": "application/json",
            "response_schema": NAMECARD_SCHEMA
        },
    )
    front_part = Part.from_data(
        data=pil_to_bytes(front_img), mime_type="image/jpeg")
    back_part = Part.from_data(
        data=pil_to_bytes(back_img), mime_type="image/jpeg")
    response = model.generate_content(
        [prompt, front_part, back_part],
        stream=False,
        labels={"client_id": "namecard"}
    )
    return response

The prompt also just adds a merge instruction after the original IMGAGE_PROMPT:

DOUBLE_SIDED_IMAGE_PROMPT = IMGAGE_PROMPT + """
These two images are the front and back of the same business card; please integrate them into a single complete record.
If both Chinese and English appear in the same field (such as name or company), please present them merged
(e.g., "Wang Daming David Wang");
if a field appears on only one side, use the value from that side; ignore obviously redundant information.
"""

One API call means we don't have to write or maintain any merging rules ourselves.

Two Easily Overlooked Pitfalls

During the overall code review before the feature went live, we caught two details that are easily overlooked but can really cause issues.

Pitfall 1: Timing of Duplicate Checks

Originally, the duplicate check (comparing if the email already exists) was done immediately after OCR. However, after the double-sided recognition went live, if the front side happened to have a duplicate email from an old record, the process would prematurely determine it "already exists" and end. This would mean any new email on the back side would never be seen.

The fix was to move the duplicate check later, performing it only after "single-side chosen not to merge" or "double-sided merge complete." This ensures we are always comparing the final version of the data:

async def _finalize_and_save_card(card_obj, event, user_id) -> None:
    existing_card_id = firebase_utils.check_if_card_exists(card_obj, user_id)
    if existing_card_id:
        # ... Reply already exists
        return
    card_id = firebase_utils.add_namecard(card_obj, user_id)
    # ... Reply save successful

Both the single-sided and double-sided merge workflows eventually converge to call this shared function, ensuring the duplicate check is executed only once when the data is finalized.

Pitfall 2: Don't Clear All States Indiscriminately

The user_states dictionary is actually shared by several features: editing memos, modifying fields, and this back-side recognition. The initial implementation, for convenience, would delete the entire state whenever a residual state was detected before processing a new event.

The problem is: if a user is "editing the phone field" and waiting to input a new number, but accidentally sends an image, this logic would clear the editing_field state as well, silently canceling the user's original editing operation.

The fix was to only clear the two states related to the back-side recognition process and leave other states untouched:

if state.get('action') in (
    'pending_backside_confirm', 'awaiting_backside_image'
):
    del user_states[user_id]

We later found a missing branch in the same logic; the part handling "operation expired" replies also cleared everything initially. It was only truly fixed after unifying it with the same selective judgment.

Incidental Resource Cleanup: Don't Let Back-Side Images Linger in Memory

The awaiting_backside_image state stores not just text, but also the raw byte data of the front image. If a user disappears after being asked "Is there a back side?", this data would theoretically stay in the process memory because the original design only checked and cleared timeout states during the "user's next interaction."

We added a sweep_expired_states() function, which runs immediately when a Webhook comes in. It clears all expired temporary states for all users, so we don't have to wait for the specific user to return for passive cleanup:

def sweep_expired_states() -> None:
    now = time.time()
    expired_user_ids = [
        user_id for user_id, state in user_states.items()
        if 'expires_at' in state and state['expires_at'] <= now
    ]
    for user_id in expired_user_ids:
        del user_states[user_id]

Whenever any user sends a message, it performs garbage collection for all users, ensuring that those who abandoned the process don't leave behind memory-consuming remnants.

Summary and Benefits

This double-sided recognition and merging feature makes the LINE business card bot much more aligned with the actual usage habits of Taiwanese users:

Single Recognition, Complete Data: Both sides are sent to Gemini at once, automatically merging Chinese and English fields, eliminating the need for manual duplicate comparison.
Non-Intrusive: Ignoring prompts, timeouts, or temporarily doing something else will naturally revert to single-sided storage without getting the workflow stuck.
Duplicate Checks Target Final Data: Ensures that comparisons are always made against the merged, complete version, so new information appearing only on the back side isn't missed.
Independent State Machines: The temporary state for back-side recognition only affects itself and doesn't interfere with other ongoing user operations.
Clean Memory Usage: Proactively cleaning up timeout states ensures that users who abandon the process don't leave an invisible memory burden.

The complete code has been pushed to GitHub, feel free to check it out!

DEV Community: Evan Lin

[AI in Practice] Building a Dynamic LINE Group Buying Bot with Edit and Unsend Webhooks

When Messages Can Be Taken Back: Building a "Shapeshifting" LINE Group Buying Bot with Edit and Unsend Webhooks

Starting with "Edit Message" in LINE Labs

From UI Features to Bot Data Consistency

Group Buying Transformation: Giving the API Demo a Story

Structure of the Edit Event

The First Trap: Edit Events May Not Arrive in Order

The Second Trap: Edited Content May No Longer Be a Valid Order

Unsend Event: Beyond Technology, User Intent

Making State Changes Visible with Flex Message

SDK and Execution Environment Versions

State Issues When Deploying to Cloud Run

How to Run This Demo

What Else Can This Be Applied To?

Conclusion

References

[AI in Practice] Gemini 3.5 Transcribe: Real-time Transcription and Speaker Diarization in a macOS Meeting Translation App

Previously

Two Models with Names Differing by Only Two Words

The Real-time One: Interim is the Key

The Batch One: Uses a Completely Different API

Phase 1: Adding Pure Transcription Mode to the App

A Preventive Trade-off: Duplicate Words

How to Run Tests in a Project Without a Testing Framework

Phase 2: Speaker Diarization Requires More Changes Than Expected

Major Pitfalls and Solutions

Pitfall 1: Chunking cannot be "fill 30 minutes and leave a remainder"

Pitfall 2: spk_1 in Segment 2 is not the same as spk_1 in Segment 1

Pitfall 3: Joining words results in "H e l l o W o r l d"

Pitfall 4: Two asynchronous processes racing to write the same summary

Pitfall 5: During the TDD RED phase, the test itself crashed

Pitfall 6: Scroll anchor fails in the new mode

Pitfall 7: Letting AI guess names—the key is stopping it from hallucinating

Privacy and Cleanup: Don't leave behind what you don't want to keep

Results and Benefits

Postscript: Documentation examples might give you an empty result without errors

Create a Minimal Reproducible Example First

Root Cause 1: Word-level annotations must be explicitly enabled

Root Cause 2: The actual label format differs from the documentation

My own fallback made failure look like success

After the Fix

[Go in Practice] Writing Modern Go with AI: Testing JetBrains go-modern-guidelines and Refactoring a 1,039-line main.go

Background

What is go-modern-guidelines?

The problem it aims to solve: Models have a knowledge cutoff, Go doesn't

How it works: Two commands, a list that grows with Go versions

The two-layer information design is actually to save context

Installation

This project: a main.go that grew to 1039 lines

What the health check found

1. Code that compiles but will never execute

2. Group messages cause a panic

3. /recent_files returns folders

4. User input directly concatenated into the Drive query

5. /quit treated as a search command

What go-modern-guidelines actually changed

http_servemux_patterns: Also removed a piece of manual path checking

cmp_or: Three fallback segments turned into three lines

strings_cut_prefix_suffix: Replacing manual string slicing

slices_sort_func + min: Fixing that fake sorting

any, errors_is: Small details

crypto/rand.Text(): A suggestion that requires a version upgrade first

Major Pitfalls and Solutions

Pitfall 1: The skill explicitly tells you not to grep, and I did exactly the opposite the first time

Pitfall 2: Your go.mod might not allow the syntax suggested by the tool

Pitfall 3: After changing go.mod, the tool's answers changed

Pitfall 4: The tool manages syntax, not architecture—and architectural pitfalls are deeper

Pitfall 5: ServeMux was written correctly, but Cloud Run won't let you use it

Pitfall 6: I thought I added context to all external calls, but I missed an entire path

Pitfall 7: Bugs that don't make a sound need to be actively hit by tests

Results and Benefits

[LINE Bot Debugging] A 403 Error and Three Underlying Issues: npm Version Drift, Node.js Requirements, and Cloudflare Verificati

Previously

Why Run Cloud Build Three Times Instead of Deploying Immediately After Changes

System Architecture

Core Implementation

1. Dockerfile: Locking Versions for Both Node.js and single-file-cli

2. Detecting Cloudflare Challenge Pages

Major Pitfalls and Solutions

`http_servemux_patterns`: Also removed a piece of manual path checking

`cmp_or`: Three fallback segments turned into three lines

`strings_cut_prefix_suffix`: Replacing manual string slicing

`slices_sort_func` + `min`: Fixing that fake sorting

`any`, `errors_is`: Small details

`crypto/rand.Text()`: A suggestion that requires a version upgrade first

Solution 3: Replace `ls`, `cat`, and `cd` with Smarter Versions

Pitfall 1: `compinit` complaining about "insecure directories"

Pitfall 2: `zsh-syntax-highlighting` must be the last line

Pitfall 3: `ast-grep` was intentionally left out of the whitelist