DEV Community: weiyi

Langfuse in Practice: Make Slow, Expensive and Wrong AI Responses Diagnosable

weiyi — Mon, 02 Nov 2026 05:22:56 +0000

After an AI application launches, the painful moment is often a user saying “it got this wrong again” while you cannot tell which step failed.

Did retrieval miss the document, or did the prompt omit a condition? Was the model slow, or did a tool time out and retry three times? Did a cheaper model save money per call but make the whole task more expensive?

Langfuse connects these questions to one task's execution. Inspect the call tree, inputs, outputs, latency, usage and evaluation results, then decide what to change. A dashboard is the entry point; fixing problems is the benefit.

This article follows official documentation checked on October 11, 2026, and Python SDK 4.17.0. The example demonstrates instrumentation without calling a real model. No production benchmarks or deployment results are invented.

Distinguish traces, observations and sessions

Concept	What it records	Common mistake
Trace	One complete task: a chat turn or agent run	Putting a day's requests into one trace
Observation	One step: retrieval, model invocation or tool execution	Recording only the final answer
Session	Related turns or successive tasks	Replacing individual task boundaries

Use one trace per question, with retrieval and generation as children. Link multiple conversation turns with a session. The current SDK uses OpenTelemetry; correct context propagation matters more than attractive names.

Do not wrap an entire agent loop in one generation. Record each model call separately and associate tools with the relevant step. Otherwise you see the total bill but cannot find the result that inflated the context.

Instrument one workflow first

One RAG question-answering endpoint is enough to start:

answer-question
├── retrieve-documents
└── draft-answer

Record the question and final result on the root, document IDs, versions and supplied passages on retrieval, and model inputs, outputs and actual usage on generations.

Keep names stable, such as retrieve-documents. Put order numbers, questions and request IDs in attributes, not names. Add release, environment and prompt version to make before-and-after comparisons possible.

A minimal Python example

This example pins Python SDK 4.17.0, requiring Python 3.10 or newer. Check SDK and server versions separately; do not mix methods from old tutorials.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "langfuse==4.17.0"

export LANGFUSE_PUBLIC_KEY="pk-lf-REPLACE_ME"
export LANGFUSE_SECRET_KEY="sk-lf-REPLACE_ME"
export LANGFUSE_BASE_URL="https://cloud.langfuse.com"

Get project keys in Langfuse settings. BASE_URL above is the EU Cloud endpoint; replace it for another region or self-hosting. Keep secrets in server-side environment variables. Merely writing a .env file does not make Python load it; these commands export variables explicitly.

Save this as trace_demo.py and run python trace_demo.py:

from langfuse import get_client

langfuse = get_client()
question = "How long are repeat article visits deduplicated?"

with langfuse.start_as_current_observation(
    as_type="span",
    name="answer-question",
    input={"question": question},
    metadata={"release": "demo-v1", "environment": "development"},
) as root:
    with langfuse.start_as_current_observation(
        as_type="span", name="retrieve-documents", input=question
    ) as retrieval:
        documents = [{"id": "privacy-v1", "text": "Repeat visits count once within 30 minutes."}]
        retrieval.update(output=documents)

    with langfuse.start_as_current_observation(
        as_type="generation",
        name="draft-answer",
        input={"question": question, "documents": documents},
        metadata={"simulation": True},
    ) as generation:
        answer = "Repeat visits count once within 30 minutes. [privacy-v1]"
        generation.update(output=answer)

    root.update(output={"answer": answer})

langfuse.flush()

Documents and answers are hard-coded, with simulation=True. The generation makes no model request and reports no invented tokens or costs. Replace both sections with your retrieval and model functions, record the actual model ID and copy real usage from the provider response.

Call flush() before a short-lived script exits. Handle long-running services through their lifecycle, rather than flushing after every token or tiny step. Verify propagation across async tasks, threads and queues: a context manager alone does not guarantee cross-process continuity.

What to inspect once traces arrive

Start with a request a user has confirmed is wrong:

Input conditions: missing product model, date or applicable version?
Retrieval: relevant, current documents with essential conditions preserved?
Model call: complete context and unexpected retries?
Tool result: did an HTTP success actually complete the business action?

Inspect step durations and total task time. Streaming also requires distinguishing first-token latency from completion. Count every call and retry when measuring cost. Estimates depend on model pricing configuration and usage; reconcile the provider's bill separately.

Langfuse does not prove answer correctness or enforce business permissions. Showing an unauthorized attempt is not preventing it. Authentication, validation, idempotency and write confirmation remain application responsibilities.

Prompt management: preserve the version actually used

Moving text into an editable interface is only the start. Link versions to calls so failures can be reproduced and changes compared.

Create a text prompt named support-answer using {{question}} and {{context}}, then label an accepted version production:

prompt = langfuse.get_prompt("support-answer", label="production")
compiled = prompt.compile(question=question, context=documents[0]["text"])

compile substitutes variables; it does not invoke a model. Fetching production selects an intentionally released version. Creating a version should not automatically deploy it. Associate the returned prompt object with the generation through your integration.

SDK caching does not remove every dependency. First fetches, refreshes and failures still matter. Important workflows can prefetch at startup and retain a validated fallback prompt, with logs showing when fallback occurred.

Evaluation: turn failures into regression cases

Start with twenty or thirty real, sanitized cases: ordinary and ambiguous questions, missing evidence, tool timeouts and questions the system should not answer. Define expected behavior before comparing prompts, models or retrieval strategies.

Method	Useful for	Watch for
Deterministic code	JSON, required fields, business conditions	Correct format with wrong content
Human annotation	Expert judgments, edge cases, rubric calibration	Inconsistent standards and review costs
LLM-as-a-Judge	Batch scoring against a clear rubric	Bias, unstable scores and evaluator fees

Online evaluation monitors live traffic; offline experiments compare candidates on a fixed dataset. Use both and inspect important failures rather than hiding them in an average. Calibrate model judges against human labels; keep explicit code checks for permissions and critical rules.

Evaluation has its own cost. Budget sampling rate, judge model and frequency together instead of evaluating everything and discovering monitoring costs more than the application.

Cloud or self-hosting: count operations too

For a first trial, use a Cloud test project and sanitized samples; assess production-data requirements before uploading them. Self-hosting offers control over deployment and data flows, but is more than running one container.

The current architecture includes Web, Worker, PostgreSQL, ClickHouse, Redis/Valkey and S3/Blob storage. PostgreSQL holds transactional data, ClickHouse observability data, cache services the queue, and object storage raw events and multimodal attachments.

Official documentation positions Docker Compose for local use and testing, without built-in high availability, scaling or backups. Production operation needs TLS, authentication, database and object backups, queue monitoring, capacity, retention and upgrade recovery.

Distinguish the open-source core from Enterprise additions requiring licenses. Cloud limits, plans and self-hosting costs are different measures; this article does not substitute an unverified price for a total-cost calculation.

Mask before export and verify after integration

Traces can be more sensitive than ordinary logs: questions, document text, tool parameters and answers may appear together. Do not collect real email addresses, phone numbers, tokens and complete internal files by default.

Sanitize before data leaves the application. Current Python documentation recommends mask_otel_spans for new setups, covering exported OpenTelemetry attributes from SDK and third-party instrumentation; legacy mask has different coverage. Inspect automatic integrations as well as hand-written fields.

Verify normal requests, failures and script shutdown on one workflow. Review exported content and project permissions before expanding.

The first goal is modest: when a user reports a bad answer, find that request, explain the failure and turn it into a pre-release check. That makes Langfuse more than another AI dashboard.

Official sources

Checked October 11, 2026. Complete integration still needs your project credentials and acceptance testing.

This article was prepared for DEV with AI assistance.

AI is splitting up the work. I'm more interested in who finishes the job

weiyi — Mon, 02 Nov 2026 05:22:23 +0000

When I saw that ChatGPT could put several agents to work at once, my first thought was writing for this blog. Checking documentation, finding examples and verifying prices could finally happen in parallel.

Then I thought about the results coming back. Someone still has to read them together. Prices are an obvious trap: one agent finds subscription pricing, another finds API pricing. Put those straight into an article and fluent writing won't save it.

Add Cursor's cloud-agent update, Google DeepMind's Minecraft experiment and Anthropic's quantum-circuit research, and my question becomes: how much work would these actually save in my own projects?

Research in parallel, then sit down and compare the results

OpenAI's announcement says GPT-6 Astra Pro in ChatGPT supports parallel agents, with individual progress tracking and completed tasks you can open to review. The announcement explicitly makes it available to Pro users in the Chat tab.

I'd try that for a technical article. One agent reads official documentation, another finds examples, another checks pricing and access conditions. That should be quicker than waiting for one agent to do everything in order.

But I wouldn't immediately combine their answers into a draft.

The web app and API of the same product can offer different capabilities. A fresh announcement might not yet be reflected in the documentation. Each answer can have a source and still mislead readers when placed beside the others.

So I'd still ask: which version, what date and which region? Only once those line up would I use the material. Research can happen simultaneously; I don't want to skip that final comparison.

With Cursor, environment management is what I'd like to try

Cursor's update lets cloud agents build interactive canvases. Finish a dashboard or report, send someone a link, and they can view it. Convenient enough.

For development, though, I'm more interested in the beta Environments API: creating and managing agents' cloud environments programmatically.

Spending ages on setup before writing any code wears down my patience. The repository is cloned but dependencies aren't right. The service finally starts, then a missing environment variable turns up. Get it working, switch tasks, and start again.

Agents need those problems solved, too. If repositories, dependencies and test environments can be prepared and reused, at least each task doesn't have to begin with a conversation about how to start the project.

Cursor says large multi-repository environments load five times faster. I'd read that as less waiting before work begins, not a fivefold improvement in the whole development process.

The repository limit isn't consistent: the X post says 500, while the changelog and agent-creation API documentation say 300. If I needed that many, I'd plan around the documented 300 and confirm access to a higher limit first.

Four agents playing Minecraft: I'd like to see the handoffs

Google DeepMind's experiment put four Gemini 4 Argon agents into Minecraft from scratch. They defeated the Ender Dragon after roughly 27 hours.

According to the announcement, they operated through code and text, wrote 2,783 programs, divided jobs over chat, shared notes and debugged together, without humans helping them complete the game tasks.

But the researcher's full account includes important details. They played on Easy, read their surroundings and acted through Mineflayer, and had a pathfinding plugin. They didn't see game pixels or use a keyboard and mouse like a human. Researchers could monitor, pause or stop the experiment from a dashboard.

It wasn't a smooth run, either: 605 respawns, with 22% of programs ending in an error. The log records victory at 27 hours and 14 minutes. “Finished in 27 hours” can hide a lot of retries.

The win is worth watching, but I'm more curious about what happened during those hours.

When one agent finds resources, how does it tell the others enough to save them unnecessary trips? When something goes wrong, who notices and brings everyone back on track?

Development has similar problems. Several agents being able to code doesn't guarantee they'll work well together. One changes an interface while another builds a page against the old version. Put them together and it will still break.

For my blog, I'd consider dividing translations by language, or having one agent check links while another checks pages. I'd be much less comfortable with several agents changing styles and routes at once. Each might think its work is finished; the combined result still needs a look.

Before deciding whether this transfers well to real projects, I'd also want repeated-run success rates and the cost of those 27 hours. Without them, I can't work out whether it's worthwhile.

Knowing whether the result is right matters more to me

There's an arrangement I like in Anthropic's quantum-circuit research: give Claude Mythos 5 circuit specifications and an automated evaluator.

After a design, a program checks correctness and scores “qubit count × gate count.” The model gets the feedback and tries again. Improvement doesn't depend entirely on the model explaining how well it did.

The report says all eight selected tasks produced designs more resource-efficient than the best previously published approaches. One Reed–Solomon decoder used roughly one-twelfth the gates of Google's earlier construction. That's a circuit-design comparison, not a twelvefold speedup of an entire quantum computer. Nor does the research claim no human participation: the point-addition task ran for 53 hours, with an initial brief and occasional human prompts to explore more broadly.

Quantum circuits are far from my daily work, but I need ways to check AI's results when building a website, too.

If it says a page is fixed, I'd like to see it on a phone. If it says a link is repaired, open it. If it says a release is live, confirm we're seeing the new version. Having those checks makes it easier to delegate more.

Otherwise, four “finished” messages just mean asking four times: finished where? Show me.

So I'd start these features with a few independent blog tasks. Agree on which files each agent can change and what it should return, then look at the combined work afterward.

I'd like them to finish their parts so I can check and use the result. If I still need four or five conversations open to track down who changed what, I'd rather start with fewer agents.

This article was prepared for DEV with AI assistance.

WorkBuddy or Codex? Choose by the Work You Need Done

weiyi — Mon, 02 Nov 2026 05:21:15 +0000

If most of your work involves documents, spreadsheets and presentations, I’d start with WorkBuddy. If you spend your day changing code, investigating errors and running tests in a project directory, I’d start with Codex.

Both can write code and handle files. The useful question is where your work starts and what you need to take away. “Which is smarter?” won’t get you far.

The main difference is the workflow

WorkBuddy is Tencent’s general-purpose AI workspace. Its official capabilities include research, documents, spreadsheets and presentations, as well as website and application development. Give it a set of materials and specify the output you need: that is a natural way to use it.

Codex has terminal, IDE and cloud entry points. For local development, it can inspect a project, edit files and run commands with tools already on your machine. Your existing build, test and Git workflows can continue.

Task	Start with	Check
Competitor research and reports	WorkBuddy	Reliable sources and supported conclusions
Combining spreadsheets and making slides	WorkBuddy	No lost fields; usable output files
Adding features to an existing project	Codex	Consistent structure and reasonable changes
Fixing bugs, testing and code review	Codex	Correct diagnosis and verified results
A new website or small tool	Try either	How easily it fits future maintenance

This is a suggested division of work, not a limit on either product. Results also depend on the model, permissions and connected tools. If a task fails, check whether it lacked sources, command access or clear requirements before switching products.

With WorkBuddy, specify what you want to take away

“Analyze competitors” is too broad. You may get lots of text that still needs organizing. Try this instead:

Use these three product websites to make a comparison table of pricing, main features and intended users. Cite a source for each item. Mark missing information “unconfirmed.” Finish with three recommendations. Deliver an editable spreadsheet and a one-page summary.

Now the materials, scope and format are clear. For spreadsheets, add the deduplication field, currency units and rules for missing values. Every omitted rule leaves another guess to fix later.

Split large jobs into two stages: facts and tables first, then the report or slides after you approve the direction. Formatting before checking facts creates unnecessary rework. Save recurring templates and rules in project materials or skills; next time, replace the inputs rather than explaining everything again.

With Codex, give it a goal you can check

“Optimize the website” can produce a long list of changes without a clear improvement. Be specific:

Fix overflowing article tables on mobile. Keep horizontal scrolling and check Chinese and English pages. Explain the cause, changed files and how you verified the fix.

For backend errors, provide the trigger, relevant logs and expected behavior. For changes spanning modules, ask it to diagnose the problem and explain its proposed edits before implementing them.

Put lasting project rules in AGENTS.md: build commands, directory conventions and deployment steps. Keep it short and useful.

You don’t need the most expensive model or strongest reasoning setting for every job. Try faster settings for copy edits, styling and simple batch changes. Increase reasoning for cross-module work, elusive bugs and complex reviews. Exact choices depend on the models currently available.

Review the changes and rerun the scenario that failed. “Done” is a notification; code and execution results are the evidence.

Using both: make the handoff clean

For a blog, WorkBuddy could organize topics, sources and outlines, while Codex handles pages, multilingual structure, APIs and deployment. Assign reports and channel copy according to the output needed.

Hand over files, settled decisions, remaining tasks and unverified information. A long chat transcript makes the next tool guess again.

Don’t have two agents edit the same code at once. Let one implement while the other reviews without editing, or use separate branches or work directories for parallel development. Otherwise you may have to repair overwritten changes yourself.

Which is better value?

Don’t compare only subscription fees or response speed. Run three to five recurring tasks and record completion time, usage or fees, and your own correction time.

An answer in one minute that takes half an hour to repair costs more of your time than a five-minute result you can check and use.

If you want just one tool, pick the one closest to your daily work. Start with WorkBuddy for office materials, Codex for existing code projects. Add the other when you find a concrete gap.

Sources checked October 13, 2026. These recommendations use official capabilities, not a comparative hands-on benchmark.

This article was prepared for DEV with AI assistance.

The Roadster tempts me. But I'm not putting down RMB 332,000 yet

weiyi — Mon, 02 Nov 2026 05:20:58 +0000

My first reaction to the latest Roadster news was: finally, something is happening again.

Then I saw the production year. 2028.

That's Musk's latest timing, according to an October 16 report from Cailian Press. After all the years since the 2017 unveiling, another date doesn't immediately get me excited. Starting production isn't the same as handing a car to a buyer in China, either.

I still want to see the car. As for reserving one now, I'll wait.

Who is it for? I wouldn't make it my only car

I don't find the likely buyer hard to picture: everyday transport is already covered, and they want something they simply enjoy owning. Take it out on a sunny weekend. Enjoy looking at it in the garage, too.

Some people love a Ferrari's sound, others a Porsche's driving feel. There will also be people drawn to Tesla's slightly adventurous approach. At this point, saving electricity isn't really the issue.

If my budget also had to cover family transport, commuting and long trips, I wouldn't be considering a Roadster. Those needs would come first.

It plays a similar role within Tesla. Model 3 and Model Y sell in volume; the Roadster gives people a reason to think Tesla can still make something surprising. The audience discussing it can be much larger than the audience buying it.

The current Chinese reservation information describes the Performance version as a two-seater, with active aerodynamics and a multimode intake system. The old descriptions of a four-seat sports car may no longer match what is being offered.

I'd like to know what all that feels like on the road. We aren't at the point where I can answer that yet.

RMB 332,000 is a reservation payment. I'm reluctant to queue this early

Tesla's Chinese site currently lists an initial payment of RMB 332,000. That's a reservation payment, not the car's price.

The frequently quoted US$200,000 and US$250,000 figures were the original 2017 prices for the base and Founders Series versions. With so much time passed and the product changing, converting those figures into a present-day Chinese on-road price doesn't tell me much.

This is what holds me back: I can't yet work out the final cost, the specification I'll receive and the delivery date together. RMB 332,000 can buy an everyday car. I'm not keen to exchange it for a place waiting for updates.

Other people will be happy to. I understand wanting to join the queue early, particularly if you've followed Tesla and waited years for this car. But I'd read the Chinese reservation agreement before paying, especially the refund process and what changes when it becomes a firm order. An American report saying a deposit is refundable doesn't make the Chinese agreement unnecessary.

As for selling it in China, the visible route today is online reservation. My reading is that Tesla can take interest from its keenest buyers first and let the car's publicity draw attention. We don't yet have enough information about Chinese allocations, delivery or final pricing to claim we know its full sales plan.

When it comes to buying one, I doubt the salesperson will need to spend long explaining electricity costs per 100 km. I'd spend more time asking where it can be repaired, how long special parts take and what happens if I damage the underbody.

An electric sports car needs more than acceleration now

In 2017, the Roadster's acceleration figures were memorable by themselves. Today, China already has plenty of very quick EVs.

The Yangwang U9 and MG Cyberster occupy different price brackets and offer different experiences, but they show how much choice has grown for an EV with character. I need the Roadster's final price before deciding what to compare it with.

If it competes for traditional sports-car money, I'm even more interested in how it drives. One hard launch is impressive. Can it sustain performance for several laps? How are the brakes? Is the steering enjoyable? Those matter more to me than a single demonstration sprint.

Then there are the dull questions that could come up every day: will it clear the garage ramp, is the seat comfortable after an hour, and can I replace a damaged tyre without a saga? Buying a toy doesn't mean wanting to spend every day looking after it.

Musk's past talk of a SpaceX cold-gas thruster package, even “flying,” is interesting to watch. I wouldn't build that into my valuation yet. Let's see the final Chinese specification and what's permitted on public roads.

One detail worth keeping straight: 0–60 mph is acceleration to roughly 96.6 km/h, not 0–100 km/h. Both the date and the units matter when reading this car's specifications.

I'd like driver assistance. The Tesla badge doesn't confirm the features

You might reasonably ask whether a Tesla this expensive would have FSD.

I'd like to know, too. But as of October 16, 2026, the Chinese reservation information I could find did not establish the complete driver-assistance specification, FSD functions or pricing for the Chinese Roadster. Calling it standard equipment, or promising city driving assistance at delivery, is premature.

I'm perfectly happy to have assistance in a sports car. After a day out, help following traffic in a jam sounds useful. Easier parking would be welcome, too. Enjoying driving and wanting an easier time in congestion aren't contradictory.

But if driver assistance were my main reason to buy, I'd try cars already delivered with their functions enabled. There's no need to imagine the Roadster's future Chinese features for it. Even if supervised FSD is offered, the driver still has to watch the road; it isn't driverless operation.

So would I buy one? For now, I wouldn't reserve it. I'd wait for production cars.

Once Chinese pricing is available and there's a car to try, I'd happily go and see it. If it's enjoyable to drive and isn't too troublesome to use, there will still be time to get excited.

What I'd really like to read now is what the first owners think after a few months. I've seen plenty of Roadster launch news over the years.

Sources: Tesla China reservations; Cailian Press, October 16 production timing report; IT Home, September 20 reservation and historical pricing report.

This article was prepared for DEV with AI assistance.

Deploy a Static Site on Alibaba Cloud: Codeup, Flow, OSS and ESA

weiyi — Mon, 02 Nov 2026 05:20:41 +0000

Once an article is written, the release process should deliver it reliably to readers. Build, upload, refresh and verification need distinct steps so you can see where a failure occurred.

A clear release path

This project currently builds and publishes through local commands, uploads static output to OSS and serves it through ESA. Codeup and Flow configuration templates are ready, but the actual repository and cloud pipeline are not connected yet.

When configuring Flow later, reuse these steps:

Install dependencies from the lockfile with npm ci.
Run npm run build to check types and generate pages.
Upload static assets first, then publish HTML and Sitemap.
Refresh ESA caches and verify that public content matches the build.
Notify search engines and save successful submission state.

Treat different content differently in the cache

Content-hashed assets can be cached for a long time. HTML and Sitemap benefit from revalidation and a refresh after publication so readers don't receive stale content.

Cache refresh is asynchronous. Verify that public pages return the new content before submitting URLs, reducing the chance that a search service sees an old page or error response.

Make failures recoverable

A multi-file OSS upload isn't atomic. If it fails midway, rerunning can complete the missing parts. Search notifications save successful batches, leaving only pending content for a later attempt.

A platform accepting a request does not mean an article is indexed. Check website publication and search notifications separately. See the project's deployment documentation for configuration and maintenance details.

This article was prepared for DEV with AI assistance.

SIG Mesh or a Proprietary Protocol? Connecting Bluetooth Devices to a Gateway

weiyi — Mon, 02 Nov 2026 05:20:24 +0000

The previous post got the Matter half working with nothing more than a PC and a phone. Next comes the other half of the gateway: what protocol should the gateway and the Bluetooth devices actually speak?

Our devices originally used a Bluetooth advertising protocol we defined ourselves, controlled through our own app. It had run for a long time without trouble. So early in the project, someone on the team asked: do we really need SIG Mesh?

That question deserves its own post, because "skipping SIG Mesh" can mean two very different things with very different consequences.

First, what does "skipping it" mean?

Option one: don't use the SIG Mesh protocol. The devices keep their proprietary advertising protocol, and the gateway speaks it too.

Option two: use the SIG Mesh protocol but skip Bluetooth SIG qualification. Technically follow the standard, just don't register or pay with the SIG.

The first is a technical choice; the second is a compliance question. A lot of debates go nowhere because they mix the two up. Let's cover the technology first and qualification later.

Four ways to connect devices to a gateway

Approach	Coverage	Number of devices	Interoperability	Security	Effort
Proprietary advertising	Around the gateway only, no relaying	No hard limit	Only your own products	All on you	Small
Direct BLE connections (each device connects to the gateway)	Around the gateway only	Limited by concurrent connections, usually a dozen or so	Fair	Standard encryption	Medium
Home-grown proprietary mesh	Relaying possible	Depends on implementation	Only your own products	All on you	Very large
SIG Mesh	Relaying, whole-home coverage	Hundreds of nodes are fine	Works across vendors	Standard encryption and replay protection	Medium, mostly in the chip SDK

Proprietary advertising is the simplest: to turn on a light, the gateway broadcasts a command and the device acts on it. But there's no relaying. If the gateway is in the living room, the bedroom light may never hear it. Provisioning, encryption and replay protection all have to be designed and maintained by you.

Direct BLE connections work like a phone and a pair of Bluetooth earbuds. Encryption is standard, but a gateway can only hold a limited number of connections at once, nowhere near the dozens of switches and lights in a typical home.

A home-grown proprietary mesh means reinventing SIG Mesh. Controlling relay storms, de-duplicating messages, rotating keys, protecting sequence numbers against replay: every one of them is a trap. Plenty of vendors built their own mesh in the early days; most have since moved to SIG Mesh.

SIG Mesh is the Bluetooth SIG's standard. Devices relay for each other and cover the whole house, the security model is already there, and, most importantly, it defines a standard set of device models.

Where SIG Mesh really pays off: standard models

SIG Mesh specifies how common devices such as switches and lights are described and controlled:

SIG Mesh model	Device	Matter equivalent
Generic OnOff	Switches, plugs	On/Off Light
Light Lightness	Dimmable lights	Dimmable Light
Light CTL	Tunable white lights	Color Temperature Light
Light HSL	Color lights	Extended Color Light

This table matters enormously for a gateway. Matter has device types that map almost one to one, so the translation is very direct. With a proprietary protocol, you'd have to define that mapping yourself and revisit it for every new device type.

Standard models bring two more benefits:

The gateway and the devices can use chips from different vendors. Our gateway uses a Silicon Labs module and our devices use Telink chips. As long as both follow the standard models, they work together, and no single chip vendor locks us in.
Other ecosystems become easier to join. Xiaomi's Bluetooth Mesh, for example, is built on SIG Mesh, and its light and switch templates use these same models. A Xiaomi version later on could reuse most of the application code, replacing mainly the provisioning and authentication layer.

What SIG Mesh costs you

SIG Mesh isn't free; most of the cost has simply been paid by someone else already.

The protocol is complex. Provisioning uses ECDH key exchange, messages are encrypted at both the network and transport layers, and there's segmentation, sequence numbers and the IV Index. All of that lives in the chip vendor's SDK and has already passed SIG testing. On the device side, use the SDK; don't write your own.

Network size has limits. SIG Mesh forwards messages by flooding relays, so the more nodes in one area, the more crowded the air gets. As a rule of thumb, a network performs best with a few hundred nodes or fewer. Commercial lighting with thousands of points needs to be split by floor or zone into several networks and gateways.

The gateway can only store so many nodes. This is easy to overlook:

Telink's official gateway firmware stores up to 200 nodes by default, or 110 with remote provisioning enabled. A four-gang switch counts as one node, not four.
If you build a gateway on Nordic's Zephyr stack, the default configuration allows only 8 nodes, with a replay-protection list of just 10 entries. Leave it unchanged and messages from the 11th device will start failing.

Two hundred nodes is plenty for a home, but check these numbers during design.

A device belongs to only one network. The same light can't be managed by our gateway and by a Xiaomi hub at the same time. Supporting both ecosystems means two product versions, provisioned separately.

Can you skip SIG qualification?

Back to the second meaning of the original question: using SIG Mesh without Bluetooth SIG qualification.

Technically it makes no difference. The devices work normally and interoperate with other SIG Mesh devices. The risks are elsewhere:

Risk	Details
Patent license	The SIG patent license only covers qualified products
Sales channels	Amazon and other platforms, large retailers and overseas distributors often ask for a Bluetooth qualification ID
SIG enforcement	The SIG notifies unqualified products and asks for qualification or withdrawal
Trademark	You can't use the "Bluetooth" name or logo on packaging or marketing

A common misunderstanding: going back to a proprietary protocol doesn't avoid SIG qualification. Any product that uses Bluetooth needs qualification to be sold, Mesh or not. Without Mesh, you simply skip the Mesh-specific tests.

The good news is that qualification doesn't necessarily mean retesting. If you use a stack the chip vendor has already qualified, without modifying its qualified parts, you can reference their qualification and just submit a declaration and pay the fee. If the Bluetooth design of the switches, lights and wireless switches is identical, they can share one design and one fee.

Note also that radio certifications such as SRRC, CE and FCC are legal requirements and separate from SIG qualification. Without them, the product can't be sold legally. A later post will cover certification costs and order.

On the gateway side, where does Mesh live?

On the devices, using the chip SDK is uncontroversial. On the gateway, the Mesh stack can live in one of three places:

Approach	How it works	Pros and cons
Implement Mesh in the gateway program	Use the Linux board's own Bluetooth to send and receive Mesh advertisements	One chip fewer, but thousands of lines of protocol stack to write and your own qualification testing
Bluetooth module as a co-processor	The Mesh stack runs on the module; the main program sends commands over UART	Proven stack and the vendor's qualification can be referenced; one extra module
Module as a plain Bluetooth controller	The module runs standard HCI; Mesh runs on Linux (for example BlueZ)	Off-the-shelf module firmware, but no development on Windows

We chose the second: a Bluetooth module as a co-processor. The hardest, most qualification-sensitive part of Mesh stays with the chip vendor, and we handle only the serial commands and the Matter translation. Which vendor's module makes the best co-processor is the subject of the next post.

Final thoughts

So, can you skip SIG Mesh?

Not using the SIG Mesh protocol: possible, but you lose relaying, standard security and cross-vendor interoperability, and you still need Bluetooth qualification.
Using SIG Mesh without qualification: functionality is unaffected, but exports, online marketplaces and major retail channels are effectively off-limits.

For us, the biggest value of SIG Mesh isn't the word "standard". It's a set of device models that map almost one to one onto Matter. That keeps the gateway's translation layer simple and leaves us free to choose our chips.

If your devices still use a proprietary protocol and you're weighing a move to SIG Mesh, or you're already building a Bluetooth Mesh gateway, I'd be glad to talk: linktr.ee/weiyii

Cover image: Japanwikisan, CC BY 4.0.

This article was prepared for DEV with AI assistance.

Same Claude Opus 5.5: Why Some People Ship an App and Others Call It Dumb

weiyi — Mon, 02 Nov 2026 05:19:59 +0000

Same model, Claude Opus 5.5. Some people say it gets a week's work done in half a day. Others say it gets two out of every three questions wrong.

I recently used it to build a mobile app called Ruzai (如在). When you're away from your computer, it lets you use your phone to view the AI assistant's conversations on that computer, send it messages, and approve the critical actions it wants to run.

Product design, UI, code, GitHub Actions pipelines that build Android APKs and push iOS builds to TestFlight, a self-hosted OTA update server, App Store listing assets: nearly all of it was done by Opus 5.5 inside Claude Code. Along the way we switched tech stacks once, evaluated two cross-platform frameworks, and I rejected the UI three times.

Looking back, model capability was almost never the bottleneck. With the same model, whether the results are good comes down to the person using it. This post breaks that down into a few concrete gaps, based on what actually happened during the project.

The tech stack, up front

Area	What we ended up with	Tried / abandoned along the way
Mobile framework	React Native 0.88 + Expo SDK 58	uni-app x (abandoned), Lynx 4.0 (abandoned after a prototype)
Routing and compilation	Expo Router, React Compiler, TypeScript 6	TypeScript 7 (incompatible with typescript-eslint)
Gestures and animation	Reanimated 4.7, Gesture Handler 3.3
Builds	GitHub Actions: Android APK, iOS upload to TestFlight
OTA updates	expo-updates protocol, hosted on my own server
HarmonyOS	Native ArkTS version, builds cleanly in CI
Development and debugging	Expo Go plus a real device, with the AI driving the phone directly over adb

Every "abandoned" and "switched to" in that table was a disagreement between me and the AI. Let's go through them one by one.

Gap 1: Say "it's ugly" and the AI can only guess, three rounds in a row

When the first version of the UI came out, I said: it's ugly.

Round one, Opus 5.5 changed the color scheme. It found three established palettes online and built them into the prototype for me to choose from:

Option	Source	Primary color
A Warm Cream	Claude's official palette	Terracotta orange `#c96442`, off-white background `#faf9f5`
B Teal	Radix Sage gray + Jade green	`#29a383`
C Indigo	Radix Slate gray + Indigo blue	`#3e63dd`

All three palettes were perfectly sensible. I still thought it looked ugly.

Round two, it stopped just swapping colors and rebuilt the whole thing in the style of the 2026 Gemini App redesign: pure white or pure black backgrounds, no borders anywhere, an oversized greeting, and a floating pill-shaped input box.

In this round it also tracked down a cause I never would have guessed: the screenshots themselves looked cheap. The mockups were captured with headless Chrome on Windows, where Chinese text renders in Microsoft YaHei by default. After switching to Source Han Sans (Noto Sans SC), the look instantly got close to MiSans and PingFang on a phone. Part of what I'd been calling ugly in the first two rounds was really the screenshot font.

Round three, I figured the problem might be the icon colors, so I had it drop gradients entirely. It produced six solid-color icon options (black background with a bold sans, black background with a serif, a vermilion seal stamp...), and we settled on a white "Z" on solid black.

Only after three rounds did I realize I'd been handing out verdicts the whole time, never direction. If my first sentence had been "conversation-first like ChatGPT, black and white, no gradients," the first version would have been close to final.

AI has no aesthetic preferences. What it gives you is the average of a huge number of designs. If you can't articulate what you want, toss it a screenshot of an app you like. That beats ten rounds of "make it look more premium."

Gap 2: Whatever the spec says, it does to the letter

This project had two computers doing the work: one handled the server and the mobile app, the other handled the Windows client. The AIs on each side didn't share memory. They coordinated through spec documents in the code repository: the protocol, the APIs, what needed to change this time. One side wrote and committed the doc, the other pulled it down, followed it, and marked the status when done.

This setup worked really well. When the desktop client got a new icon and was hooked up to the mobile relay, this side wrote the handoff doc first, and the other side just followed it and got it done.

But it has a side effect: any decision written into the doc, the AI will carry out without compromise.

At one point, the spec switched the mobile tech stack to uni-app x. Opus 5.5 followed it and wrote an entire uni-app codebase, committed it, fixed the issues, and did a beautiful job. Only after it was done did I realize this locked the product into a single platform vendor, DCloud: push notifications, builds, and OTA updates would all depend on them. So I tore it down and started over with React Native + Expo.

The AI won't ask on its own, "Will you regret this choice three years from now?" Irreversible decisions like tech stack selection have to be made by a human.

Gap 3: Have it write code first, or have it explain why first

Later I wanted to try ByteDance's Lynx: one codebase for Android and iOS, and possibly HarmonyOS down the road. Opus 5.5 quickly put together a prototype that compiled, connected to the server, and rendered conversations.

But the UI had gone ugly again, nothing like the finalized design.

This time I didn't let it jump straight into fixes. Instead I said: no code yet. First let's work out why, then come up with a plan, and only then write code.

The reasons it gave were very specific:

Area	React Native version	Lynx prototype	Why
Icons	SVG line icons	☰ ＋ ↑ text symbols	Lynx can't render SVG out of the box
Sidebar	Slides in from the left, 84% width, with an overlay	Replaced by a full-page list	No ready-made drawer or modal components
Bottom sheet	Picking a computer, confirmation details with screenshots	None	Same as above
Overall	Tuned piece by piece against the prototype	Thrown together in an hour to prove feasibility	Lynx can't use React Native components; only pure logic can be reused

On top of that, testing turned up regressions: in Lynx 4.1.0, taps inside a scroll-view didn't fire, and in ReactLynx 0.128 all taps stopped working entirely, so we had to roll back to 4.0.3 and 0.126.

Once the reasons were laid out, the decision was easy: stick with React Native.

If I'd just said "make the UI look good," it would probably have spent hours re-tuning everything on Lynx, only for us to switch back in the end. The faster AI writes code, the further it runs in the wrong direction.

Gap 4: The more power you give it, the clearer the boundaries need to be

For on-device testing, I had the AI drive an Android phone directly over adb: tapping buttons, typing text, taking screenshots to check results. That's how a lot of issues surfaced that the desktop prototype simply couldn't reveal:

Symptom	Cause	Fix
Keyboard covers the input box	With edge-to-edge enabled, the layout didn't make room for the keyboard	Switched KeyboardAvoidingView to padding
Taps do nothing after the screen dims	The first tap gets used to wake the screen	Send KEYCODE_WAKEUP before testing
Sidebar swipe gesture ends the moment it starts	The native ScrollView stole the touch	Switched to Gesture Handler's ScrollView, plus a 36dp-wide transparent hit zone along the left edge
Long conversations stop halfway when opened	During batched layout, the code's own scroll-to-bottom also fired scroll events, which got misread as "the user scrolled up"	Only finger drags can change the "at bottom" state
Text typed via adb never appears	The WeChat keyboard intercepted the input	Tap a candidate word to commit it

But things did go wrong once: it hit the back button, landed on the phone's home screen, and its next few taps went straight into WeChat and Alipay.

No harm done, but it was enough of a wake-up call: an AI that can only chat can only talk; an AI hooked up to tools can verify its own work, and can also cause trouble on its own. The same model performs at a completely different level depending on whether it has tools. How much access to grant, which device to test on, and which apps are off-limits all need to be spelled out up front.

Gap 5: Where you review the work determines what you see

The screenshot font issue earlier was really a review problem: if you look at results in the wrong environment, even good work gets dismissed.

Builds are the same. Android packages built by GitHub Actions were signed with a different debug certificate every time, so installing a new version meant uninstalling the old one first. Committing a fixed debug.keystore to the repo finally solved it. You'd never catch that without installing twice on a real phone.

App Store listing assets are an even bigger pile of details:

Asset	Required dimensions
Title artwork	5244 × 2950 or 3840 × 1646
Search result artwork	5244 × 2950, 3840 × 2560, or 1920 × 1280
iPhone 6.3-inch screenshots	1206 × 2622 or 2622 × 1206

The AI got all these dimensions right on the first try, but the sample data in the screenshots only got swapped for neutral content after I looked them over and asked.

Then there was the store description. It wrote it based on our real internal use cases, with every feature covered, but describing it that literally would very likely have gotten stuck in review. The AI didn't know that; I did. We ended up reframing it as "use your phone to remotely control and view the AI conversations on your computer," with the subtitle "Remote-control and view the AI on your computer."

AI can get things right, but only someone who's been through it knows which things, done right, are actually risky.

Gap 6: Do you know what to ask?

Late in the project, I asked: "For small UI tweaks from now on, can we just push an update instead of rebuilding every time?"

That one question bought me a full OTA update system: using the expo-updates protocol, with update bundles hosted on my own server. runtimeVersion uses the fingerprint policy, so JS-only releases update automatically when the phone opens the app, and only changes to native code require a new build.

If I hadn't asked, it wouldn't have done it on its own. Not because it couldn't, but because it didn't know this mattered to me.

Likewise, the HarmonyOS version only started because I asked, "Can you build a native HarmonyOS version?" It wrote one in ArkTS that builds cleanly on GitHub Actions, and along the way worked through a few pitfalls you only find by actually doing it: Huawei's ohpm registry frequently returns 503 to GitHub's runners, so retries are needed; imageknife 3.2.8 and above requires API 18, so we had to stay on 3.2.7.

AI goes exactly as far as your questions take it. People who know how to ask naturally get more out of it.

Gap 7: Are you really using "the same model"?

One last fact that often gets overlooked: when two people say "I'm using Opus 5.5," they may not be using the same thing at all.

Difference	What it actually means
Client	Web chat, Claude Code, all sorts of wrapper apps, each with a completely different system prompt
Tools	Whether it can read files, run a terminal, drive a browser or a phone
Reasoning effort	Thinking levels, fast mode; the same model varies a lot between high and low settings
Context	1M context or standard length; whether old conversations get compacted, and what gets lost when they do
Route	Official API, a cloud provider, or various third-party relays; the version and limits behind them aren't necessarily the same

I built Ruzai with Claude Code: it can read and write the entire repository, run commands, call adb, and use GitHub. Comparing that to a one-question-one-answer session in a web chat was never a fair fight.

Final thoughts

Putting the seven gaps side by side:

	Feels "dumb"	Feels "smart"
Aesthetics	Just says "it's ugly"	Gives reference screenshots, says clearly what not to do
Spec docs	Once it's written, it's settled	Makes irreversible decisions yourself
Order	Jumps straight into code	Reasons and plan first
Tool access	None, or wide open	Gives tools, and draws boundaries
Review	Looks at screenshots on a computer	Checks on a real device, with real fonts and real data
Questions	Waits for it to think of things	Knows what to ask
Setup	Only looks at the model name	Looks at client, tools, effort level, route

Across the whole project, Opus 5.5 wrote nearly all the code, but every turning point was a human decision: dropping uni-app x, passing on Lynx, removing gradients, rewriting the store copy, adding OTA updates.

So next time you feel like the AI has "gotten dumber," don't rush to switch models. Take another look at the job description you gave it, and ask whether that was the part that wasn't very smart.

For the more technical side of things, like retrieval, context curation, reasoning budgets, and verification, I wrote about those in Before Switching Models, Make Your AI Smarter. The two pair well together.

If you're also building your own product with Claude Code or another AI, I'd love to chat: linktr.ee/weiyii

Cover image: Crew, CC0.

This article was prepared for DEV with AI assistance.

Choosing a Matter Gateway Processor: RK3506 vs RK3562 vs RK3576 vs RDK X5

weiyi — Mon, 02 Nov 2026 05:19:31 +0000

The previous post settled the gateway's Bluetooth chip: a Silicon Labs module runs the Mesh stack and takes commands from the host over UART. This post is about the other chip, the main processor that runs Linux and the gateway program.

There's far more choice here than for the Bluetooth chip. We looked at Rockchip's RK3506, RK3562 and RK3576, plus D-Robotics' RDK X5, which a partner company recommended. The conclusion wasn't to pick one chip but to offer three tiers by product positioning.

First, how much performance does a gateway really need?

Lay out the requirements and a lot of "high-end" turns out to be unnecessary:

Requirement	What it takes
Gateway program	A single Rust program using a few dozen MB of RAM
Matter	Very little traffic; commissioning crypto is heavier but happens once
Network	Ethernet or Wi-Fi; 100 Mbps is plenty
Bluetooth module	One UART, with RTS/CTS hardware flow control (Silicon Labs BGAPI has no checksum)
System	Minimal Linux, read-only root filesystem, A/B partitions for OTA
Security	The Matter device certificate's private key needs protection, ideally with secure boot and OTP

For a pure Bluetooth Mesh to Matter bridge, compute barely matters. What does matter: UART flow control, secure boot, SDK quality, long-term supply and price.

RK3506: the cheapest, enough for bridging

The RK3506 is a fairly new entry-level Rockchip chip:

Three Cortex-A7 cores at 1.2 GHz, 32-bit, plus a Cortex-M0.
DDR3L memory, typically 128 MB to 512 MB.
100 Mbps RMII Ethernet, no PCIe; Wi-Fi only over SDIO or USB.
Flexible pin multiplexing with plenty of UARTs.

Core boards are cheap; Luckfox's Lyra modules with 512 MB RAM and 8 GB eMMC list at a little over 100 yuan (about US$15).

For a gateway it's enough: the program needs a few dozen MB, and three A7 cores handle Matter and Mesh translation easily. We confirmed the gateway program compiles for the RK3506's 32-bit ARM target (armv7-unknown-linux-musleabihf) without code changes.

Two caveats:

Choose 512 MB. The 128 MB variants are too tight beyond a demo.
Check secure boot, OTP and OP-TEE support first. The chip is new, documentation is thin, and this determines how the private key is protected for Matter certification.

The downsides are clear: 32-bit, little memory and no NPU leave almost no room for future features.

RK3562: some headroom, the mainstream choice

The RK3562 is our "standard" choice:

Four Cortex-A53 cores at 2.0 GHz, 64-bit.
LPDDR4, 1 to 4 GB.
Gigabit Ethernet, PCIe, USB 3.0 and around ten UARTs.
A 1 TOPS NPU.

MYIR, Forlinx, Industio and others sell core boards listed at roughly 150–300 yuan (US$20–40); development boards run 500–600 yuan.

The reason to choose it is headroom: 64-bit, gigabit Ethernet and 2 GB of RAM leave space for local automation, a web admin page or more protocols later. Rockchip's Linux SDK support is mature, and one Buildroot command produces a complete firmware image.

One lesson: buy the development board and the production core board from the same vendor. The BSP and device tree you develop against then match production, saving a porting round.

RK3576: only for a control panel with a screen

The RK3576 is a big step up: eight cores (4×A72 + 4×A53), a 6 TOPS NPU and much stronger display and video. Core boards start around 500 yuan (about US$70).

For a pure bridge it's clearly overkill. It suits a different product: a smart home control panel with a screen, such as a 7-inch wall display that shows device status and scenes while doubling as the gateway. The NPU would also help with local voice or AI features later.

RDK X5: workable, but not a good fit for a gateway

A partner company suggested D-Robotics' RDK X5. The specs are impressive:

Eight Cortex-A55 cores at 1.5 GHz.
A 10 TOPS BPU (D-Robotics' AI accelerator) and 4K video encoding and decoding.
A module integrating RAM and eMMC, optionally with Wi-Fi 6 and Bluetooth 5.4.
The 4 GB development board costs 549 yuan (about US$75).

Technically it runs a gateway fine: it's 64-bit ARM Linux, our program works as is, the needed interfaces are there, and there's a Buildroot SDK with a secure firmware option.

In a gateway, though, the issues pile up:

Issue	Details
Wasted compute	The selling points are 10 TOPS and vision processing, which a gateway never uses
Cost	Module prices vary widely by channel and are clearly above the RK3562 overall
Ecosystem	A single vendor, with far fewer community resources and solution partners than Rockchip
System	The main image is Ubuntu desktop; production needs the Buildroot SDK, whose maturity needs checking
Maturity	The module documentation we found still said "product under development"; production status and supply need confirming
Duplicate Bluetooth	The Wi-Fi module also includes Bluetooth, but our Bluetooth is handled by a separate Mesh module, leaving an unused radio

It's a great robotics and vision AI platform, just not the right base for a cost-sensitive gateway.

The four side by side

	RK3506	RK3562	RK3576	RDK X5
CPU	3×A7, 32-bit	4×A53, 64-bit	4×A72 + 4×A53	8×A55
AI compute	None	1 TOPS	6 TOPS	10 TOPS
Typical RAM	128–512 MB	1–4 GB	4–8 GB	2–8 GB
Ethernet	100 Mbps	Gigabit	Gigabit	Gigabit
Listed core board price	~US$15	~US$20–40	From ~US$70	Varies by channel
Ecosystem and SDK	New, thin docs	Mature	Mature	Single vendor
Best for	Bridging only, lowest cost	Bridging plus extras	Screen panel, AI	Robotics, vision AI

Our choice: three tiers

Version	Processor	Positioning
Budget	RK3506	Bluetooth Mesh to Matter only, lowest cost
Standard	RK3562	Headroom for local automation, web admin and more protocols
Premium	RK3576	Control panel with a screen that doubles as the gateway

All three share one software stack: Rockchip's SDK with Buildroot, with the gateway program built for different targets and identical Bluetooth modules and device firmware. That's the layered design from the first post at work: swap the processor and neither the Bluetooth nor the Matter layer changes.

For validation we start with an RK3562 development board, which has the most headroom and is easiest to debug. Once everything works, the same program moves to the RK3506 to bring costs down.

Questions to ask before ordering

Whichever chip you choose, ask the vendor these before buying boards:

Can the UART run RTS/CTS hardware flow control, and are those pins broken out?
How is the SDK delivered, does it include Buildroot, and which kernel version?
What's the status and documentation for secure boot, OTP and OP-TEE?
What's the production price and lead time for the core board, and is there an end-of-life plan?
Which onboard Wi-Fi module is used, and is its driver already working?

Final thoughts

It's easy to let specs drive the choice of processor: cores, TOPS, RAM, always reaching for more. For a gateway, what decides success are the unglamorous details: UART flow control, secure boot, a usable SDK and long-term availability. Decide how far the product needs to go first, then decide how much performance to buy.

If you're choosing a processor for a smart home gateway, or customizing Buildroot with Rockchip's SDK, I'd be glad to talk: linktr.ee/weiyii

Cover image: Mbrickn, CC BY 4.0.

This article was prepared for DEV with AI assistance.

Matter Gateway Certification: What You Need and What It Costs

weiyi — Mon, 02 Nov 2026 05:19:14 +0000

The earlier posts were all about technology: the overall architecture, protocols, the Bluetooth chip and the main processor. But whether a gateway can actually be sold usually comes down not to technology but to certification.

In the second post, Apple Home showed an "Uncertified Accessory" warning when adding the gateway. During development you can tap "Add Anyway". A production product can't rely on that: users hesitate, and retailers simply refuse it.

This post walks through the certifications a Bluetooth Mesh to Matter gateway needs: what they are, what each covers, roughly what they cost and in what order to do them.

First, three kinds of certification

The certifications a gateway faces fall into three groups that cover completely different things:

Type	What it covers	Example	If you skip it
Ecosystem	Using the Matter name and working properly with each platform	Matter (CSA)	"Uncertified Accessory" warning, no Matter logo
Protocol	Whether the product's Bluetooth implementation complies	Bluetooth SIG	No Bluetooth trademark, many sales channels closed
Radio	Whether RF emissions meet local regulations	SRRC, CE, FCC	Legally required, no lawful sale

The first two are industry requirements; the last is the law. Most of the cost and time goes into these three.

Matter certification: only the gateway needs it

Good news first: the switches and lights behind the bridge don't need Matter certification.

In Matter's model, Apple Home sees only one Matter device, the gateway. The switches and lights behind it are bridged devices the gateway represents. So only the gateway is certified, and the devices' chips and protocols have nothing to do with Matter certification. That was one of the main reasons for choosing a gateway in the first post.

Matter certification is run by the Connectivity Standards Alliance (CSA). Roughly:

Join the CSA as a member.
Apply for a Vendor ID (VID) and assign Product IDs (PIDs).
Test at an authorized lab.
After passing, pay the certification fee; the product is listed in the CSA's certification database.
In production, flash a Device Attestation Certificate (DAC) into every unit, bought from a certificate authority.

CSA membership and certification fees

These are the prices currently listed on the CSA website (check there for the latest):

Membership	Annual fee	Certification fee per product	Notes
Associate	Free	Can't test directly; certification transfer only, US$2,500 per product plus US$500 a year	Only rebrands already certified products
Adopter	US$7,500	US$3,000 (US$2,500 for your own derivatives)	The lowest level for certifying your own product
Participant	US$21,500	US$2,000 (US$1,500 for derivatives)	Voting rights; needed to transfer certification for OEM customers
Promoter	US$100,000+	US$2,000	Large companies

Many articles still quote US$7,000 for Adopter and US$20,000 for Participant; those are prices from around 2023.

For a company starting out with Matter, Adopter is the usual choice.

Lab testing

Testing must be done at a CSA-authorized lab. Labs in China quote roughly 30,000–50,000 yuan (about US$4,000–7,000) per product, depending on product type and test scope. Bridges need more testing than single devices, for behavior such as adding and removing bridged devices at runtime.

Device certificates (DAC)

When a Matter device is commissioned, the phone checks its device certificate to confirm it comes from a legitimate manufacturer. During development the official Matter test certificates are used, which is exactly why Apple Home warns about an uncertified accessory.

In production you buy certificates from a CSA-approved certificate authority, one per unit, priced by volume and quoted individually. Two engineering points:

Protect the private key, ideally in the chip's secure storage. That's why secure boot and OTP support matter when choosing the processor.
The production line needs a station that flashes a unique certificate and pairing code into each unit.

Bluetooth SIG qualification: unavoidable if you use Bluetooth

Any product that uses Bluetooth needs Bluetooth SIG qualification to be sold, Mesh or not. As the third post noted, going back to a proprietary protocol doesn't avoid the fee.

Fortunately, retesting usually isn't required:

If you use a stack the chip or module vendor has already qualified, without modifying its qualified parts, you can reference their qualification ID (QDID) in the SIG system, submit a declaration and pay the fee.
The declaration fee is per design; the SIG's current Adopter price is around US$10,000 (check the SIG website).
If switches, lights and wireless switches share an identical Bluetooth design (same chip, same stack version), they can be combined into one design and one fee.

Our design has two Bluetooth designs, the Silicon Labs module in the gateway and the Telink chips in the devices, each referencing its vendor's QDID.

Radio certification: legally required, no shortcuts

Radio certification covers RF emissions and is mandatory by law. By region:

Region	Certification	With a certified module
China	SRRC type approval	Usually still needed for the product; ask the certification body
EU	CE (RED directive)	Reference the module's test reports and issue a declaration of conformity
US	FCC	Reuse the module's FCC ID and test the finished product

A gateway usually has two radios: Wi-Fi and the Bluetooth module. Using certified modules without changing antennas or RF parameters avoids most RF testing. That's another reason our gateway uses an off-the-shelf Silicon Labs module instead of a custom Bluetooth RF design.

Beyond RF, electronics usually need safety and EMC certification, such as China's CCC and the LVD and EMC parts of CE. Gateways with power adapters deserve particular attention.

Does Apple Home need separate certification?

A common question: to work with Apple Home, does the gateway also need HomeKit certification?

For Matter devices, no. A Matter-certified device can be added to and controlled in Apple Home directly, without Apple's MFi process. The same goes for Google Home, Amazon Alexa and Home Assistant; that's the whole point of Matter.

Only if you want to print a "Works with Apple Home" badge on the packaging do you need to look into Apple's program.

Roughly what the first year costs

Estimated for one company, one gateway, certifying Matter as an Adopter (US dollars; exchange rates and quotes vary):

Item	Approximate cost	Notes
CSA Adopter fee	7,500	Annual
Matter certification fee	3,000	Per product
Matter lab testing	4,000–7,000	About 30,000–50,000 yuan
Bluetooth SIG declaration	~10,000	Per design, no retesting with vendor QDID; gateway and devices may each need one
Radio certification	Depends on region	Certified modules avoid most of it
Device certificates	Per unit	Ongoing in production

Counting only the gateway's Matter certification and one Bluetooth declaration, first-year fixed costs come to roughly US$25,000–30,000, excluding radio certification and device certificates. From the second year on, it's mainly the CSA fee and certification for new products.

Selling in China, Europe and the US at once raises the radio and safety costs noticeably.

Suggested order

Decide the sales regions to know which radio certifications you need.
Check certificates when choosing modules: the Bluetooth module's QDID and the original SRRC, CE and FCC certificates for the Wi-Fi and Bluetooth modules, matching model and SDK version.
Join the CSA before the hardware is frozen, get a VID and understand test requirements early, rather than finding missing features after samples are built.
Pre-test engineering samples. Many labs offer pre-testing, cheaper than formal testing and good for catching problems early.
Formal Matter testing, while preparing certificate flashing for production.
Submit the Bluetooth SIG declaration, referencing the chip vendor's QDID.
Radio and safety certification, which can run in parallel with Matter testing.

Final thoughts

Certification sounds like paperwork, but it shapes technical choices: whether to use certified modules, whether the processor has secure storage, whether switches and lights can share one Bluetooth design. Factor certification in when choosing chips and you'll save a lot of money and time later.

The prices here come from the CSA and Bluetooth SIG websites and lab quote ranges. They change often, so check the latest official prices and lab quotes before committing.

If you're preparing for Matter certification, or weighing the cost and order of Bluetooth and radio certification, I'd be glad to talk: linktr.ee/weiyii

Cover image: Grethe Spongsveen, CC BY-SA 4.0.

This article was prepared for DEV with AI assistance.

Want smarter AI? Don't rush to change models

weiyi — Mon, 02 Nov 2026 05:18:57 +0000

What frustrates me about AI is not knowing when I can rely on it.

It explains a complicated problem convincingly, then repeatedly gets a small change wrong. You've supplied the information, yet it overlooks it. It says the code is fixed; run it and the problem is still there.

The natural reaction is to blame the model and buy a more expensive one.

That might help. But I'd first check whether it received the right information, whether its working process makes sense, and how it decides its work is correct.

Leave those unresolved and a stronger model may stumble in exactly the same place.

The document is there. Why can't it find it?

Connecting AI to a knowledge base is common now. Split documents into chunks, turn them into vectors, store them in a database, retrieve a few relevant passages for a question, then ask the model to answer. That's retrieval-augmented generation, or RAG.

It sounds straightforward. The trouble often starts at “retrieve a few relevant passages.”

Ask about the specific error code TS-999, for instance, and the system returns generic troubleshooting articles. Related material, yes, but the answer is missing.

Vector search understands meaning; error codes, product models and API names also need exact matching. I'd consider combining semantic search with BM25 keyword search, merging and deduplicating the results, then using a reranker to select the useful passages.

There's a subtler problem, too: splitting a document can strip away its meaning.

Suppose a retrieved passage says, “This version does not support bulk imports.”

Which product? Which version? An obsolete release or the one we're using? Without that background, the model can easily apply the statement to the wrong thing.

Adding the title, version, section and necessary context to each passage helps. In Anthropic's experiments, contextualized chunks combined with hybrid retrieval and reranking reduced the top-20 retrieval failure rate from 5.7% to 1.9%.

That doesn't promise the same improvement for every knowledge base. What it does suggest is this: when an answer is wrong, check what information the AI actually received first.

If retrieval missed the right material, polishing the prompt won't easily rescue the answer.

A longer conversation doesn't necessarily mean it understands you better

Many people keep everything in one chat: project background, error logs, passing ideas and old plans. After dozens of turns, it feels as though the model should understand them better.

But the conversation may now contain conflicting instructions.

Last week the plan was A; yesterday it became B. An earlier direction was discussed and later abandoned. A person would need to sort that out, too.

The earlier Lost in the Middle study found that, for the models and tasks it tested, putting key information in the middle of long inputs often produced worse results than placing it at the beginning or end. That cannot simply be generalized to every current model. Still, accepting a long input and using all of it well are different abilities.

Context engineering sounds technical. In day-to-day work, a large part of it is deciding what the AI needs to see at this particular step.

For a website indexing problem, start with robots.txt, the sitemap, page responses and the webmaster dashboard. Read the relevant logs when they're needed. There's no reason to load the whole codebase and every previous conversation at the start.

Partway through a long task, leave a handoff: decisions made, files changed, checks passed and unresolved issues.

I'd rather it continue from an accurate progress note than guess our current goal from an enormous conversation history.

Compression shouldn't mean removing everything, though. Lose a critical constraint and you'll still have to redo the work.

Is paying for more thinking worthwhile?

Test-time compute studies how additional computation while answering can improve a model's results.

It might try several solutions and select among them, or generate an answer and revise it using feedback.

Research shows that, on particular tasks and under particular compute budgets, allocating this computation well can let a smaller model outperform a much larger one. How you search and how you judge the candidates both matter.

The easy mistake is confusing more thinking with more talking.

An answer twice as long isn't necessarily better. Generate five proposals and you still need a way to choose the reliable one.

Run tests for code, substitute a calculation back into the original problem, and compare factual claims with primary sources. That feedback gives the next attempt something useful to work with.

If what's missing is current pricing or recently updated API documentation, look it up first. Reasoning cannot supply a fact the model never received.

I'd prioritize speed for organizing, classifying and extracting information. For difficult reasoning or important code changes, I'd increase the reasoning budget and prepare a way to check the result.

The spending worth making is the spending that reduces errors and rework.

Give AI real results to check

I'm interested in agent harnesses: the surrounding arrangements for splitting tasks, using tools, preserving progress and checking outcomes.

In March 2026, Anthropic described an experiment separating planning, generation and evaluation. The generator built an application; the evaluator interacted with running pages, checked APIs and data, then returned specific issues for revision.

One detail deserves attention: cost.

In one game-editor example, the solo agent ran for about 20 minutes and cost $9. The full harness ran for about six hours and cost $200. It produced a more complete result, but at a much higher cost.

I wouldn't conclude that adding agents is automatically better value. The time and budgets weren't remotely equal.

The useful lesson is to define completion, inspect the result, and bring back concrete failures to fix.

Publishing a blog post, for instance, isn't proved by an upload endpoint returning success. Check that the page opens, the body is complete, the canonical URL is right and language links point to the correct versions.

Nor would I rely entirely on “take another look for problems.” Models can grade their own output generously. A second model doesn't automatically become a strict reviewer, either.

Acceptance criteria should be specific and, ideally, verifiable through tests, actual page interactions or real data. Set retry and spending limits as well, so a minor issue doesn't keep the system looping all night.

Does remembering more mean understanding you better?

I don't want to explain my writing style every time, or repeatedly remind an assistant that WeChat articles should go into drafts only.

Those are lasting preferences worth remembering.

“Don't send this one to WeChat yet,” however, applies to the current task. The system needs to understand scope, rather than cancel all future WeChat work.

Research such as Mem0 explores extracting important information from conversations, consolidating it and retrieving it later. The difficult part is that information changes and conflicts.

Alongside the content, I'd want a memory to carry its source, date, scope and whether a later decision superseded it.

Remember user preferences, but inspect live project state. Server ports, dependency versions and deployments should come from the current environment when possible, not a conversation from two weeks ago.

I'd reserve fine-tuning for relatively stable, recurring needs: fixed classifications, output formats and particular domain tasks. Frequently changing material belongs in retrieval; live state belongs in tool results. Each has a job.

I'd start with the tasks that failed

These techniques can all help. They don't all need installing at once.

I'd collect twenty or thirty real tasks and record how the AI failed.

Can't find the material? Improve retrieval. Forgets requirements late in a task? Organize context and handoffs. Has the facts but reasons badly? Consider a stronger model or more inference compute. Claims completion without finishing? Add verification.

Change one or two things at a time, then compare the same tasks: success rate, cost, waiting time and manual cleanup.

Keep some tasks out of the tuning process, too, to see whether improvements hold for unfamiliar questions. Otherwise, a better-looking prompt may simply be overfitted to a few examples.

My expectations for smarter AI are practical: something that needed five explanations now needs two; work that used to require a complete recheck now arrives with verification.

Model upgrades are worth following. There's already plenty to improve while waiting for the next one.

References:

This article was prepared for DEV with AI assistance.

Make Time to Write Down What Is Happening

weiyi — Mon, 02 Nov 2026 05:18:39 +0000

Give an idea somewhere to land

Writing things down isn't only about sharing. It helps you think a problem through. Each paragraph gives you another chance to look at an idea again.

Starting with a specific question works better than waiting for fully formed inspiration.

Find a sustainable rhythm

You don't have to publish every day, but you can keep collecting ideas. This blog will grow gradually in the same way.

This article was prepared for DEV with AI assistance.

How I’ve Improved My Blog with GPT, One Step at a Time

weiyi — Mon, 02 Nov 2026 05:17:48 +0000

My blog, weiyii.cn, has used GPT for page changes, multilingual content, search submissions and WeChat drafts.

At first, my questions were broad: “How can I optimize the website?” “Why hasn’t Baidu indexed it?”

I learned to point directly at what bothered me. Cramped typography? Change it. Tables unreadable on a phone? Fix them. Specific problems make results easier to see.

Now I generally describe the problem, GPT reads the project, changes files and runs checks, and we look at the live result.

Start with what doesn’t feel right to read

For a blog, readable articles matter most to me. Is the homepage easy to scan? Is body text too small? Are headings too heavy? Can a phone user read the tables? Ordinary details, but readers meet them first.

I tell GPT directly:

Adjust the article page’s font and size, and check mobile too. Let tables use the content width; keep horizontal scrolling when they’re too wide.

It finds the relevant pages and styles before changing them. “Optimize the UI” alone won’t necessarily produce something you like. Say what feels wrong, then open the result on desktop and phone. Adjust again if needed.

I handle PageSpeed reports similarly: give it the link, ask it to investigate, and compare the findings with the project. The report helps locate issues, but I still want to know whether loading improved and reading remained comfortable. A score is useful; people need a usable site.

Multilingual work turned out to involve more than translation

I added English, Spanish, Russian, Japanese and Arabic so more people could read the articles. Beyond the body, titles, summaries, categories, tags, menus and sharing text all needed attention. Arabic also needed right-to-left layout; swapping text alone would look odd.

Each language now has its own URL: /en/articles/…/ for English, /ja/articles/…/ for Japanese. Opening it gives you the corresponding text.

I had GPT check hreflang, x-default and canonical too. The names sound complicated, but the purpose is straightforward: tell search engines which URLs are language versions of the same article.

The latest check passed for 370 foreign-language pages and their Chinese counterparts. Batch checking saves me opening them one by one. Translation quality still needs spot checks, especially when a technical term can change a sentence’s meaning.

I’ve also asked: submitted already, so why no indexing?

Of course I want people to find the site. I asked about the Sitemap, Baidu indexing, and having GPT submit the whole site.

The easy mistake is treating a successful submission as successful indexing. I now check which step actually happened:

Platform	How I submit
IndexNow	Public page URLs
Google	Sitemap through the Search Console API
Baidu	URL submission API within the site quota

After the latest release at the time of writing, IndexNow accepted 438 site URLs and Google accepted the Sitemap. Baidu’s daily quota was exhausted, so remaining submissions had to wait.

That sounds good, but 438 accepted URLs doesn’t mean 438 indexed pages. Crawling and indexing still need checking in the search platforms. GPT can complete the submission; search engines decide what and when to index.

For ordinary blog posts, Google doesn’t offer a general per-page indexing-request API equivalent to IndexNow. I use its Sitemap API.

The more I use it, the less I say “optimize everything”

I try to state three things: the problem, expected behavior and how to verify it.

Article tables overflow on mobile. Fix them, keep horizontal scrolling, and check Chinese and English pages. Explain the cause and verification.

Or:

Submit public pages to IndexNow and record the response. Keep failures pending. Don’t call submission success indexing success.

The wording doesn’t need to be fancy. It needs enough information.

I still check live pages after “done.” A local build doesn’t prove the live cache updated; a successful site release doesn’t prove search submission succeeded. We encountered Baidu quota limits and an IndexNow historical-state validation failure. An existing full-site submission script resolved the latter. Clear status and finishing the remaining work help more than a blanket “everything succeeded.”

The best part is not starting over every time

GPT saves scattered bits of time: finding styles, checking links, organizing translations and preparing drafts. Small tasks add up.

I can now hand it a concrete problem and examine the result. Saved workflows remain useful next time. What the site should cover, which articles deserve publishing and whether a page feels comfortable are still my decisions.

Start with one small task: a mobile table, an article’s language pages or a working draft-upload process. Get it working, then add the next piece. That’s how my blog has improved.

This article was prepared for DEV with AI assistance.