DEV Community: Tran Tien Van

Agentic Loops in Marketing: Lessons From Microsoft's C++ Demo

Tran Tien Van — Tue, 21 Jul 2026 13:00:39 +0000

On July 21, 2026, Microsoft’s C++ team demonstrated agentic loops on real engineering projects with an unusually crisp judge: builds and tests.
That makes the demo useful to marketers—but only if we resist pretending Microsoft showed a marketing agent.

What the C++ loop actually proved

At Pure Virtual C++ 2026, the cycle was inspect state, modify code or configuration, build, test, evaluate, and continue. The examples covered C++ modernization, CMake builds and tests, MSVC upgrades, and build-performance optimization.

Those workflows have something agent builders crave: rapid, machine-readable feedback. A compiler can reject a change. A test can fail. A performance run can show whether an optimization moved in the intended direction. The loop does not have to infer success from applause or a vague dashboard.

The boundary is important. Microsoft did not show or endorse marketing agents, AI SDRs, autonomous ad bidding, or sales infrastructure. The transferable part is architecture, not product validation.

Marketing is missing a test harness

A developer would not call a patch complete because an agent successfully invoked the compiler. Likewise, a marketer should not call a workflow autonomous because a model can open a research tool, draft a sequence, or push content into a CMS.

Tool use is an action. A loop requires evaluation.

Marketing evaluation is harder because signals can be delayed, disputed, or easy to game. A campaign might increase an intermediate metric without producing the business outcome the team intended. Attribution may be unresolved when the next action is due. A quality failure can appear before the outcome signal arrives.

The practical question is therefore not “Which model can operate our stack?” It is “What is our equivalent of a passing build, and what evidence can invalidate it?”

The minimum control surface

Before granting an agent authority, define the contract around its loop. A workable design needs:

A business outcome signal that can tell the loop whether to continue, fail, or stop.
Independent quality gates, so a favorable outcome proxy cannot excuse broken content or unsafe behavior.
Bounded permissions that limit which systems and actions the agent can reach.
Complete action logs that let an operator reconstruct what happened.
Human approval before outward-facing actions.
A tested recovery path for undoing or containing a bad step.
Written success, invalidation, attribution, and stop conditions.

This is more than governance paperwork. Each item becomes part of the runtime. Permissions constrain possible actions; gates reject bad states; logs preserve evidence; approval creates a deliberate boundary; recovery makes failure survivable.

If the team cannot specify those elements, the honest operating mode is recommendation. The agent can inspect, analyze, and propose. A person remains responsible for executing the outward action.

Recommendation mode is not failure

For developers, this is similar to an agent preparing a patch and test results without merging. You still gain research, synthesis, and drafting speed. You simply retain the authority boundary until the evaluator is dependable.

The cost is obvious: human review adds latency, and a manual checkpoint can become a queue. Teams may also overcorrect by requiring approval for every harmless internal step, removing much of the benefit of a loop.

Full autonomy has the inverse tradeoff. It reduces waiting, but it magnifies a weak signal. If the evaluator rewards the wrong proxy, the agent can repeat the wrong behavior more efficiently. Fast iteration is only valuable when the feedback function deserves that influence.

A useful compromise is to separate internal and external actions. Let the agent research, compare, calculate, or assemble drafts inside a narrow workspace. Require approval when it publishes, sends, spends, or changes a customer-visible system. Expand authority only after logs and recovery tests show that the boundary is working.

How this maps to content operations

Vanaxity applies the loop pattern to research, writing, illustration, publishing, and syndication. SEO, GEO, and AEO checks run together with review gates.

That separation matters. Search optimization is not identical to factual quality, editorial quality, business impact, or permission to publish. Treating search checks as permission to publish would collapse distinct decisions.

At Van Data Team, the design order begins with the signal and the authority map, then moves to model selection. This reverses a common procurement instinct, but it matches the lesson from the C++ example: the model participates in the loop; the evaluator and boundaries make the loop operable.

A developer’s test for marketing autonomy

Treat the proposed workflow like a system you may have to debug at 2 a.m. Can you identify the current state? Can you reproduce every action? Can you explain which gate passed? Can you stop execution without losing evidence? Can you recover from the last outward step?

If those answers are unclear, another prompt or a larger model will not create the missing control plane. Keep the workflow advisory, instrument it, and make the feedback contract explicit.

What marketing signal would you be willing to encode as a pass/fail test, and what failure case would make you distrust it?

📖 Read the full guide → Agentic Loops in Marketing: What Microsoft's C++ Demo Teaches

GPT-5.6 on Amazon Bedrock: A Practical Agent Routing Guide

Tran Tien Van — Tue, 21 Jul 2026 10:03:04 +0000

GPT-5.6 became generally available on Amazon Bedrock in three tiers—Sol, Terra, and Luna—through the Responses API in AWS's July 20, 2026 Weekly Roundup.

The useful engineering question is not "Which tier wins?" It is "Which tier belongs at each step, under which controls?"

Route work instead of crowning one model

AWS positions Sol for flagship reasoning, Terra for balanced performance, and Luna for fast, cost-efficient inference. Those descriptions are starting points for a routing policy, not a mandate to run an entire agent workflow on one option.

An agent rarely performs one uniform job. It may interpret a request, plan work, retrieve context, call tools, check output, and ask for approval. Treating all of those steps as equivalent hides the tradeoffs that the three tiers expose.

A practical default is to reserve Sol candidates for steps where reasoning quality carries the most weight, evaluate Terra as the balanced path, and test Luna where speed and inference cost matter more. These are hypotheses, not universal assignments. Your own evaluations should decide the route.

Build the first routing matrix

Keep the initial implementation small enough to inspect. A useful sequence is:

Split the workflow into named steps with a clear input, output, and owner.
Give each step an initial Sol, Terra, or Luna assignment based on AWS's stated positioning.
Run the same representative tasks through viable tiers and record quality, speed, and cost observations.
Mark repeated context explicitly, then decide where cache breakpoints belong.
Document every tool permission and identify the changes that require human approval.

Version that matrix alongside prompts and evaluations. When a tier assignment changes, reviewers should be able to see which evidence justified it and which downstream steps could be affected.

This also makes fallback behavior discussable. A fallback should not silently gain broader tool permissions or skip a review gate merely because the preferred route is unavailable.

Read the 90% cache figure carefully

Explicit cache breakpoints can make repeated context eligible for a 90% billing discount. The important words are "repeated context," "eligible," and "breakpoints."

That figure is not a promise that the complete agent run will cost 90% less. Total workflow cost still depends on agent behavior, including what context is repeated and how the workflow moves across steps. Cache design therefore belongs in the workflow specification, not in a late infrastructure cleanup.

AWS also says Bedrock pricing matches OpenAI first-party rates and that usage counts toward AWS commitments. Those procurement details may affect the buying decision, but they do not remove the need to observe cost at the step level.

Keep setup separate from production readiness

AWS's separate Lambda item provides a copy-paste prompt that configures supported coding agents with AWS Serverless skills and the Serverless MCP server. That can reduce setup friction. It does not approve, deploy, or harden a production system.

A successful configuration should begin the operational review, not end it. Define which tools may read data, which may change state, and where a person must approve the action. Add evaluations, review gates, telemetry, and recovery paths before treating the workflow as production-ready.

This distinction matters because a fast bootstrap can look deceptively complete. Configuration proves that components can be connected; it does not prove that their permissions and failure modes are acceptable.

Treat protocol support as contract work

The roundup showed MCP, A2A, UTCP, AG-UI, and x402 working together through the Strands Agents SDK. That is an example implementation, not a bundled agent product.

Open protocols can reduce framework coupling, but only when the team governs schemas, permissions, versions, errors, and audit data. Without that work, the coupling has not disappeared; it has moved into undocumented assumptions between components.

Give each interface an owner. Specify version compatibility, rejected requests, authorization boundaries, error handling, and the audit fields needed to reconstruct an agent action. Portability becomes credible when another implementation can honor those contracts without inheriting hidden behavior.

Make the go/no-go decision observable

Before choosing a default stack, run a controlled workflow through the candidate routes. Compare the tiers on the tasks they might actually own. Test repeated context with deliberate cache breakpoints. Exercise denied tool calls, human approval paths, protocol errors, and recovery behavior.

The ordering is the point: map operating decisions first, then select the stack. Model access is useful, but production confidence comes from explicit routing, permissions, evaluations, review gates, telemetry, and recovery.

Which step in your current agent workflow would you route to a different tier first, and what result would justify the change?

📖 Read the full guide → GPT-5.6 on Amazon Bedrock: What Agent Teams Should Do Next

Salesforce Agentforce Commerce: A Developer Readiness Guide

Tran Tien Van — Mon, 20 Jul 2026 15:16:36 +0000

Three commerce agents—Shopper Agent, Buyer Agent, and Merchant Agent—are generally available, while ChatGPT catalog selling reaches GA in July 2026. For developers, the hard part is proving that synchronized product data produces recommendations people can trust.

Treat catalog synchronization as an interface

Retailers can synchronize catalogs from Business Manager to ChatGPT without extra software or third-party middleware. That is a meaningful simplification, but direct synchronization removes middleware, not the need for product-data governance.

The useful engineering model is an interface contract. Before exposing a catalog to an AI shopping channel, document where each product value originates, who owns it, and which review gate can stop publication. This mirrors Van Data Team's starting point: map product sources, field ownership, and review gates.

A successful transfer only proves that data moved between systems. It does not prove that an agent selected the right item, supported its answer with useful evidence, or respected the availability of a requested variant. Those are different failure modes and should be observable separately.

Test what the shopper will experience

The release includes Agentic Commerce Search, which uses inferred intent and is designed for both Salesforce and non-Salesforce storefronts. That makes test design more important than a simple exact-name lookup.

Build test cases around realistic shopping intent and compare the output with the product records your team considers authoritative. Keep the acceptance criteria concrete:

Recommendation accuracy: Did the response select products that fit the stated intent, and can a reviewer trace the choice to current catalog data?
Evidence quality: Does the answer use product information that supports the recommendation instead of merely naming an item?
Variant availability: Is the recommended size, color, or other requested variant actually available in the catalog at review time?
Storefront coverage: If your implementation spans Salesforce and non-Salesforce storefronts, does the same review method work across both?

The first three checks come directly from the operational risk: AI visibility is not a win when recommendations are inaccurate, poorly supported, or attached to unavailable variants. The fourth keeps the validation approach aligned with the product's stated storefront scope.

For each failed test, capture the source record, the responsible field owner, and the review decision. That turns an incorrect answer into a fixable data or workflow problem instead of a vague "AI quality" issue.

Use the release stages as rollout boundaries

ChatGPT catalog selling is GA in July 2026. Google Search, including AI Mode, and the Gemini app are scheduled for later this summer. Treat that sequence as a reason to stage readiness work, not as permission to assume one channel's behavior will predict another's.

Start with shared catalog controls: source mapping, ownership, validation, publishing, monitoring, and failure recovery. Then keep channel-specific tests separate. The reusable layer should establish whether a claim and variant are dependable; the channel layer should show how each destination represents them.

This is also why a single "catalog synced" status is too coarse. A useful dashboard should distinguish transfer success from review status and observed representation. Vanaxity follows that separation across SEO, GEO, and AEO: create structured content, validate claims, publish across channels, and monitor how search and answer engines represent it.

Be explicit about the tradeoffs

Direct synchronization reduces the integration path, which can make distribution simpler. The same shorter path can also expose weak catalog controls sooner. More review gates add operational work; fewer gates leave less opportunity to catch a questionable claim or unavailable variant before an agent presents it.

Inferred-intent search also changes what a useful test looks like. Exact product names are still valid inputs, but they do not cover a mechanism designed to infer intent. A broader test set takes more effort to maintain, yet it gives teams a clearer view of recommendation behavior.

The stated design for non-Salesforce storefronts widens the intended scope. It also makes source ownership worth documenting explicitly, because that reach does not remove differences in how a retailer manages its own product data.

Define readiness before visibility

A practical readiness gate can be simple: every exposed product source is known, every critical field has an owner, every publish path has a review decision, and the three shopper-facing checks have passed. Monitoring should then verify how the product is represented after publication, with a route for correcting failures.

That definition keeps the milestone honest. Distribution is complete when a catalog reaches a channel. Readiness is demonstrated when the resulting recommendation remains accurate, evidenced, and available.

If you were implementing Salesforce Agentforce Commerce, which check would you automate first—recommendation accuracy, evidence quality, or variant availability—and what exact result would fail the build?

📖 Read the full guide → Salesforce Agentforce Commerce: The GEO Playbook for AI Shopping

Amazon Bedrock AgentCore Observability: Verify Per-Agent CloudWatch Logs and Traces

Tran Tien Van — Mon, 20 Jul 2026 09:42:27 +0000

CloudWatch Transaction Search must be enabled before Amazon Bedrock AgentCore spans and traces become searchable, and first-time users may need to wait about ten minutes for those spans to appear.

That requirement should shape your observability rollout: prove the telemetry path before relying on it during an investigation.

Separate confirmed behavior from proposed defaults

Do not design around a platform default until you have verified it.

AWS documentation does not currently guarantee dedicated, per-agent log groups for all new AgentCore agents. Check the AgentCore release notes for explicit release-level confirmation, then compare them with the configuration visible in the target environment.

Dedicated isolation can narrow the debugging boundary and make telemetry ownership clearer. Neither benefit is dependable if the team merely assumes the isolation was created.

If separate groups are part of your operating model, verify or configure them explicitly. Record what the deployment produced instead of treating an uncited behavior as a contract.

Verify both telemetry destinations

AWS documents separate destinations for agent runtime logs and trace spans. Seeing runtime messages therefore does not prove that a workflow's traces are available for search.

Create a deployment verification artifact that records the evidence an operator will need later:

Capture the actual destination used for agent runtime logs.
Capture the destination configured for trace spans.
Confirm that CloudWatch Transaction Search has been enabled for first-time use.
Allow for the documented delay of about ten minutes before judging span searchability.
Run a known workflow and verify that its resulting trace can be found.
Record the owner responsible for resolving any gap before production use.

Use this AWS CLI command to check the CloudWatch log group name and ARN against the prefix shown in your deployment configuration:

aws logs describe-log-groups --log-group-name-prefix '<prefix-from-deployment-configuration>' --query 'logGroups[].{LogGroup:logGroupName,ARN:arn}' --output table

Record the returned log group name and ARN in the deployment verification artifact. This turns an environment assumption into evidence that can travel with the deployment record and give the next investigator a known starting point.

Debug the workflow, not only the answer

An autonomous workflow may plan, choose a tool, retry an action, call another service, or pause for human review. A final response alone cannot show which stage produced the wrong outcome.

Start by mapping workflow states, tool permissions, review gates, and recovery paths. Pair that map with the AWS CLI check above so the deployment record contains the concrete runtime log destination rather than only a proposed architecture.

The useful unit of investigation is the path the agent took. Review runtime logs alongside trace spans when reconstructing that path. Because AWS documents different destinations for those signals, both paths deserve an explicit check.

Use per-agent boundaries without losing fleet visibility

Per-agent isolation can narrow where a team looks and clarify which owner should respond. The tradeoff appears when the same organization needs to reason across many agents.

Fleet-wide queries and policies do not emerge automatically from isolated telemetry. Teams still need a deliberate design for cross-agent investigation, retention, sampling, failure alerts, and escalation rules.

A boundary that helps one owner can leave the fleet harder to govern if no shared operating layer exists. Design local ownership and fleet oversight together rather than treating them as competing choices.

Make trace-based SLOs operational

AgentCore telemetry can provide evidence, but trace-based SLOs remain the operator's responsibility. The same is true for failure alerts, retention, sampling, and escalation.

Build the SLO framework around the workflow states that matter. Decide which trace evidence demonstrates successful progress, which failures should alert an owner, what telemetry must be retained, how sampling will be governed, and when an investigation escalates.

Per-agent separation can make a single workflow easier to inspect, while the policies needed to keep many workflows consistent require additional design. AgentCore supplies observability mechanisms; it does not supply the team's operating decisions.

Ship evidence with the deployment

A production-ready observability package should include an architecture, a deployment verification artifact, investigation patterns, an SLO framework, and a production checklist. Together, those outputs connect configuration with the way people will investigate and govern the workflow.

Keep the release gate concrete. Check the actual log and span destinations. Enable Transaction Search. Wait for a known span to become searchable. If dedicated per-agent groups are required, verify them against configuration and authoritative release documentation.

Which proof do you require before promoting an AgentCore workflow: confirmed destinations, a searchable known trace, or both?

📖 Read the full guide → Amazon Bedrock AgentCore Observability: Per-Agent Logs and Traces

Why $10B Meta-Anthropic talks matter: Measuring your agent’s true cost-per-task

Tran Tien Van — Sun, 19 Jul 2026 13:19:28 +0000

CNBC reported that Meta Platforms and Anthropic are in early, preliminary talks; Quartz reported that the proposed arrangement could reach $10 billion over roughly two years, with Anthropic paying Meta monthly. The arrangement may never close, but for developers running continuous agents, it is a prompt to inspect the cost and failure semantics of every task.

Start with the right denominator

A continuous-agent workflow incurs cost when it retries, calls a tool, enters review, or needs remediation; it also depends on capacity being available whenever work arrives. If affordable compute is unavailable, the agent cannot reliably run campaigns, route leads, or publish useful content.

Seat price tells you what access costs. Token price tells you what model consumption costs. Neither tells you what an accepted result costs after failed attempts and downstream cleanup.

A more useful operating metric is:

cost per successful task = model, tool, retry, review, and remediation cost / accepted tasks

Define success with acceptance criteria, not with a completed API response. Measure the numerator and denominator over the same window, then segment by task class. Lead routing and content publishing should not share a target simply because they happen to share infrastructure.

Put controls in the execution path

The practical controls are small, but they need to be explicit:

Batch compatible work when latency allows it, so throughput does not depend on one request at a time.
Route each task class to a suitable model instead of assigning every job to one default.
Run an early quality gate before expensive tools or downstream publishing.
Cap retries, record the reason for failure, and escalate to a person when the bound is reached.
Keep orchestration portable and exercise fallback routes before capacity becomes constrained.

These controls reinforce one another. A gate blocks unusable output before it triggers more work. A retry limit prevents a weak route from consuming compute indefinitely. Human escalation gives ambiguous or repeatedly failing work a deliberate exit.

Instrument every run with the task class, selected route, attempt count, tools used, gate result, escalation status, final acceptance result, and associated costs. For example, a Task Event can use placeholder values like these:

{
  "task_class": "<task class>",
  "selected_route": "<selected route>",
  "attempt_count": "<recorded count>",
  "tools_used": ["<tool name>"],
  "gate_result": "<pass or fail>",
  "escalation_status": "<escalation status>",
  "final_acceptance_result": "<accepted or rejected>",
  "costs": {
    "model": "<recorded cost>",
    "tools": "<recorded cost>",
    "review": "<recorded cost>",
    "remediation": "<recorded cost>"
  }
}

That record lets you explain why two tasks with similar token totals produced very different outcome costs.

Separate routing policy from vendor choice

Task-based routing works best when policy is independent of a specific provider. The policy decides what quality, capacity, and fallback conditions a task needs. Provider adapters execute that decision.

Portability has a cost. An abstraction layer requires maintenance, and vendor-specific capabilities may still justify a specialized route. The goal is not to pretend all models behave alike. It is to prevent one unavailable route from silently stopping every eligible workload.

A tested fallback is also more than a second API credential. It needs the same acceptance gate as the primary route. If the fallback cannot meet that gate, fail visibly and escalate rather than quietly accepting a weaker result.

Treat quality as a budget control

Quality review is often placed at the end of a workflow. Moving a gate earlier can keep low-quality output from accumulating tool calls, review time, and remediation.

Research, writing, illustration, publishing, and syndication may be automated, but success still depends on whether the result meets its acceptance criteria.

This changes the developer question from “Did the agent finish?” to “Did the workflow produce an accepted result at a defensible cost?”

Keep the tradeoffs visible

Batching can protect throughput while increasing wait time for an individual task. Stricter gates can lower accepted volume or increase escalation. Portable orchestration adds adapter and test work. A fallback can produce different output characteristics that need validation.

Those costs should be visible beside model spend. Cost per successful task is not a replacement for capacity and reliability measures; it connects them to the outcome the system was meant to deliver.

The proposed arrangement is uncertain, but the operating problem already exists. Where in your agent pipeline would an early quality gate remove the most retry or remediation cost?

📖 Read the full guide → Meta Anthropic Compute Talks Reveal AI Agents' Real Cost

Beyond the Hypervisor: How Azure Cobalt 200 Changes Multi-Tenant Isolation

Tran Tien Van — Sun, 19 Jul 2026 09:53:55 +0000

Azure Cobalt 200 puts Rowhammer protection in Microsoft's custom memory controller, close to DRAM.

That should get a platform engineer's attention. Rowhammer is not a bad IAM policy, a noisy neighbor CPU spike, or a container escape in the usual sense. It is a physical memory-integrity attack that can corrupt data outside the attacker's allocation.

That is the part that keeps cloud teams up at night: your YAML can be clean, your policies can be correct, and your hypervisor assumptions can still depend on physical memory behaving the way the software stack expects.

At Van Data Team, we make that problem operational. This guide turns Cobalt 200 into a reusable pre-production review: what changed, what did not, and what evidence should close the gap before a sensitive multi-tenant workload gets approved.

Why developers should care

Most application teams think about multi-tenant isolation at the layers they directly configure: namespaces, containers, hypervisor settings, network policy, workload identity, encryption settings, and deployment permissions.

Cobalt 200 is a reminder that one important boundary sits beneath those controls. If bit flips in physical DRAM can undermine the assumptions made by the hypervisor or container runtime, then the software policy may still be correct while the isolation guarantee is weaker than the review suggests.

Microsoft's design moves the Rowhammer control into a custom memory controller. That placement matters because the memory-access behavior behind the threat can be observed and managed close to DRAM, rather than treated only as a software concern after the fact.

Do not flatten every control into one bucket

The practical mistake is to hear "hardware hardening" and treat it as a substitute for everything above it.

It is not.

Rowhammer mitigation, memory encryption, confidential computing, and hypervisor isolation address different threats. None replaces the others.

A useful control map separates them like this:

Control	What it helps address
Rowhammer mitigation	Physical DRAM disturbance effects that can corrupt memory outside an allocation.
Memory encryption	Protection for memory contents under a different threat model. In Cobalt 200, memory encryption is enabled by default.
Arm CCA support	Confidential computing scenarios where isolation and attestation matter.
Hypervisor isolation	Tenant execution boundaries at the virtualization layer.

If you collapse those into one checkbox called "secure hardware," your review loses the thing it is supposed to protect: a clear mapping from threat to enforcement layer.

What changes in a pre-production review

For developers and platform engineers, the buyer question should move from "does this processor have a feature?" to "is this exact service configuration using the control, and can we see the evidence path?"

That distinction matters. A processor capability in the abstract does not prove your workload runs with the expected setting, in the expected region or service family, under the expected operational model.

A stronger review asks for the exact service configuration, the evidence that supports it, the review gate that approves it, the monitoring signal that would show drift, and the fallback if the assumption fails.

That is what zero trust architecture at the hardware layer means in practice. It does not mean pretending hardware is untrusted in a vague philosophical sense. It means naming the unavoidable dependency, demanding suitable evidence, and planning for failure.

Honest tradeoffs

Cobalt 200's Rowhammer design strengthens the cloud trust boundary by placing protection below the software boundaries many teams audit. That is a real improvement for multi-tenant isolation.

But it also raises the standard for procurement and architecture reviews. You now need people who can read across hardware controls, confidential computing, attestation, hypervisor behavior, and service configuration. That is harder than asking whether encryption is on.

The risk is not that hardware controls are bad. The risk is overclaiming what they prove.

If your review says "Cobalt 200 supports Rowhammer protection, memory encryption, and Arm CCA," that is only the beginning. The production question is whether the workload's approved environment actually uses the required guarantees and whether your team knows what evidence closes the gap.

A review checklist I would use

Before approving a sensitive multi-tenant workload, I would ask:

Which isolation guarantee are we trying to prove?
Which layer enforces it: memory controller, encryption mechanism, confidential computing boundary, hypervisor, container runtime, or application policy?
What evidence confirms the exact service configuration?
What signal would tell us the assumption changed?
What fallback exists if the provider evidence is incomplete?

That checklist keeps the Cobalt 200 discussion practical. The point is not to admire the processor feature. The point is to decide whether the service boundary is strong enough for the workload.

For teams building on cloud infrastructure, this is the useful shift: multi-tenant isolation is no longer reviewed only where your YAML, policies, and runtime settings live. The physical memory layer belongs in the threat model too.

About the review

Reviewed by Tran Tien Van, Founder, Van Data Team.

What evidence would you require before approving a Cobalt 200-backed service for a sensitive multi-tenant workload?

📖 Read the full guide → Multi-Tenant Isolation: What Azure Cobalt 200 Changes

Stop Measuring AI Tokens—Start Measuring Successful Tasks

Tran Tien Van — Sat, 18 Jul 2026 15:17:21 +0000

On July 17, 2026, Fortune reported OpenAI CFO Sarah Friar's proposed lens: "useful intelligence per dollar." For developer teams shipping agents, that is the measurement shift that matters: stop treating token usage as the main ROI signal, and start asking whether the agent produced work a reviewer accepted.

Cost per successful task is the metric I would put next to every agent run in CI.

If tokens go down while accepted work stays flat, you optimized inference, not the workflow.

Why tokens are the wrong denominator

Token cost is useful for capacity planning. Seat licenses are useful for access control. Neither tells you whether the agent produced dependable output.

That distinction matters for developer teams building automated SDRs, copy agents, media-buying orchestrators, or content pipelines like Vanaxity, Van Data Team's review-gated SEO, GEO, and AEO agent. These systems can create visible activity all day: sent messages, draft pages, generated variants, scheduled bids, and handoff artifacts.

Activity is not success.

A low token bill can still hide an expensive workflow if humans reject the output, rerun prompts, patch data errors, clean up unsupported claims, or rebuild the artifact downstream.

Start with accepted tasks

The important shift behind useful intelligence per dollar is not a new dashboard widget. It is moving measurement from model consumption toward dependable completed work.

For an engineering team, I would translate that into one rule:

Only count a task after it clears a stable quality bar.

That bar should be explicit enough that two reviewers can apply it the same way. For a code agent, maybe the patch builds, tests pass, and review comments are addressed. For an SDR agent, the message might need compliant targeting, correct company context, and no fabricated personalization. For an AI copywriter, accepted work could mean the draft matches the brief, uses sourced facts, and needs only normal editing.

Those marketing and sales examples are my application of the accounting pattern, not an OpenAI recommendation. The broader principle is simple: measure completed work against the full cost required to get it accepted.

Charge the whole path

The simple formula:

cost_per_successful_task = full_workflow_cost / accepted_tasks

The hard part is the numerator.

Full workflow cost should include more than inference. Count model spend, orchestration, software licenses, retrieval or data services, human review, failed runs, retries, rework, and cleanup. If the same task takes multiple attempts and a reviewer pass before acceptance, all of that belongs to the accepted task.

A concrete tracking chain helps:

Usage: tokens, compute, seats, API calls.
Attempts: every agent run that tried to complete a task.
Completions: tasks the agent claimed were done.
Accepted work: completions that passed the quality gate.
Business value: verified revenue, savings, risk reduction, or other measured impact.

Those are separate because they answer separate questions. Token cost asks how expensive inference was. Attempts ask how busy the automation was. Accepted work asks whether the system produced usable output. Business value asks whether the output mattered.

Build it into the agent

Do not leave this as a finance exercise at the end of the month. Put task accounting into the workflow.

Emit a task id when the agent starts. Record each attempt, reviewer decision, retry cause, and acceptance status. Store the quality bar version with the result, because definitions drift. Capture human time in a coarse but consistent way. Keep rejected work visible instead of dropping it from the report.

A minimal event shape can be enough to start:

{
  "task_id": "uuid",
  "task_type": "agent_task_name",
  "quality_bar": "review_gate_name",
  "attempt_id": "uuid",
  "attempt_status": "accepted|rejected|retry",
  "reviewer_decision": "accepted|changes_requested|rejected",
  "retry_cause": "missing_context|quality_gap|policy_issue|null",
  "costs": {
    "model": "decimal",
    "tools": "decimal",
    "human_review": "decimal",
    "rework": "decimal"
  }
}

For review-gated content systems like Vanaxity, the same model applies across research, writing, illustration, publishing, and syndication. The accepted blog post is not just the final draft. It is the cost of every stage required to get quality-approved work through the pipeline.

Tradeoffs worth accepting

This metric is slower than watching a token dashboard. It also creates uncomfortable visibility. A team may discover that the cheapest model produces more rejected work, or that a high-quality agent is bottlenecked by review policy rather than generation.

There is also a risk of overfitting the quality gate. If acceptance rules become too narrow, the agent learns to satisfy the checklist instead of the business need. If the rules are too loose, cost per successful task becomes vanity math.

The answer is to report two layers honestly. Use production efficiency when you can verify accepted output but not downstream value yet. Use business ROI only when there is verified value attached to the work.

The developer takeaway

If you are shipping agents, do not stop at "the run succeeded." Add the accounting chain beside your logs.

A practical first version can be small: define one task type, one acceptance rule, one reviewer outcome, and one full-cost calculation. Then compare models, prompts, tools, and agent designs by accepted work per dollar instead of usage per dollar.

What is the toughest part to instrument in your agent stack today: human review time, retry causes, or accepted-task quality gates?

📖 Read the full guide → AI Agent ROI: How to Measure Cost per Successful Task

Conversational AI platform evaluation beyond Gartner charts

Tran Tien Van — Sat, 18 Jul 2026 07:11:13 +0000

July 7, 2026 gave buyers two different Gartner artifacts to read: the Magic Quadrant and Critical Capabilities.

Treating them as one verdict is where a conversational AI platform evaluation starts to drift.

Separate the signal from the test

The Magic Quadrant evaluates market position through Completeness of Vision and Ability to Execute. Critical Capabilities is a separate use-case-oriented assessment. That distinction is not analyst trivia. It changes what you should do next.

A Leader placement can help a team decide who belongs on the shortlist. It cannot tell you whether a platform fits your customer journeys, permissions model, compliance obligations, voice path, integrations, cost model, or operational controls.

Google Cloud published its announcement on July 17, 2026. The underlying Gartner reports are dated July 7, 2026. The post says Google was named a Leader for the second consecutive year and says Google ranked #1 in three of four use cases. The important detail for practitioners: that #1 use-case ranking belongs to Critical Capabilities, not the Magic Quadrant.

That does not make the ranking useless. It makes it contextual.

Why developers should care

Developers often inherit the evaluation after the shortlist is already politically warm. A vendor has a chart placement, an executive has seen a demo, and the engineering team is asked whether integration is feasible.

That is too late to discover that the agent cannot be paused safely, that tool access is too broad, that audit trails are incomplete, or that human escalation does not match the way support actually works.

A better evaluation treats analyst coverage as input, then forces every candidate through the same gates. The article names the ones that matter for production approval: grounded answers, least-privilege tools, audit trails, monitoring, human escalation, pause controls, rollback, and retained ownership.

Those are engineering concerns, not procurement decoration.

A practical evaluation shape

For a developer or platform team, I would structure the work around one frozen rubric before pilots begin. Do not let each vendor optimize for a different demo path.

Concrete checks to include:

Report separation: read the Magic Quadrant for market position, then read Critical Capabilities for use-case orientation. Do not mix the two into a single winner narrative.
Vendor normalization: test Google, Salesforce, SoundHound AI, Kore.ai, or any other candidate against the same security, compliance, integration, handoff, cost, and operability gates.
Control readiness: require least-privilege tools, audit trails, monitoring, escalation, pause controls, rollback, and retained ownership before production approval.
Workflow fit: validate how the agent retrieves enterprise knowledge, interprets intent, acts across systems, and hands off when confidence or authority runs out.

The named vendors matter because the market is real. CX Foundation identifies four Leaders: Google, Salesforce, SoundHound AI, and Kore.ai. But the list is still a starting point, not the operating model.

The tradeoff

There is a real benefit to using analyst research. It compresses a noisy market and gives teams a shared vocabulary. Completeness of Vision and Ability to Execute are useful lenses when you are trying to understand category maturity.

The tradeoff is that market position can feel more objective than it is for your environment. Your support workflows, data boundaries, compliance duties, and escalation paths are not visible from a quadrant.

Critical Capabilities helps because it is more use-case oriented. But even that does not replace buyer-run validation. A use-case ranking does not prove the agent will behave correctly against your permissions, your knowledge sources, your failure modes, or your production controls.

The main implementation lesson

The broader market is moving from scripted chat toward agents that interpret intent, retrieve enterprise knowledge, and act across systems. That shift raises the cost of a shallow evaluation.

A scripted bot can fail by giving a bad answer. An agentic system can fail by taking the wrong action, using the wrong source, skipping escalation, or operating beyond the ownership model the business thought it had retained.

That is why conversational AI platform evaluation should feel closer to production readiness review than software category shopping. Analyst placement can open the door. The gates decide whether the platform gets to meet customers.

What would you put first in a developer-owned conversational AI platform evaluation: tool permissions, auditability, escalation behavior, or rollback?

📖 Read the full guide → Conversational AI platform evaluation beyond Gartner

on-device AI agents: what developers should publish now

Tran Tien Van — Fri, 17 Jul 2026 12:05:09 +0000

July 17, 2026 matters here because Samsung Newsroom Australia's AI Appreciation Day piece gave marketers a useful signal without giving them a new API.

For developers building brand sites, content systems, docs, and product data surfaces, that distinction changes the job. The practical task is not to target private phone context. The supplied evidence identifies no marketer access to that context. The task is to make public brand facts clean enough that an assistant can retrieve them when it needs reliable information.

Treat the device boundary as real

Van Data Team's analysis is that on-device AI agents could become proactive discovery surfaces. The important mechanism is specific: a proactive assistant could surface a likely next step before the user opens a search box, website, or brand app.

That is a different discovery path than search ranking. Search starts after a query. A device-level suggestion may happen before the query exists.

The privacy tradeoff is just as important. The article does not identify a way for marketers to inspect or target the private context inside the phone. If the assistant uses private context to decide what might help next, the brand side of the system still has to live outside that private boundary.

So the developer-facing question becomes: what can your public information layer say clearly, consistently, and extractably?

Build for agent-legible facts

Agent-legible does not mean stuffing pages with vague AI copy. The article names the kinds of facts that matter: entity, offer, availability, and policy facts.

A useful implementation pass is boring in the best way:

Define the entity facts once: brand name, product names, service categories, locations served, and ownership relationships where relevant.
Keep offer facts current: what is available, who it is for, what it does, and what limits apply.
Make availability explicit: regions, service scope, plan access, support coverage, and any conditions that change the answer.
Publish policy facts in extractable form: privacy terms, cancellation rules, data handling language, guarantees, restrictions, and eligibility requirements.
Reduce contradictions across pages: if the homepage, pricing page, help center, and blog disagree, an answer system has to choose between them.

None of this requires claiming access to a device agent. It is information hygiene for a world where retrieval may happen outside your funnel.

GEO and AEO are not enough by themselves

The article is careful about this point: GEO and AEO improve answer readiness, but neither guarantees inclusion in a device-level suggestion.

That matters for implementation planning. You can make content easier to interpret, cite, and retrieve. You cannot honestly promise that a Samsung, Galaxy AI, or other on-device assistant will surface the brand.

That is also why measurement gets harder. The supplied evidence identifies no public Samsung brand-agent interface or device-agent attribution system. If the discovery moment happens before a search and inside a protected consumer AI flow, your analytics may only see downstream behavior.

Developers should expect incomplete attribution, delayed signals, and more reliance on first-party relationship points that users choose to enter.

Keep product claims separate from channel analysis

The article also draws a line around Vanaxity. Vanaxity is Van Data Team's AI content agent for SEO, GEO, and AEO. It structures and publishes answer-ready content, but it does not integrate with Samsung, Galaxy AI, or another device agent, and it does not process personal device data.

That boundary is worth copying in your own systems. If your product helps teams publish better facts, say that. If it does not control device suggestions, do not imply that it does.

Developers can help marketing teams avoid accidental overclaiming by encoding the distinction into templates, metadata fields, review checklists, and release notes.

The honest engineering tradeoff

The upside is clear: better public facts improve readiness across search, answer engines, and possible proactive surfaces.

The tradeoff is that the most interesting trigger may remain invisible. You can improve the public supply of trustworthy information, but you may not see the private demand signal that caused an assistant to look for it.

That makes boring correctness more valuable than clever tracking.

If you were preparing a brand site for on-device AI agents, which fact group would you clean up first: entity, offer, availability, or policy?

📖 Read the full guide → On-Device AI Agents and Privacy-First Brand Discovery

AI coding assistant migration after Gemini Code Assist shutdown

Tran Tien Van — Fri, 17 Jul 2026 09:41:38 +0000

June 18, 2026 was the first operational warning sign: new consumer installations of the Gemini Code Assist GitHub app stopped. On July 17, 2026, that consumer app shut down.

For developers, the useful lesson is not "switch vendors faster." It is "know which parts of your delivery flow depend on a tool you do not control."

Start with the exact scope

The shutdown was not universal. The consumer Gemini Code Assist app on GitHub is gone, but the enterprise GitHub app was not affected. Standard and Enterprise subscription access remains unchanged.

There was also a separate consumer IDE and CLI change. Consumer requests from Gemini Code Assist IDE extensions and Gemini CLI stopped on June 18, 2026 for Individuals, Google AI Pro, and Google AI Ultra tiers. Login with Google also became unavailable for those services. Google points affected consumer IDE and CLI users toward the Antigravity family of products, including a Gemini CLI to Antigravity CLI migration guide.

That distinction matters. If you treat every AI assistant notice as a platform-wide outage, you create noise. If you treat a consumer retirement as irrelevant because enterprise access remains, you may miss unsupported workflows in personal accounts, side projects, prototypes, or developer-local tooling.

Migration is a delivery-control problem

An AI coding assistant is rarely just a textbox that writes code. In real teams it may touch identity, repository permissions, review comments, generated artifacts, local IDE behavior, CLI scripts, and audit evidence.

That is why licensing, identity, code retention terms, exportability, and fallback behavior should be architecture inputs, not procurement footnotes. If a tool disappears, your question is not only whether another model can write similar code. Your question is whether the team can still prove what changed, who approved it, what evidence was preserved, and how work continues when the assistant is unavailable.

At Van Data Team, the first pass is to map identities, integrations, review gates, artifacts, and failure paths. That sounds boring until a tool retirement makes it the difference between a planned cutover and a scramble.

A concrete migration checklist

A controlled AI coding assistant migration should include:

Acceptance criteria: what the replacement must do before it is allowed into the workflow.
A representative pilot: real repos, real tasks, real review behavior, not a toy prompt.
Dual-running where possible: compare outputs, review load, artifact quality, and developer friction.
An explicit cutover: decide when the old path stops being supported.
An exercised rollback or manual fallback: prove the team can keep shipping without the assistant.

The rollback point is easy to skip because it feels pessimistic. It is also the part that tells you whether you own the workflow or merely rent it.

Tradeoffs developers should expect

A vendor-neutral workflow is not free. You may add more documentation, more review discipline, and more boring checks around generated code. You may also slow down the first migration because you are testing evidence trails instead of only testing code output.

The payoff is durability. Vendor-neutral governance keeps AI-authored code reviewable even after the assistant is removed. That means pull requests still make sense, artifacts can still be inspected, and reviewers are not dependent on a specific assistant UI to understand why a change exists.

There is also a product-tier tradeoff. Consumer tools are useful for experimentation, but they can change independently from enterprise subscriptions. In this case, the consumer GitHub app shutdown and the consumer IDE/CLI changes had specific dates and scope, while Standard and Enterprise access remained unchanged. Your migration plan should preserve that nuance instead of collapsing everything into one panic category.

The practical takeaway

The durable response to the Gemini Code Assist consumer shutdown is to own the workflow, its evidence, and its exit path. Do not assume a product tier will remain available. Do not assume a login method will keep working. Do not assume review history is portable unless you have checked.

If an assistant is part of your delivery path, treat migration like engineering work: define criteria, pilot on representative tasks, dual-run where possible, cut over deliberately, and test the fallback before you need it.

For teams already using AI coding assistants in production workflows, what is the first dependency you would audit: identity, review evidence, generated artifacts, or local CLI behavior?

📖 Read the full guide → AI coding assistant migration after Gemini Code Assist

Answer Engine Optimization After Google AI Mode Ads Testing

Tran Tien Van — Thu, 16 Jul 2026 15:02:04 +0000

On May 20, 2026, Google named four Gemini-powered Search ad formats.

That date matters less than the boundary it exposed: ads can buy labeled placement, but Answer Engine Optimization has to earn retrieval and citation.

The implementation mistake to avoid

AI Mode and AI Overviews are not the same surface.

AI Mode is Google's dedicated conversational experience. AI Overviews are summaries above conventional results. If your tracking, content model, and reporting treat them as one channel, you will blur two different user experiences and probably misread performance.

For developers and technical SEO teams, the practical problem is not just page copy. It is whether a search system can identify your entity, trust your commercial data, parse your page, and reuse a clear answer when the user changes the question.

That is a different design target from ranking for one static keyword.

Ads do not replace the earned layer

The announced formats are paid placements that remain labeled Sponsored. Conversational Discovery Ads and Highlighted Answers are being tested in the US. AI-Powered Shopping Ads and Business Agent for Leads are planned for the coming months.

That means there are two visibility tracks.

A brand can rent a Sponsored placement. It still has to earn organic retrieval, citation, and recommendation through Answer Engine Optimization. There is no payment path that turns an ad buy into an organic citation.

For teams building content systems, that boundary is useful. Paid Search can have its own owners, budgets, experiments, and conversion reporting. Organic answer visibility needs a separate evidence trail: what was cited, where the answer appeared, which entity facts were used, and whether the cited page supported the claim.

Conversation paths beat keyword lists

Traditional keyword research often treats the query as the unit of work. In conversational search, the unit is closer to a path.

A user might start broad, add constraints, compare vendors, challenge pricing, ask about integrations, then raise an objection. A single article that only targets the first query may be too thin for the follow-up questions.

Build pages and supporting data around the refinements people actually make:

Need: what problem the user is trying to solve
Constraint: budget, team size, region, stack, compliance, or timeline
Comparison: how one option differs from another
Objection: risk, cost, trust, setup effort, or missing feature
Commercial detail: pricing, availability, supported use cases, and product limits

That list is not a prompt hack. It is a content architecture checklist. If those answers live in disconnected pages, outdated PDFs, or copy that contradicts your schema, an answer engine has more chances to skip you or cite you badly.

Make facts boringly consistent

Useful, accessible content matters. So do consistent entity facts, accurate commercial data, and valid structured data. None of them guarantees inclusion, but missing or conflicting signals make inclusion harder to earn.

This is where developers can help beyond writing metadata.

Treat brand facts like production data. Product names, pricing claims, feature availability, locations, support terms, and organization details should have clear sources of truth. If the marketing page says one thing, schema says another, and the sales page is stale, the problem is not just SEO. It is data integrity.

Structured data should match visible content. Commercial pages should expose current facts in crawlable HTML. Important answer blocks should not depend on fragile client-side rendering. Internal links should make entity relationships and product relationships easy to follow.

Instrument paid and organic separately

One honest tradeoff: paid reporting will usually be cleaner than organic citation monitoring. Ads have campaign structures, labels, spend, and platform reports. Organic answer visibility is messier because inclusion is not guaranteed and conversational paths vary.

Do not compensate by merging the claims.

A paid campaign can say it bought a labeled Sponsored placement. An AEO program can say it improved the evidence base for retrieval, citation, and recommendation. Those are different success claims, and they need different owners.

That separation also protects engineering time. If leadership expects structured data work to produce the same deterministic reporting as paid media, the team will be judged against the wrong mechanism.

Where an agent fits

Vanaxity, Van Data Team's operator-built AI content agent, works on the earned side. The useful pattern is not magic generation. It is pipeline discipline: research, reconcile brand facts, structure answer-ready pages, verify claims through review gates, publish, syndicate, and monitor citations.

The same pattern applies if you build your own internal workflow. Start with facts, create answerable pages, validate claims, then monitor what actually gets cited.

What would you track first in an AEO pipeline: entity consistency, structured data validity, or citation appearances across conversation paths?

📖 Read the full guide → Answer Engine Optimization After Google AI Mode Ads

MLOps on AWS: Reproducible Models During 2026 AWS Handover

Tran Tien Van — Thu, 16 Jul 2026 06:29:21 +0000

On August 1, 2026, Dave Treadwell is scheduled to move to AWS while Dave Brown leaves Amazon after nearly 19 years. That is a personnel handover, not a signal to rewrite your ML platform.

The useful question for engineers is smaller and sharper: if AWS leadership changes, can your production model still be rebuilt, promoted, observed, paid for, and rolled back without oral history?

Treat the org change as a dependency test

The Brown-to-Treadwell transition matters because it gives teams a clean prompt to inspect assumptions. Amazon did not attach roadmap changes to EC2, containers, SageMaker, Bedrock, pricing, or ML strategy. Brown's note also frames continuity rather than panic.

That means the wrong reaction is speculative re-architecture. The better reaction is to ask whether your MLOps system would survive any leadership team, vendor roadmap, or internal reorg because its controls are explicit.

If a model depends on a training dataset nobody can trace, a feature transform owned by a former team, a runtime image rebuilt by hand, or a rollback path that only one person understands, the risk is already present. Leadership news just makes it easier to see.

Reproducibility needs one contract

A reproducible AWS ML workflow is not just code in Git and a model artifact in storage. The article's core standard is one traceable contract across code, data, features, runtime, configuration, evaluation, artifact, and deployment.

That contract is what lets a team answer practical questions. Which data snapshot trained this model? Which feature definitions were active? Which container image and instance class ran the job? Which evaluation result allowed promotion? Which endpoint or batch job received the artifact?

Without that chain, MLOps becomes a collection of services rather than an operating system. You may still be using good AWS primitives, but the model lifecycle is not defensible when something changes.

Promotion control is earned

A model registry sounds like governance, but it only creates control when it affects promotion. Approval, compatibility, deployment history, and rollback eligibility need to influence whether an artifact moves forward.

The practical test is simple:

Can an artifact be blocked because its evaluation is missing or stale?
Can promotion check whether serving code and feature contracts are compatible?
Can operators see where the model has been deployed before production?
Can rollback eligibility be determined before the incident starts?

If the registry is only a catalog, it is useful for discovery but weak as a release control. If it changes deployment outcomes, it becomes part of the production boundary.

Serving boundaries should follow failure modes

Platform teams often debate SageMaker endpoint versus containers versus custom EC2 patterns as if there is one fashionable answer. The better boundary follows scaling, permissions, release cadence, and failure isolation.

A model with strict latency requirements, independent scaling pressure, and separate release ownership may deserve a different serving surface from a batch model embedded in an existing pipeline. A model that shares permissions and deploys with an application might be simpler inside that application's release path.

The tradeoff is real. More separation can improve isolation and rollback clarity, but it also adds deployment surfaces and monitoring work. More consolidation can reduce operational overhead, but it can blur ownership and make failures harder to localize.

Portability is not an ideology

Portability is most valuable where a dependency is critical, hard to reverse, weakly observable, or expensive to reconstruct. That is a narrower and more useful standard than trying to make every layer cloud-neutral.

For many teams, the first portability investment should be documentation and artifact discipline, not an abstraction layer. If you can reconstruct training inputs, evaluation gates, runtime images, and deployment state, you have more practical resilience than a thin wrapper around every AWS API.

The operational standard

MLOps on AWS should keep every production model reproducible, promotable, observable, cost-accountable, and recoverable even when vendor leadership changes. That is not about guessing what AWS will do next. It is about reducing the number of production facts that live only in meetings, Slack threads, and memory.

At Van Data Team, the starting point is tracing the workflow, artifacts, compute surfaces, review gates, signals, and owners already shaping production. That map usually reveals whether the next useful change is a dependency register, a promotion gate, a serving boundary decision, or a recovery plan.

What is the first place your ML workflow would lose traceability: data, features, runtime, evaluation, deployment, or ownership?

📖 Read the full guide → MLOps on AWS: Systems That Outlast Leadership Changes