AI prototypes are easy to celebrate and surprisingly hard to ship. In 2026, a Unity developer can connect a model to a dialogue box in an afternoon, but that says nothing about latency, safety, platform support, testing, or maintenance. After 16 years in game development, I care less about whether a model can produce an impressive answer and more about whether the feature survives a bad connection, a provider update, and an inventive player. My work has ranged from Eagle Flight Arcade at Ubisoft Montreal in 2016 to Touch Camera PRO and client projects such as Meta Spirit Sling and Loreal Viva Tech 2024. Those projects do not prove that every game needs AI. They explain why I approach runtime AI as production infrastructure rather than magic.
Key Takeaways- Give AI narrow responsibilities and keep authoritative game state deterministic.
- Put provider credentials and validation behind a backend you control.
- Treat every model response as untrusted external input.
- Test requirements and failure modes instead of expecting identical sentences.
- Design explicit fallbacks for latency, outages, cost limits, and offline play.
What Should an AI Feature Be Allowed to Control?
The first production question is not which model to use. It is what the model is allowed to control. I divide possible responsibilities into three categories: presentation, recommendation, and authority. Presentation includes rewriting a hint or adapting the tone of a tutorial. Recommendation includes selecting a likely response, suggesting an item, or ranking authored options. Authority includes changing inventory, awarding currency, resolving combat, saving progression, or making a purchase. I am comfortable experimenting with the first two. I strongly resist giving a generative model the third.
That line comes from conventional game development. Eagle Flight Arcade was a VR flight game shipped in 2016 on PSVR, Oculus Rift, and HTC Vive. Its feel depended on controlled movement, predictable rules, and platform-aware behavior. Touch Camera PRO has the same basic obligation in a different context. A camera controller must respond consistently when a player pinches, pans, follows a target, or hits a boundary. A clever but unpredictable answer is not a substitute for reliable interaction. Runtime AI should usually sit beside the simulation, not become the simulation.
I therefore define an output contract before writing a prompt. A hint system might let a model choose one hint identifier from an allowlist. A conversational character might select an authored intent plus optional display text. A level assistant might propose parameters, but deterministic code validates and applies them. For every output, I ask what happens if it is empty, malformed, hostile, irrelevant, or ten seconds late. If the answer is that progression breaks, the model owns too much. A useful feature can be summarized as: AI proposes, code verifies, and the game decides. That sentence is far more valuable than a giant system prompt.
Should the Model Run on the Device or Behind a Server?
On-device inference, server inference, and hybrid architecture solve different problems. Local execution can improve privacy, remove per-request network latency, and support offline play. It also adds model files to the build, consumes memory, competes for CPU or GPU time, and behaves differently across hardware. That last issue matters in Unity because one project may target desktop, mobile, consoles, WebGL, or standalone VR. A model that feels acceptable on a development PC may be inappropriate on the lowest supported phone or headset.
Server inference makes model upgrades easier and gives the team tighter control over credentials, rate limits, logging, and provider selection. The tradeoffs are network dependency, operating cost, regional availability, and additional privacy work. A provider key must never be embedded in a Unity client. Players can inspect builds and network traffic, so a secret stored in the application should be treated as already exposed. My preferred server design sends compact game context to an endpoint I control. That backend authenticates the player, removes unnecessary data, calls the model provider, validates the response, and returns a small application-specific result.
For many games, the practical answer in 2026 is hybrid. Deterministic local logic remains the foundation. A server model adds optional language or recommendation features, while authored content handles offline and failure states. Smaller local models can support narrow classification tasks when the target hardware justifies the download and performance cost. I make this decision from a platform matrix, not a model leaderboard. List every supported device, expected connection state, memory constraint, privacy requirement, and acceptable wait. Then profile the actual build on the weakest hardware. Shipping Eagle Flight Arcade across three VR platforms reinforced a lesson that still applies: platform differences are product requirements, not cleanup tasks for the final week.
How Can Nondeterministic Output Be Made Safe?
A low temperature does not turn a generative model into deterministic game code. Providers update infrastructure, model versions change, and tiny context differences can alter an answer. I treat model output exactly like data received from an unknown external service. It crosses a trust boundary and must pass validation before any gameplay system, save file, UI renderer, or analytics event uses it. Prompting is useful guidance, but a prompt is not a security boundary and it is not a schema validator.
I use three validation layers. The syntactic layer checks whether the response matches the expected JSON shape, types, required fields, and size limits. The semantic layer checks allowlists, text length, supported locales, prohibited markup, and numeric ranges. The game-state layer asks whether the requested action is legal right now. A generated command to unlock a door is rejected if the player has not met the deterministic unlock conditions. A generated hint identifier is rejected if it does not belong to the current objective. Free text should be escaped before rendering, and arbitrary URLs, rich-text tags, asset paths, or executable command names should not be accepted.
Every request also needs a boring fallback. I usually prefer an authored default over repeated model calls. One controlled retry may be reasonable for a transient transport failure, but repeatedly asking a model to repair its own invalid output increases latency and cost without guaranteeing success. Prompts, schemas, model identifiers, and validation rules should be versioned together so a regression can be traced. Logs can record timing, failure category, token usage, and version identifiers, but they should avoid raw personal or sensitive content. Most importantly, players must not be able to inject instructions through names, chat, imported files, or community content that the application blindly places inside a privileged prompt. Data is data, even when it contains convincing instructions.
What Does a Maintainable Unity Integration Look Like?
I do not let gameplay scripts know which AI provider is being used. The Unity side should depend on a small interface expressed in the language of the game, such as requesting a hint, classifying an intent, or summarizing an authored journal entry. A provider adapter can live behind that boundary, preferably on a backend for cloud models. This keeps networking details, authentication, provider response formats, and model migrations out of scenes and MonoBehaviours. It also makes the feature testable with a fake implementation.
using System;
using System.Threading;
using System.Threading.Tasks;
using UnityEngine;public interface IAiHintService
{
Task<AiHintResult> GetHintAsync(
AiHintRequest request,
CancellationToken cancellationToken);
}[Serializable]
public sealed class AiHintRequest
{
public string objectiveId;
public string locale;
public string[] allowedHintIds;
}public readonly struct AiHintResult
{
public bool Success { get; }
public string HintId { get; }public AiHintResult(bool success, string hintId) { Success = success; HintId = hintId; }}
public sealed class HintController : MonoBehaviour
{
private IAiHintService _service;
private CancellationTokenSource _request;
private int _requestVersion;public void Install(IAiHintService service) { _service = service; } public async void RequestHint( string objectiveId, string[] allowedHintIds) { if (_service == null) { ShowFallback(); return; } _request?.Cancel(); _request?.Dispose(); _request = new CancellationTokenSource(); int version = ++_requestVersion; var input = new AiHintRequest { objectiveId = objectiveId, locale = Application.systemLanguage.ToString(), allowedHintIds = allowedHintIds }; try { AiHintResult result = await _service.GetHintAsync( input, _request.Token); if (version != _requestVersion) return; if (result.Success && Array.IndexOf(allowedHintIds, result.HintId) >= 0) { ShowAuthoredHint(result.HintId); } else { ShowFallback(); } } catch (OperationCanceledException) { if (version == _requestVersion) ShowFallback(); } catch (Exception exception) { Debug.LogException(exception); ShowFallback(); } } private void OnDestroy() { _request?.Cancel(); _request?.Dispose(); } private void ShowAuthoredHint(string hintId) => Debug.Log($"Show hint: {hintId}"); private void ShowFallback() => Debug.Log("Show the authored fallback hint.");
}
The important detail is that the model returns an identifier from an allowlist, not an instruction that directly manipulates the scene. Local code still verifies the identifier and maps it to authored content. For a feature that genuinely needs generated prose, I would expand the result object with explicit validation status, moderation status, model version, and a safe display string. I would not return an unstructured provider response to the UI.
Cancellation and request ownership also matter. Players close menus, change objectives, reload scenes, and make a second request before the first finishes. Old responses must not overwrite current state. Timeouts belong in the service layer, while visible fallbacks belong in the feature layer. Finally, all Unity object access should remain on the Unity main thread. Parsing, transport, and validation can be separated, but a background callback should not casually modify a GameObject. Clean boundaries make these rules much easier to enforce.
How Do You Test a Feature Whose Answers Keep Changing?
Traditional unit tests compare a known input with an exact output. That is still appropriate for validators, allowlists, parsers, fallbacks, and state transitions, but it is usually the wrong test for generated language. I test properties instead. Did the response use the requested language? Is it below the display limit? Does it avoid revealing hidden information? Did it select an allowed intent? Does the feature reach a valid fallback when the response is malformed? These requirements are stable even when the wording changes.
I recommend building an evaluation set before launch. Start with representative normal cases, then add empty context, contradictory context, very long input, unsupported languages, prompt injection attempts, offensive player names, network timeouts, and requests made during scene changes. Fifty carefully chosen cases can reveal more than hundreds of casual prompts, although the right size depends on the feature. Each case should have machine-checkable rules plus a small human review rubric for relevance, tone, factual consistency, and usefulness. Do not collapse everything into one vague quality score. A response can sound excellent while violating a progression rule.
My preferred pipeline has three layers. Fast continuous integration tests exercise deterministic code with fake and recorded responses. A scheduled evaluation calls the current model and compares pass rates by prompt, schema, and model version. Human reviewers inspect sampled failures and sensitive categories before a release. This avoids paying for live model calls on every code commit while still detecting provider drift. It also creates evidence for model changes instead of relying on someone saying the new output feels better.
Localization deserves its own evaluation set. Text expansion, gender, formality, cultural context, unsupported characters, and right-to-left layouts can expose both model and UI failures. Test on the actual Unity screens, not only in a provider playground. A sentence that is acceptable in isolation may cover a button, break subtitles, or conflict with an authored voice line. The final product is the game experience, not the raw response.
How Should Latency, Cost, and Offline Play Shape the UX?
Runtime AI introduces a new timing category. A normal button press should feel immediate, but a network model may take seconds or fail entirely. My UX rule is to acknowledge input within roughly 100 milliseconds, show honest progress if work continues, and provide a cancel or fallback path when a wait becomes noticeable. Those are design targets, not promises that every provider will meet them. The feature should never freeze the main thread, block scene loading indefinitely, or leave a player staring at an animation that implies success is guaranteed.
Streaming text can reduce perceived latency, but it creates additional problems. Partial output may contain markup, unfinished sentences, or content that has not passed final validation. For many game features, I prefer waiting for a complete, validated response and showing an authored transitional state. If streaming is central to the experience, validate chunks conservatively and reserve the right to replace the output with a safe fallback. In VR, unstable frame timing is far more damaging than a slow text response. My experience with Eagle Flight Arcade made me strict about keeping optional services away from frame-critical movement, rendering, and input paths.
Cost should be designed like memory or bandwidth, not discovered after launch. Estimate requests per active session, input size, output limits, retries, moderation calls, and the cost of abuse. Then enforce server-side quotas and rate limits. Compact structured context is usually better than sending an entire conversation or save file. Cache only when the request is nonpersonal, the result is safe to reuse, and invalidation is understood. Deduplicate repeated button presses and set hard output limits. A cheaper model can handle classification while a more capable model is reserved for rare, genuinely complex requests.
Offline behavior must be visible in the feature specification. A game may use an authored hint, disable optional generation, queue a nonurgent request, or use a small local classifier. What it should not do is silently break. If the core loop cannot function without an external model, the team is operating a live service whether it planned to or not.
What Privacy and Security Work Is Required Before Launch?
Before integrating a model, I create a data map. What leaves the device? Does it include chat, account identifiers, location, voice transcripts, screenshots, save data, or user-generated content? Where is it processed, how long is it retained, who can access it, and how can it be deleted? The safest input is information the feature never collects. Data minimization also reduces token cost and makes prompts easier to reason about, so privacy and engineering quality often point in the same direction.
The Unity client should communicate with an authenticated backend over secure transport. That backend should enforce request size limits, per-user and per-device rate limits, model allowlists, timeouts, and spending controls. Provider credentials stay on the server and should be rotatable. Logs need access controls and retention rules. If debugging requires examples, use redacted or synthetic cases whenever possible. Projects aimed at children, workplaces, health-related contexts, or public installations may need additional legal and policy review. A developer should not guess at those obligations from a model provider's marketing page.
Prompt injection is only one part of the threat model. A player might submit huge inputs to increase cost, automate requests, place hostile instructions in imported content, attempt to expose hidden prompts, or persuade the model to emit unsupported commands. Output can also become an injection channel if the game interprets rich text, URLs, filenames, or tool names. The defense is architectural: separate instructions from untrusted data, expose narrowly scoped tools, validate every argument, authorize actions in conventional code, and encode text for its destination.
I also think teams need honest player communication. Explain when content is generated, when data is sent to a service, and what happens if the service is unavailable. Provide reporting tools when players can encounter generated public content. On freelance work, whether the client is an indie team or a much larger organization, I ask these questions before polishing the prompt. Security added after a successful prototype is usually expensive because the prototype already gave the model too much data and authority.
When Is Conventional Game Logic Better Than Generative AI?
Generative AI is the wrong tool when a feature has a small state space, strict timing, exact balancing, or a clear algorithmic solution. State machines, behavior trees, utility systems, procedural algorithms, search, authored dialogue, and ordinary databases remain excellent technology. They are fast, inspectable, testable, and available offline. A camera controller such as Touch Camera PRO should not ask a model how far to pan. A level rule should not become probabilistic merely because a prompt looks shorter than the equivalent code.
I use a simple filter. Does the task require open-ended language or fuzzy interpretation? Can the result be verified before use? Can the player recover from a bad answer? Is a fallback available? Is the value worth the latency, cost, privacy work, and vendor dependency? If several answers are no, I reject runtime generation. Sometimes AI can still help during development, but that is a separate decision from putting a model call in the shipped product. Tools such as Tutorial Engine, Level Designer, and Touch Camera PRO serve developers by making repeatable behavior easier to author. Predictability is often the feature.
The same skepticism applies to fashionable autonomous agents. Giving a model more tools and more context can make a demonstration look capable, but it also expands the number of actions, failure states, and security checks. I prefer the smallest useful capability. One validated request that selects an authored hint may create more player value than an elaborate agent that reads the entire save and attempts to manage the experience. Scope is not a failure of ambition. It is how a team creates something supportable.
AI in 2026 is useful enough that developers should understand it, but unstable enough that architecture matters more than hype. My shipping checklist is straightforward: define authority, choose the execution location, validate input and output, provide a deterministic fallback, measure latency and cost, build evaluations, review privacy, and plan provider replacement. If a feature still makes sense after that work is visible, build it. If it only looked attractive when failure was ignored, conventional code has already given you the answer.
Top comments (0)