A user taps Send on a mobile AI app. The request starts normally. A few tokens appear on the screen. Then the user locks their phone. The network connection changes. The app goes into the background. A few seconds later, the user opens it again.
Now what? Did the request finish? Is the server still processing it? Should the client reconnect? Should it start the request again? What if the first request is already completed and the retry creates a duplicate operation?
This is where building an AI feature starts to look very different from building a traditional mobile API integration. For a normal API request, we often think in terms of: Tap → Request → Response
The request starts, the server processes it, and eventually the client receives a response. An LLM interaction is often closer to: Tap → Request → Processing → Tool Calls → Partial Output → Streaming → Completion
That difference isn't just an API implementation detail. It changes the architecture of the mobile application.
An LLM Request Is an Operation, Not Just a Request
Consider a traditional endpoint such as: GET /users/123
The client sends the request and waits for the response. If the connection fails, the client can usually retry according to a straightforward policy. AI interactions are less predictable.
A request might take several seconds. The model may generate output incrementally. The server might call a search service, database, or external API before producing the final response. The operation can fail after partial output has already reached the client.
That means the client isn't simply waiting for a response. It's observing the state of a long-running operation. This distinction becomes especially important on mobile because the environment itself is unreliable. Connections change. Apps are backgrounded. Processes can be suspended or terminated. Users switch networks. The operating system can reclaim resources. A mobile AI client needs to assume that these things will happen.
Streaming Changes the State Model
With a traditional request, you can often think of the response as one object. Either you received it or you didn't. Streaming changes that.
Suppose the model has generated: “The deployment failed because the service…” The connection drops. The client already has part of the answer. What should happen next? A naive implementation might simply retry the entire request.
Now the server generates the response again, potentially producing duplicated work and a different output.
The client might end up with: “The deployment failed because the service… The deployment failed because the service…”
A better design treats streamed output as an incremental state. The client should know which operation it is observing and what portion of the output it has already received.
This is where an operation identifier becomes useful. Instead of treating the interaction as: POST /generate → response
think in terms of:
operation_id = abc123
status = running
output = partial
The client can then reconnect and ask about the existing operation rather than assuming that the original request disappeared.
Connection Drops Are Normal on Mobile
A desktop application can often maintain a relatively stable connection. A mobile application can't make that assumption. The user can move from Wi-Fi to cellular. They can enter an elevator. They can switch applications. The device can temporarily lose connectivity. The application can be suspended while the server is still processing the request.
For an AI interaction, a connection drop doesn't necessarily mean the operation failed. That's an important distinction. There are at least two separate states:
Transport state: Is the client currently connected?
Operation state: Is the AI operation still running?
Those states should not be treated as the same thing. A socket can disconnect while the server continues processing.
Likewise, a connection can remain available while the underlying operation has already failed.
If the client treats every network failure as an operation failure, it can create duplicate requests and inconsistent UI state.
Reconnection Shouldn't Automatically Mean Retry
This is one of the most dangerous patterns in an AI mobile client:
Connection lost
↓
Retry request
↓
New operation
What if the original operation is still running? Now you have two operations. Both may execute tools. Both may consume model tokens. Both may modify external state. Both may eventually produce responses.
For read-only generation, this might primarily create unnecessary cost and duplicated output. For an agent that can perform actions, it can become much more serious.
Imagine the model is processing: “Send the customer an email confirming their refund.”
The connection drops immediately after the email tool executes. The client doesn't know whether the operation completed.
If it blindly retries, the email could potentially be sent twice. The solution is to make operations idempotent or explicitly trackable wherever possible. The client needs enough information to distinguish: “The request failed before execution.” From: “The request is still running.” And: “The request was completed, but I never received the final response.” Those are very different situations.
Cancellation Is More Than Closing the Connection
Now consider another common interaction. The user asks the AI a question. The model starts generating. The user taps Stop.
A naive implementation might simply close the streaming connection. But that only changes the client's transport state. It doesn't necessarily stop the server-side work.
The model may continue generating. A tool call may continue executing. A database query may still be running. The client has disappeared, but the operation hasn't necessarily been cancelled.
A proper cancellation flow should communicate cancellation to the server. Conceptually:
Client
↓
Cancel operation_id
↓
Server
↓
Stop generation / cancel tools where possible
↓
Operation = cancelled
Not every operation can be interrupted instantly. A tool that's already committed an external side effect may not be reversible. But the system should at least have an explicit concept of cancellation rather than assuming that disconnecting the client automatically stops everything.
Timeouts Need More Thought Too
Traditional API clients often have relatively straightforward timeout logic. If the server hasn't responded after a certain period, fail the request. For an LLM operation, a long delay doesn't necessarily mean failure. The server may be waiting for a tool. The model may still be generating. A response may simply be slow.
Streaming makes this even more interesting because you can have an active connection without receiving output for some period.
Instead of thinking only about a single request timeout, AI clients often need to distinguish between things such as connection timeout, inactivity timeout, and overall operation timeout.
For example, receiving tokens every few seconds might indicate that the operation is healthy even if the final answer hasn't arrived yet. A completely silent connection may indicate a problem. Again, the important part is understanding the operation rather than treating it like a single HTTP response.
Backgrounding Creates Another State Transition
Mobile applications have another problem that backend applications don't usually face in the same way: the user can simply leave. They can press the home button. They can lock the phone. They can open another application. The operating system may suspend or terminate the application.
What happens to an LLM operation during that time? The server may continue processing. The client may stop receiving streamed output. When the user returns, the application needs to recover its state. This is why AI conversations should not depend entirely on in-memory client state.
The application should be able to reconstruct something like:
- Conversation
- Operation ID
- Operation status
- Partial output
- Final output
- Error state after a reconnect or application restart. The user shouldn't have to resend their question simply because the operating system suspended the application.
State Recovery Is Part of the UX
Once you think about AI interactions as operations, state recovery becomes much easier to reason about.
Imagine the client reconnects after being offline. Instead of immediately sending the user's message again, it can ask the backend: “What happened to operation abc123?”
The server might respond: running. The client resumes observation. Or: completed. The client retrieves the final result. Or: failed. The client displays the error. Or: cancelled. The client restores the appropriate UI state.
This is fundamentally different from blindly retrying an HTTP request. The client is recovering the state of an operation. That small architectural distinction can prevent a lot of difficult edge cases.
AI Changes the Mobile Interaction Model
The biggest mistake is treating an LLM endpoint as just another REST endpoint. It might technically be exposed through HTTP. But the interaction behaves differently. A traditional request is often short-lived: Request → Response → Done
An AI operation can be: Start → Process → Stream → Pause → Reconnect → Resume → Complete
It can involve multiple backend services and tools. It can produce partial results. It can be cancelled. It can outlive the client connection. It can continue while the mobile application is in the background. That means the mobile architecture needs to model these states explicitly.
The client should know whether an operation is pending, running, streaming, completed, failed, cancelled, or recovering. The backend should provide enough state to make those transitions reliable. And actions with side effects should have strong guarantees around authorization and idempotency.
Design the Operation, Not Just the API Call
When building an AI feature for mobile, I wouldn't start by asking: “What's the endpoint?”
I'd start with: “What happens if the connection disappears halfway through?”
Then ask: What happens if the user taps Stop? What happens if the app goes into the background? What happens if the network changes? What happens if the server finishes while the client is disconnected? What happens if the client reconnects but doesn't know whether the operation completed? What happens if the user retries?
These aren't unusual edge cases on mobile. They're normal operating conditions.
An AI-powered mobile application therefore needs to treat an LLM interaction as a long-running asynchronous operation, not simply a request that eventually returns JSON. The API is only one part of the design.
The harder engineering problem is everything around it: streaming, cancellation, retries, timeouts, connection recovery, duplicate prevention, background execution, partial responses, and state recovery.
The model may take seconds to answer. The network may disappear in milliseconds. A good mobile AI architecture has to handle both. AI doesn't just change what your mobile app calls. It changes how the app has to think about the interaction itself.
Top comments (0)