A slow AI reply can mean two different things: a long wait before the first word, or a slow stream of words after the answer starts. Knowing which one you have makes the next step less random.
OGAD (Off Grid AI Desktop) can show generation details below each answer, including available speed, timing, token, and context information. You can use those details to compare local replies on your own computer before changing models or settings.
What would you like to do with Off Grid AI?
Have a feature or use case you would like us to support? Tell us what you want to do and which device you use.
Write to support@offgridmobileai.co, join our Slack community, or talk to us on Reddit.
See what happens during a reply
The core desktop workflow in OGAD 0.0.51 includes Generation details. Use a downloaded local text model on Mac or Windows for a comparison that does not depend on a remote server connection.
- Open the chat settings and select Text.
- Turn on Generation details.
- Start a short chat with one clear request, such as: “Explain what a file backup is in five short sentences.”
- Let the answer finish, then open its generation details.
- Read the timing and token values that are available for that reply.
Some values come from the model server, and some come from the time OGAD observes. A missing value means the information was not supplied or could not be calculated. It does not mean zero work occurred.
Read the numbers in plain language
| Detail | What it helps you understand |
|---|---|
| TTFT | How long you waited before the first visible token |
| Tokens per second | The rate at which output was generated |
| Prefill rate | How quickly the model processed its input, when reported |
| Output tokens | How much text the model generated in token units |
| Total time | How long the reply took from request to completion |
| Context use | How much of the available context the input used |
A token is a unit of model text, not always a word. Comparing a 100-token answer with a 1,000-token answer by total time alone is not useful.
Context figures can be estimates when the server does not report exact prompt usage. OGAD marks an estimated prompt count with ~. Use that as a rough guide, not an exact token audit.
Make one useful comparison
Keep the model and prompt the same for your first comparison. Let the model load, finish the first answer, then try the same short request in a fresh chat. Record the prompt, model, output length, and available timings if you want to compare them later.
The first request can include startup work that a later request does not need. A long existing conversation also gives the model more input to process than a fresh chat. Those differences matter before you conclude that a setting made the model faster.
If the first words take a long time to appear, compare a short fresh chat with the longer conversation. If words arrive slowly after the response starts, try a smaller local model that fits your computer's available memory. If the answer streams well but takes too long overall, ask for a shorter answer and compare output length.
These are useful checks, not a diagnosis from one number. Other work on the computer, model loading, input length, and the selected runtime can affect a reply.
Keep remote and local comparisons separate
A remote reply can include connection delay and waiting on another computer. Its wall-clock timing is not a pure measure of your laptop's processing speed.
OGAD uses a server's decode rate when available and can calculate an estimate when enough timing and token information exists. Do not treat every displayed rate as a laboratory benchmark, or compare two machines from a single casual prompt.
The purpose is simpler: find out whether you are waiting for the answer to start, waiting for more text, or asking for more output than you need.
Try OGAD, enable generation details, and compare one short fresh-chat reply with a reply from the conversation that feels slow.
Top comments (0)