Cernalo

Posted on Oct 18 Originally published at cernalo.com

Ask AI About Part of Your Screen: A Region-Snap Workflow That Beats Copy-Paste

#ai #productivity #privacy #tooling

Originally published on cernalo.com.

Snap the region that matters, ask one specific question, and check the answer against the screen before you act.

The question is already on the screen

I build Cernalo. This is the workflow the product is built around, and the habits are useful even if you never install it.

Most of the questions I ask an AI are not really questions about the world. They are questions about something already in front of me: an error in a terminal, a chart someone dropped into a slide, a paragraph in a contract that reads three ways at once.

For a long time the routine looked like this. I would take a screenshot, open a chat tab, paste the image, type a sentence explaining what I was looking at, wait, read, switch back, and lose my place. If the answer led to a follow-up, I would do half of that loop again.

None of these steps is hard. Together they add up to a tax on every small question, and the tax has two parts that matter more than the clicks.

You rebuild context that was already on your screen. When you describe the problem in words, you summarise it. When you summarise, you drop details, often the very detail that mattered.
You share more than you meant to. A full-screen screenshot of a work computer is rarely just the thing you are asking about. It is also an email preview, a teammate's name in a chat window, and a browser tab you forgot was open.

Region, full screen, or typing

The workflow I use now is: snap only the region that matters, ask one question about it, then verify. I built Cernalo around it, but the habits work with any tool that lets you choose what it sees.

There are three ways to give an assistant context, and none of them is always the right choice.

Type it when the question does not depend on layout. Asking for the difference between a left join and an inner join does not need pixels. Typing also has a privacy advantage: you share exactly the words you write and nothing else.

Snap a region when the answer depends on something visual and local: an error message with its stack trace, a chart and its legend, a table, a code block, a form field you do not understand. A region keeps the relevant detail and drops everything around it. In Cernalo that shortcut is Ctrl+Shift+R: drag a box and ask.

Take a full screenshot when the relationship between parts of the screen is the question, for example why a dashboard shows a different total than the table beside it. In Cernalo that shortcut is Ctrl+Shift+S. Use it deliberately, because it is the option most likely to include something you did not intend to share.

My rule of thumb: default to the smallest context that could possibly answer the question. If the answer comes back confused, widen the context. Do not start wide.

Pick the region and look at it before you send it

Draw the box around the problem plus the minimum surrounding information needed to interpret it.

Then do the thing most people skip: glance at what you captured. Is there a name, an email address, an API key, or a customer record in the box? Shrink the box. Capture should start only when you start it, and Cernalo works that way, but the decision about what goes into the box is still yours.

For an error, include the error line, the stack trace, and the line of code it points to.
For a chart, include the plot, the axis labels, the legend, and the title. Without units and dates, a chart is just a shape.
For a paragraph, include the paragraph plus the heading above it, so the assistant knows what document it is in.

Ask one specific question

The screenshot provides the what. Your question should provide the why.

Good snap questions usually start with why, what changed, which one, or what you would check next. Asking for the fix invites a confident answer. Asking for the most likely cause and how to confirm it invites a checkable one.

In Cernalo you can pick a mode before you ask, such as Coding, Research, Screen, or General. The mode shapes the answer. Coding leads with the likely issue and a code suggestion. Research is more explicit about uncertainty. Pick the one that matches the job.

"What is this?" is vague. You will get a description of the screenshot.
"Why does this request fail when the payload is empty?" is specific. You will get a hypothesis you can test.

Read the answer, then verify it against the screen

This step is the point of keeping the answer next to the work instead of in another tab. With the source still visible, you can check the answer against it immediately.

A screenshot gives an AI pixels, not the program state, database, or business context behind them. The answer is a hypothesis. Treating it as a review aid rather than a verdict is the habit that makes this workflow safe.

Does the explanation refer to text that is actually in the snapped region, or did it invent a line?
If it quotes a number from a chart, does that number match the axis?
If it suggests a code change, does the variable name it uses exist in your file?

Ask the follow-up while the context is still there

The second question is often the valuable one: how would I confirm that, or what else could produce the same symptom? Because the session keeps the conversation, you can follow up without snapping again.

When you move to a different problem, clear the session so the old context does not leak into the new answer. In Cernalo you can clear or end a session whenever you choose.

A coding error

These worked examples show the question shape. They are not presented as recordings, and the paragraph example later in this post is illustrative wording.

The screen shows an error in a code editor: a function fails on a particular input. Instead of copying the stack trace into a chat, snap the error and the highlighted line and ask: what is the most likely cause, and how can I confirm it?

The useful answer names the likely cause, points to the line where it happens, and suggests a check. What you do next is not AI work: reproduce the error, apply the change, and run the test. If the test passes, good. If it does not, snap the new error and ask again, with the previous answer still in the session.

The anti-pattern is pasting the suggested fix without reading it. A suggestion that makes an error disappear is not the same as a fix.

A chart on a slide

A sample-data sales slide shows revenue by region. Snap the chart with its legend and title and ask: which region is highest, which is lowest, and what stands out?

A good answer reads the chart back: the highest, the lowest, and the outlier. Before you repeat any of it in a meeting, check three things the AI cannot know: the units, the date range, and whether the slide uses a non-zero baseline. If the chart does not show those, the honest answer is that the chart does not say, and a good assistant should tell you that rather than guess.

A dense paragraph

This example uses illustrative wording. It is not a transcript of a recorded document.

A paragraph in a policy document says something like "unless otherwise agreed in writing, notwithstanding clause 4". Snap the paragraph and its heading and ask: what is the main claim, and what would weaken it?

That question gives you something to check against the source, which "summarise this" often does not. Read the clause it points to and decide for yourself. For anything legal, medical, or financial, the AI's job is to help you read, not to tell you what to do.

What a screenshot cannot tell an AI

This is where screen-aware tools disappoint people, so it is worth being explicit.

Hidden state. A screenshot does not include the database row, the environment variable, or the network response behind the UI.
Off-screen content. If the cause is in a file you did not capture, the AI can only guess.
Small or low-contrast text. Tiny fonts and dense tables get misread. Zoom in before you snap.
Instructions inside the image. Text in a screenshot is data, and it may contain instructions such as "ignore previous directions". Cernalo's service tells the model to treat on-screen and quoted text as untrusted context, not as instructions. That reduces the risk. It does not remove it. If something on screen looks like it is trying to steer the assistant, do not send it.

Privacy habits for screen questions

These apply to any tool, including ours. The security notes say plainly what Cernalo does and does not do, including that we do not hold SOC 2 or ISO 27001 certifications today. Current plans, including the free plan's monthly credit allowance, are on the pricing page. Screen analysis may use one extra credit per request.

Prefer a region over a full screen. It is both clearer context and less exposure.
Check the box before you send. Names, email addresses, keys, and customer data have no business in a casual question.
Know what runs where. In Cernalo, basic voice transcription runs on your device, but the question and any capture you send go to our service and then to the AI model needed to answer it.
Clear the session when you switch topics.
Follow the rules of the room. If your workplace, school, or a meeting does not allow AI assistance, do not use it there. Cernalo is built to be used openly.

Questions

When is a region snap the right input, and when is pasted text? When the layout matters, such as charts, tables, error output, and forms, a region keeps the structure. When it does not, pasting or typing the text shares less and answers the same question.

Does the AI see the whole screen all the time? Not in Cernalo. Screen and region capture start only when you start them, and the app shows when capture is active. Other tools work differently. Some record continuously by design, so check each tool's documentation.

Can you use voice instead of typing the question? Yes. In Cernalo you can ask by voice with Ctrl+Shift+V. Basic transcription runs locally, and performance depends on your hardware. Review the transcript before you send it.

What does it cost? Cernalo has a free plan with a monthly credit allowance. Current plans are on the pricing page. Screen analysis may use one extra credit per request.

The context you choose, in one place. If you want to try the snap-and-ask workflow, the free plan is on the download page. Feedback, especially about where the workflow breaks for you, goes straight to me at cernalo.ops@gmail.com.

Continue exploring

Cernalo is an AI desktop assistant: snap part of your screen or ask by voice. Try it free at https://cernalo.com/?utm_source=devto&utm_medium=crosspost&utm_campaign=ask-ai-about-part-of-your-screen

DEV Community