DEV Community

yulingg zhang
yulingg zhang

Posted on Fully Autonomous

Measure completed actions, not just clicks, in a small AI tool

A new AI tool can attract visitors without telling its owner whether anyone finished a useful task. Page views alone cannot distinguish someone who copied a prompt from someone who selected a photo, submitted a generation request, or downloaded a result. Putting all these actions into one usage total makes the next product decision harder.

I maintain RetroPrompt, a small portrait and prompt project. The lessons below come from its current event implementation, rather than a claimed conversion experiment. This article was prepared with AI assistance and does not present invented traffic or conversion numbers.

Start with event definitions

The first job is to write down what each event actually proves. In this implementation, prompt_copy is emitted after the browser's clipboard write resolves. photo_selected means the file picker supplied a file. generate_submit marks the start of a generation request. generate_success means the response passed basic checks and the result was assigned to the image area. generate_error records a controlled error category. download_click records a click on the download link.

These distinctions matter. Selecting a photo does not prove that its upload completed. Clicking Download does not prove the file reached the visitor's device. Even a generation-success event needs a precise definition: the current frontend checks the response and its image-data prefix, then sets the image source. It does not wait for the image to finish decoding. If broken rendering becomes a problem, a separate decode check would provide stronger evidence.

Recognize more than one successful journey

A visitor who copies a prompt and leaves may have completed their entire task. It would be misleading to count every such departure as a failed image-generation session.

Treat the prompt-copy journey and the image-generation journey separately. For the second journey, look at photo selection, request submission, result delivery, and download clicks. For the first, check whether the visitor could find and copy a useful prompt. Keep the event meanings stable when comparing versions of a page.

Keep your own tests visible in the data

When a site is new, the maintainer's repeated testing can overwhelm the small sample of real activity. RetroPrompt uses an explicit QA switch, enabled with the qa=1 query parameter and remembered in browser storage and a cookie. Events carry a QA flag, and the page shows a test-mode indicator. Switching modes resets the current session identifier to reduce mixed test and non-test sessions.

The reporting query still has to exclude flagged events. A frontend flag by itself does not clean a dashboard. This mechanism also operates per browser: a new device or cleared browser storage requires setting it again. It cannot establish that every visit by the maintainer has been excluded.

Let the tool work when analytics fails

Analytics requests should not block the action being measured. Here, event delivery uses a non-blocking fetch with keepalive and catches delivery failures. The visitor should still be able to copy a prompt or request an image when the analytics endpoint is unavailable.

The trade-off is missing events. Privacy settings, blockers, interrupted connections, and closing the page can prevent delivery. Keepalive is not a guarantee. Compare browser events with separately defined server-side job records when investigating a mismatch; do not assume the two totals must agree exactly.

The event payload is deliberately narrow: event name, page path, style identifier, controlled error category, random event and session identifiers, and the QA flag. It does not include the photo, full prompt, or raw exception stack.

Turn observations into questions

Many copies but few generation requests could mean visitors prefer the prompt library. It could also mean the mobile generator entry is difficult to find. Photo selections without submissions suggest checking the button state, instructions, or quota display. Delivered results with few download clicks suggest checking the result presentation and download control.

These are investigation paths, not causal conclusions. Verify the event definitions and delivery quality, inspect the relevant interaction, and then compare a targeted page change. A small, interpretable funnel is more useful than a large dashboard whose event names overstate what happened.

For context, the workflow discussed here is on RetroPrompt's portrait generator. I maintain the linked project.

Top comments (0)