A content workflow has several distinct failure points: receiving a source, deciding whether it is a duplicate, producing a draft and authorizing publication. This article lays out checks for those boundaries using my content factory as context. The checks are proposed test scenarios, not results from an executed test suite
The factory is a team of AI agents in Telegram. It receives material from 13 news sources, filters repeated items, highlights topics, prepares posts in the author's voice, checks facts and adds an image. A person reviews the preview; my approval is required before publication. Telegram is the known interaction surface here. I am not specifying unverified package versions, queue technology or deployment details
Define a boundary before testing it
Start with a single stage. For source intake, the input is news material from the specified sources. The output is material available for the next stage, with enough context to find the original. The observable question is whether the reviewer can trace an item back to the correct source
A useful test set includes a normal item, an item missing a number and an unavailable source. These are hypothetical inputs for a reader to construct, not events I am claiming happened in my system. Expected behavior should distinguish missing data from a factual value. If a source is unavailable, a reviewer should not be told that the source was successfully checked
Do not infer that a missing item was filtered as a duplicate until intake is known to have received it. A failure earlier in the chain and a decision later in the chain can have the same visible result: an item does not appear in the preview. Separate observations at the two boundaries make diagnosis possible
Test filtering with counterexamples
A duplicate filter has two kinds of error to consider. It can leave two reports of the same event in the result, or it can remove a distinct event because the wording looks similar. The latter can be missed if the reviewer looks only at retained items
Construct two pairs for a local review: the same event described with different headlines, and different events using similar terms. Compare the retained and rejected material with the source text. The expected result is one representative of the repeated event and both distinct events retained. This is a proposed check; no classifier accuracy is asserted
When changing a rule, repeat the earlier checks too. A rule tuned to catch a missed duplicate may start suppressing distinct news. Recording the source and the reason for a decision gives a human reviewer something to inspect when that happens. It does not require revealing private credentials or production data
Treat a draft as an intermediate artifact
The drafting stage turns a selected topic into text in the author's voice. Fact checking and image selection add further review requirements. A polished paragraph can still turn a forecast into an observed event, or carry over an incorrect number. Review the actual claim against the source, not only whether the paragraph reads fluently
For a manual test, pick a sentence with a number and find that number in the input. Pick a sentence describing a future event and check whether the draft preserves that status. If an input lacks a figure, the draft should not fill the gap with a plausible-looking estimate. These checks are examples for a reader to run on their own material
An image needs review too. The existence of a file does not establish that it depicts the topic or is appropriate to use. In my workflow, the preview presents the text and image for review before publication
Make approval a separate state
The system prepares a preview, and I can request changes. The post is published only after my yes. The factory has published 84 posts, none without my approval. The specific boundary is between preparing the material and authorizing an external action
For a design review, explicitly consider a preview with no approval, one that was rejected for changes, and one approved by its owner. The first two must not be treated as publication authorization. The last can move forward only under the actual application's rules. These are desired scenarios for testing, not claims that a particular state machine or test suite is implemented
If a process times out after attempting an external action, an unknown result is not proof of failure. Check the target before retrying; otherwise a second attempt may duplicate the action. This is a general operational recommendation, not a description of a measured incident in the factory
Keep corrections reviewable
Since October 28, the factory has compared my edits with channel rules and added to those rules to avoid repeating mistakes. A changed rule can improve a later draft, but it does not replace review of the actual result. Keep the original error and a counterexample when evaluating a change so that the correction does not silently break a case that previously worked
In my process, preparing a post went from about an hour to roughly 5 minutes to a finished preview. That is not a latency guarantee for another stack. It is also not proof that each intermediate stage is error-free
For a broader explanation of why I separate preparation from human decisions, see what tasks an AI employee can take on. For your own implementation, choose one boundary, define its input and expected output, construct a counterexample and inspect the result before adding another autonomous action
This article was created with the help of AI.
Top comments (0)