DEV Community: Krasimir Petkov

Why an AI Research Answer Can Have Sources and Still Be Wrong

Krasimir Petkov — Sat, 24 Oct 2026 17:57:52 +0000

Why an AI Research Answer Can Have Sources and Still Be Wrong

I'm building the research engine behind Prizston, an AI platform I'm developing to help people investigate customer problems and evaluate business opportunities.

One of the biggest challenges I've encountered isn't getting AI to produce an answer.

It's deciding whether that answer deserves to be trusted.

An AI system can return a convincing explanation, include several source URLs, and still reach a conclusion that the evidence doesn't support.

That's the problem I've been focusing on.

A real URL doesn't make a claim reliable

Consider a research assistant investigating whether a particular customer problem has become more common over the past year.

It finds several articles discussing the issue.

At first glance, everything looks fine. The sources exist, the content is relevant, and the model can summarize what it finds.

But then come the difficult questions:

When was each article originally published?
Was the article updated recently, or actually published recently?
Does the source support the conclusion, or merely mention the topic?
Are several articles independently reporting the same problem, or repeating one original claim?
What happens when two sources disagree?

Finding relevant information is only the beginning of research.

A small example: conflicting publication dates

One of the cases I'm testing involves extracting publication dates from source content.

Imagine that the same page contains two explicit publication statements:

Published on September 20, 2024.

Published on October 15, 2026.

Now imagine the system is evaluating whether that page qualifies as recently published on October 18, 2026.

A naive implementation might select the newer date and classify the page as recent.

But that would be a dangerous assumption.

The two dates conflict. Without additional reliable evidence, the system cannot confidently determine the actual publication date.

The safer result is:

not_determined

This is a deliberately constructed test case, not a claim about an actual article.

The principle matters beyond dates: when the evidence is contradictory, uncertainty should be preserved rather than silently resolved.

Separating observation from verification

I'm approaching the research engine as a sequence of different responsibilities.

1. Source observation

Retrieve content and identify potentially useful information, including dates, relevant passages, and links to supporting material.

At this stage, extracted information is an observation, not a verified fact.

2. Evidence evaluation

Determine what the source actually establishes.

For example, a customer complaint may demonstrate that one person experienced a problem. It doesn't automatically prove that the problem is widespread.

3. Answer validation

Before presenting a conclusion, check whether the available evidence supports it.

If the evidence is insufficient or contradictory, the answer should communicate that limitation.

These are design goals I'm working toward, not a claim that the entire research workflow is already solved.

The next question I'm exploring

A suggestion from a recent founder discussion gave me another useful testing idea.

Suppose the research engine produces a conclusion supported by three sources.

Now remove the strongest source.

Should the conclusion remain unchanged?

Should confidence decrease?

Or should the system withdraw the conclusion entirely?

I think this is a useful way to test whether an AI research system actually depends on its evidence rather than simply generating the same convincing explanation with different citations.

I haven't established the results of that experiment yet, but it's one I want to investigate.

What I'm learning

Building an AI research tool is making me appreciate how much work exists between retrieving a source and producing a trustworthy conclusion.

The hard part isn't always making the AI say more.

Sometimes it's making the system recognize when the available evidence isn't enough to say anything confidently.

I'm still working through these problems and testing individual components.

I'm curious how other developers approach this.

When building research agents or RAG systems, how do you test whether a generated conclusion is genuinely supported by its sources?

Disclosure: This article was drafted with ChatGPT assistance. The date example is an illustrative test case, not a production benchmark.

My background job failed silently for days. Here's how I catch it now.

Krasimir Petkov — Sat, 02 May 2026 17:42:41 +0000

I found out one of my background jobs had stopped running only after the data looked wrong the next day.

There was no dramatic crash. No big incident. The job just quietly failed, and I only noticed because something downstream looked stale.

That is the annoying part about cron jobs and scheduled scripts. Most of the time they run in the background, write some logs, and nobody thinks about them until something is missing.

I have a few jobs like this:

data updates
cleanup scripts
small imports
external API calls
recurring background tasks

None of them are very exciting. But when one of them does not run, or starts and never finishes, it can create a surprisingly annoying problem.

That is the kind of failure I wanted to make more visible.

I also built a small V1 of this idea here:

https://missedrun.com

There is also a self-hosted version for people who prefer to run this kind of monitoring on their own infrastructure:

missedrun / missedrun-selfhosted

Self-hosted cron job monitoring with ping URLs and email alerts.

MissedRun Self-hosted

Self-hosted cron and scheduled job monitoring for detecting silent failures.

MissedRun monitors recurring jobs such as cron scripts, backups, imports, ETL pipelines, billing syncs, cleanup tasks, and scheduled reports.

It works by giving each monitor a unique ping URL. Your job calls that URL when it runs, starts, finishes successfully, or fails. If the job does not check in within the expected interval plus grace period, MissedRun marks it as missing and can send an alert.

Hosted version: https://missedrun.com
Self-hosted version: https://github.com/missedrun/missedrun-selfhosted
License: AGPL-3.0

What problem does it solve?

Some production failures are not loud.

A job can stop running without throwing an exception. For example:

cron did not run
the server was down
a Docker container stopped
credentials expired before the job reached your alerting code
a backup script never started
an import stopped updating data
a scheduled report was not generated
a background worker silently stopped

…

View on GitHub

This is not a big launch. I am mostly trying to understand if this is a real enough problem for other developers who run cron jobs, ETL jobs, backups, imports, cleanup scripts, or other scheduled tasks.

The problem

Cron jobs are easy to forget about.

They usually do not have a UI. They run somewhere on a server, maybe write logs, and then disappear into the background.

A job can fail because:

an API token expired
an environment variable is missing
a database connection failed
the server restarted
the script crashed
the job started but never finished
the cron entry was changed or removed

Logs are useful, but only if you go and check them.

In practice, I usually only check logs after I already suspect something is broken.

For recurring jobs, I often want a much simpler answer:

did it start?
did it finish?
did it fail?
did it miss the expected time?

The ping approach

One simple way to monitor this is to make the job report its own status.

The basic pattern is:

send a start ping when the job begins
send a success ping when it finishes
send a failure ping if it crashes
mark it as late or missed if the expected ping does not arrive

It is not a complicated idea, but I have found it very useful in practice.

Instead of checking logs manually, the job tells you whether it is still alive.

For example:

if the start ping arrives, the job is running
if the success ping arrives, the job finished
if the fail ping arrives, the job crashed
if nothing arrives when expected, the job is late or missed

That last case is the important one for me.

A lot of failures are not loud. The job does not always send an error. Sometimes it just does not run.

Bash example

Here is a simple shell wrapper.

This uses placeholder URLs. In a real setup, these would be the ping URLs generated by your monitoring tool.

#!/bin/bash

START_URL="https://example.com/ping/YOUR_TOKEN/start"
SUCCESS_URL="https://example.com/ping/YOUR_TOKEN"
FAIL_URL="https://example.com/ping/YOUR_TOKEN/fail"

curl -fsS -X POST --max-time 5 "$START_URL" >/dev/null || true

your-real-command-here

EXIT_CODE=$?

if [ $EXIT_CODE -eq 0 ]; then
  curl -fsS -X POST --max-time 5 "$SUCCESS_URL" >/dev/null || true
else
  curl -fsS -X POST --max-time 5 "$FAIL_URL" >/dev/null || true
fi

exit $EXIT_CODE

Useful links

I also wrote a few short pages around the specific cases I keep running into:

Cron job monitoring guide: https://missedrun.com/cron-job-monitoring
Detect missed cron jobs: https://missedrun.com/missed-cron-job
Self-hosted cron monitoring: https://missedrun.com/self-hosted-cron-monitoring

Building a Real-Time Stock Scanner with Redis, WebSockets, and Postgres

Krasimir Petkov — Thu, 22 Jan 2026 17:32:33 +0000

Background

This project started as an internal tool to understand unusual volume and intraday activity across thousands of US stocks.

I recently built a real-time stock scanner that ingests thousands of symbols, processes live updates, and serves them to users with minimal delay. This post is a short breakdown of the architecture decisions that worked (and the ones that didn’t).

The Core Problem

Market data is:

high-frequency
bursty
time-sensitive

Writing every update directly to a relational database quickly becomes expensive and unnecessary — especially when most of the data is only relevant for seconds or minutes.

Architecture Overview

The system is split into three layers:

Ingestion — pulling snapshot and streaming data from external APIs
Processing — normalizing, filtering, and calculating derived metrics
Distribution — pushing live updates to the frontend

At a high level:

Redis handles hot, ephemeral data
Postgres stores validated, historical data
WebSockets deliver updates to clients in real time

Why Redis for Live Data

Redis turned out to be a much better fit than Postgres for intraday updates:

In-memory speed
Natural TTL support for expiring symbols
Pub/Sub for fan-out
Zero WAL / IO pressure

Postgres is still excellent — just not for data that changes every few seconds.

WebSockets for Distribution

Instead of polling:

active symbols are tracked dynamically
only “in-play” tickers receive updates
clients subscribe to exactly what they need

This keeps both bandwidth and server load under control.

What I’d Do Differently Next Time

Separate “hot” and “cold” data earlier
Treat intraday data as disposable by default
Avoid over-modeling early — schemas can come later

Live Demo

The project is live here:

https://zipston.com

It’s still evolving, but the core real-time pipeline is already doing the heavy lifting.

Closing Thoughts

Real-time systems are less about frameworks and more about respecting the nature of your data.

If your data expires quickly, your architecture should too.
Happy to answer questions or discuss real-time data tradeoffs.