DEV Community: Robson Müller

I built the "anti-Elementary" for dbt data quality and here's why

Robson Müller — Mon, 10 Aug 2026 15:08:24 +0000

Elementary is great. I use it. But after writing YAML for the 200th anomaly test, I asked myself: why am I teaching the tool what "normal" looks like when the data already knows?

So I built Scherlok — a zero-config data quality monitor that learns your data's patterns and detects anomalies automatically. No YAML. No thresholds. No rules to maintain.

It just shipped on the dbt Package Hub, so you can install it as a native dbt package today. Here's what makes it different, and when you should use which.

The core difference

Elementary asks you to define what's normal:

# schema.yml — per model, per test, per column
models:
  - name: fct_orders
    tests:
      - elementary.volume_anomaly:
          timestamp_column: created_at
          time_bucket:
            period: day
          sensitivity: 3

You configure the timestamp column, the time bucket, the sensitivity, the training period, the detection method. For every model. For every test. It's powerful, but it's work — and the configurations accumulate.

Scherlok inverts the model:

scherlok connect postgres://user:pass@host/db
scherlok investigate    # profiles everything
scherlok watch          # detects anomalies

Three commands. Done. No YAML, no per-model configuration, no timestamp columns to specify. Scherlok profiles every table and column automatically, learns the distributions, and flags when something deviates from the learned baseline.

An honest comparison

I'm not going to pretend Scherlok is better at everything. Here's where each tool wins:

	Elementary	Scherlok
Setup time	30-60 min (package + config per model)	5 min (connect + investigate)
Configuration	Granular per-model YAML	Zero (auto-discovers everything)
Detection methods	Multiple algorithms, configurable	Shewhart control limits (mean ± kσ)
dbt integration depth	Deep (runs inside dbt)	CLI + native dbt package
Dashboard	Beautiful hosted UI (Elementary Cloud)	Self-contained HTML file
Lineage-aware alerts	Yes	Yes (reads manifest.json)
AI explanations	No	Yes (Claude, opt-in)
MCP server	No	Yes (AI agents can run checks)
Non-dbt usage	Requires dbt	Works with or without dbt
Price	Free package, paid Cloud	Free, forever
Connectors	dbt adapters	Postgres, BigQuery, Snowflake, MySQL, DuckDB

Use Elementary when you want fine-grained control over every detection parameter, need the hosted dashboard for non-technical stakeholders, or your team already has the YAML config in place.

Use Scherlok when you want anomaly detection running in minutes without writing configuration, need it outside of dbt, or want AI-powered explanations on your alerts.

The dbt package — best of both worlds

As of v0.9.0, Scherlok ships as a native dbt package. This means you can use Scherlok's tests inside your existing schema.yml without leaving dbt:

# packages.yml
packages:
  - package: rbmuller/scherlok
    version: [">=0.9.0", "<1.0.0"]

# schema.yml
models:
  - name: fct_orders
    tests:
      - scherlok.volume_anomaly:
          sensitivity: 3.0
      - scherlok.row_count_between:
          min_value: 100
    columns:
      - name: email
        tests:
          - scherlok.not_null_proportion:
              max_rate: 0.01
      - name: updated_at
        tests:
          - scherlok.recency:
              days: 2

Six tests available:

Instant (no setup, works on first dbt test):

not_null_proportion — NULL rate exceeds threshold
row_count_between — row count outside bounds
recency — data is stale
unique_proportion — cardinality drops

Auto-learning (builds baseline over time):

volume_anomaly — row count outside Shewhart control limits
null_anomaly — NULL rate spike vs historical

The auto-learning tests use an incremental scherlok_metrics model that runs with dbt run and captures row counts per model. After 5 runs, the anomaly tests activate automatically. Before that, they pass silently — first runs are baseline, not false alarms.

The CLI — where Scherlok really shines

The dbt package is one distribution channel. The CLI is where the "zero-config" promise delivers the most value.

Connect to any warehouse

scherlok connect postgres://user:pass@host/db
scherlok connect bigquery://project-id/dataset
scherlok connect snowflake://account/database/schema
scherlok connect mysql://user:pass@host/db
scherlok connect duckdb:///path/to/file.db

Profile everything in one command

$ scherlok investigate

  Profiling 12 tables...
  ✓ users         — 45,231 rows, 8 columns
  ✓ orders        — 1,203,847 rows, 15 columns
  ✓ products      — 892 rows, 12 columns
  ...
  Done. Profiles saved.

Detect and alert

$ scherlok watch --webhook $SLACK_URL --explain

  🔴 CRITICAL  orders    volume_drop     Row count dropped 52%
  🟡 WARNING   users     null_increase   Column "email": NULL rate 2.1% → 18.7%

  💡 Hypothesis: The orders volume drop correlates with stg_orders
     upstream. The source table likely stopped receiving data from
     the payments API after the 2026-08-09 deploy.

The --explain flag is opt-in. When anomalies fire, Scherlok sends the anomaly batch (not your data — just the anomaly messages) to Claude for a root-cause hypothesis. On dbt projects, it includes upstream lineage so cascading failures get traced to the source model. Costs under $0.003 per run. If the API fails, the alert is delivered unchanged.

CI in one line

- run: scherlok ci $DATABASE_URL --fail-on critical --webhook $SLACK_URL

AI agents can use it directly

Scherlok ships an MCP server since v0.7.0. Claude Code or Claude Desktop can profile tables and detect anomalies without you typing commands:

{
  "mcpServers": {
    "scherlok": {
      "command": "scherlok-mcp",
      "env": { "SCHERLOK_CONNECTION": "postgresql://..." }
    }
  }
}

The agent gets list_tables, investigate, watch, status, history, and check as tools. Credentials stay server-side. Every operation is read-only.

Technical decisions

A few choices that might be interesting if you're building in this space:

Shewhart over ML. I chose Shewhart control limits (mean ± kσ) over fancy ML models. It's boring, debuggable, and transparent. When a test fires, you can explain why in one sentence: "row count was 3.2 standard deviations below the 30-day average." No black box. The tradeoff: no seasonality modeling. If your data has strong weekly patterns, you'll get false positives on Mondays. For most tables, Shewhart catches 90% of real incidents with zero tuning.

SQLite for profiles. Your baseline data lives in ~/.scherlok/profiles.db. You can inspect it with sqlite3, copy it to another machine, or sync it to S3/GCS/Azure for team use. No hosted service required.

Read-only by contract. Scherlok never writes to your warehouse. The test suite enforces this. Your DBA will appreciate it.

One repo, two channels. The same GitHub repo (rbmuller/scherlok) ships as a PyPI package (pip install scherlok) and a dbt package (package: rbmuller/scherlok). pyproject.toml and dbt_project.yml coexist at the root without conflicts.

When NOT to use Scherlok

Being honest about limitations:

Strong seasonality patterns — Shewhart control limits don't model day-of-week or seasonal cycles. You'll need to tune sensitivity higher or use Elementary's timestamp-aware algorithms.
Custom business rules — "revenue must match between systems A and B" is a rule, not an anomaly. Use dbt test or Great Expectations for that.
Non-technical stakeholders — Elementary Cloud's dashboard is built for people who don't live in the terminal. Scherlok's HTML dashboard is useful but developer-oriented.

Try it

pip install scherlok
scherlok connect YOUR_DATABASE_URL
scherlok investigate
scherlok watch

Or as a dbt package:

packages:
  - package: rbmuller/scherlok
    version: [">=0.9.0", "<1.0.0"]

Repo: github.com/rbmuller/scherlok
PyPI: pypi.org/project/scherlok
dbt Hub: hub.getdbt.com (pending approval)

MIT licensed. 344 tests. 5 connectors. Zero config.

Valor Software Medusa app and GitHub

Robson Müller — Thu, 12 Jan 2023 12:39:02 +0000

The Problem.

Here at Valor Software we had the challenge of analyzing some metrics from developers productivity. So, we've started questioning ourselves; What are the core daily activities of a developer? On a macro level we could say that is to deliver clean and reliable code on top of a consistent base, but with which commit frequency? And how to correlate code deployment with bugs and other stuff? Is this related to teams, to specific projects or even to the used technology stack?

In Data Science, we usually start our investigations based on the scientific method ( and CRISP-DM) approach), to know better the target situation, its surroundings and mainly doing the questions that will drive us to catch the root cause of our problems.

OK, so much fancy stuff. What is the relationship between all of this and the Valor Software Medusa project? We realized that GitHub is a good provider of data when it comes to developers productivity as we can track information about repositories, commits, tests and much more, this drove us to develop a Data Pipeline to extract and process data from it.

The Solution.

We have designed a business framework containing the daily core activities of a developer and have splitted it into four Pilars, when it comes to GitHub:

Version Control Management
Compatibility
Infrastructure
Team

The problem organization is a very important step in Data Science/Engineering projects as it gives us the direction of what data sources we should consume what metrics we are willing to build, as makes no sense to build a rocket dashboard, having metrics that are not related to the operation.

How it was built.

The idea is based in hitting the GitHub API, collecting the necessary metrics from a range of endpoints, Saving the result in an intermediary layer and than load it to the DataBase.

Why do we use an intermediary layer rather than saving it directly to the DataBase?

We usually follow this approach in order to make the pipeline more resilient, imagine we spent hours iterating over API pagination, and than some error occur, in some cases we can suffer data loss and have to restart it all again, saving the data in an intermediary layer such as AWS S3 or Google Storage make the pipeline to execute in steps, and also allows us to process the data later, use it in Data Science experiments and so on.

The application design is based in OOP and contains the following mechanisms:

App Deployment
Pipeline orchestration
Storage layer
Visualization layer
Infrastructure management
Data pipeline source code

App deployment

The application deployment is done using Docker and the containers needed to run Airflow with it's services are all described in a docker-compose file

Pipeline Orchestration

The triggering of the data ingestion and processing jobs can be done throughout the Airflow UI, which uses DAGs to manage all the working code (DAGs stands by Direct Acyclic Graphs and are responsible to manage the tasks of the data pipeline)

Storage Layers

There are two of them in this project, one as the intermediary layer, that stores the raw data from the API calls, organizing it into year/month/day of the request. The other one is the Data warehouse, a database based in PostgreSQL to store the tables containing the processed information.

Visualization layer

The chosen app for visualizing data at our data warehouse is Apache Superset it is an incredible tool considering it is free, and from my experience it has most of the features we can find the famous and paid Power BI. In addition, Superset is also ready for streaming needs and is cluster scalable.

Infrastructure management

The infra is deployed at the Google Cloud, and the necessary resources are created and managed by Terraform

Data pipeline source code

The code design is based in two strong objects which are responsible to interact with each other to ingest, process and write data from different data sources to different destinations, all based in json configuration files.

- Hook: Responsible to interface with external services, like the GitHub API, the Cloud storage (GCP) and the Data warehouse, holding its credentials and authentication methods.

- Operator: Responsible to operate (call) different methods on top of the data and trigger functions like:

Download data
Filter data
Calculate metrics
Upload logs

also holding configurations, restrictions and other information about the data object that is being ingested.

For the specific case of GitHub, the authentication is done using the account token. It can be a User or an Organization, resulting in a flexible object as User and Organizations have different API calls to retrieve similar categories of data. Just like in the representation bellow:

Once the data is requested from the API, based in the configuration file, it is stored in Google Cloud Storage, once the data is properly downloaded to this intermediary layer, the Operator calls the configuration file to filter the correct information from the raw data, opening a way to the next step: data transformation, and metrics calculation.

Conclusion

The creation of data pipelines can become something really complex if we do not care about details like the creation of generic functions, the config files approach, as per reuse code and make the processing more flexible.

So, at the end we have a data warehouse to consume data, making it available to the Medusa App and to the dashboard tool. This way, managers, product managers or PO's can create their own views, test their hypothesis or even find answers with the power of the data.

View of the data pipeline architecture