poolion

Posted on Aug 20

Schema Checker CLI: Validate JSON/YAML Config Files Against Required Fields in Python

#productivity #python #tooling #opensource

Schema Checker CLI: Validate Configuration Files Without External Dependencies

Configuration management is a critical part of any application deployment. Whether you're managing database URLs, API endpoints, or feature flags, ensuring your config files are properly defined before production can prevent runtime errors.

The Schema Checker CLI is a pure Python tool that validates JSON and YAML configuration files against required fields—without requiring external dependencies like jsonschema or validators.

What It Does

The tool checks:

Required field presence: Ensures all @field patterns are defined in the config
Empty value detection: Reports when @field variables have null/undefined values
Format validation: Properly handles numbered keys like DATABASE_01, DATABASE_02 etc.

All validation uses pure Python and regex pattern matching—no heavy JSON/YAML parsing libraries needed.

Installation

# Clone the repository
git clone https://github.com/Poolion/schema-checker-cli.git
cd schema-checker-cli

# Run directly with python3 (no install needed)
python3 schema-checker.py your-config.json

# Or add to PATH
cp schema-checker.py /usr/local/bin/

No dependencies required—this uses only Python's standard library.

Usage Examples

Basic Validation

python3 schema-checker.py app.config.json

Validates that the config file has properly defined configuration keys.

Check from Stdin

echo '@database: postgres://localhost:5432' | python3 schema-checker.py -

Reads configuration patterns directly from stdin—useful for scripting and CI/CD pipelines.

Specify Required Fields

python3 schema-checker.py configs/*.json \
  -r name,url,version,@DATABASE_URL

# Multiple required fields (use -r multiple times)
python3 schema-checker.py myapp.json \
  -r name \
  -r version \
  -r @API_ENDPOINT

The -r flag takes one field name per use. Prefix with @ to indicate an environment-variable-like pattern.

Handle Numbered Keys

The tool properly handles patterns like:

DATABASE_01_HOST: localhost
DATABASE_02_HOST: db-replica.internal

# These are recognized as distinct valid fields

How It Works

Under the hood, the validation uses:

Regex pattern matching: Finds @field references in raw text without full parsing
Simple line-by-line scanning: Efficient for large config files
No JSON/YAML dependencies: Avoids heavy libraries like PyYAML

Key Functions

The core validation logic is implemented in find_empty_values():

def find_empty_values(content):
    import re
    empty_keys = []

    for line_num, line in enumerate(content.splitlines(), 1):
        stripped = line.strip()

        # Skip comments and blank lines
        if not stripped or stripped.startswith('#'):
            continue

        # Match @field patterns with empty values
        match = re.match(r'(@[\w\-\.]+)\s*[:=]', stripped)
        ```
{% endraw %}


This approach is memory-efficient and quick—ideal for scanning hundreds of config files in CI/CD pipelines.

### Finding References

The {% raw %}`find_references()`{% endraw %} function scans for various field naming conventions:
{% raw %}


```python
def find_references(content):
    import re

    refs = set()

    # Match $@NAME, @$NAME, or bare @NAME patterns
    refs.update(re.findall(r'(?:\$)?\s*@(\w+)', content))

    return refs

This handles multiple common shell/config syntax styles.

Why Build This?

Existing Python config validators often require:

jsonschema: Requires installation and schema definition files
PyYAML: Adds a dependency just to read the file
validators: More complex setup for minimal functionality

The Schema Checker CLI solves the common case—checking that environment-variable-like patterns are defined—using pure Python. This makes it:

Drop-in replaceable in existing pipelines
Faster startup with zero imports overhead
Simpler CI/CD integration (works anywhere Python runs)

Real-World Use Cases

Pre-deployment Validation

#!/bin/bash
# Check configs before deployment
for config in /etc/myapp/*.json; do
    python3 schema-checker.py "$config" || \
        echo "Skipping $config due to validation errors"
done

CI/CD Integration

Add this step to your pre-deployment checks:

# .github/workflows/deploy.yml
- name: Validate configurations
  run: |
    python3 schema-checker.py app.config.json \
      -r name,url,version,@DATABASE_URL || exit 1

Environment Variable Checking

Check that your .env files or shell scripts define all expected variables:

# In your CI pipeline
python3 schema-checker.sh .env.example -r @DB_HOST>@PORT>>@SECRET_KEY

Limitations

This tool focuses on the common case of field reference checking. It doesn't:

Perform full JSON/YAML semantic validation (that requires a parser)
Check type constraints beyond "defined vs undefined"
Generate schema files automatically

For those use cases, consider tools like PyYAML or jsonschema—but this fills the gap for simple field-definedness checks.

Support

If you build configs that benefit from quick field validation, please support development: https://www.buymeacoffee.com/poolion

DEV Community