DEV Community

Cover image for Replacing fragile python3 -c scripts in CI with fimod
Gaëtan Montury
Gaëtan Montury

Posted on

Replacing fragile python3 -c scripts in CI with fimod

#python #devops #cicd #cli

🦀🐍 A small Rust CLI for Python-like data transforms without Python installed.

I keep running into the same kind of script in CI pipelines:

python3 -c 'import json,sys; data=json.load(sys.stdin); print(json.dumps(...))'
Enter fullscreen mode Exit fullscreen mode

It usually starts as a quick one-liner. Then it grows a condition. Then it needs to read YAML instead of JSON. Then someone has to escape quotes inside quotes inside YAML. Eventually it becomes either a small script committed somewhere, or another piece of shell glue that nobody really wants to maintain.

That is the problem I built fimod for. 🛠️

fimod is a small Rust CLI that transforms structured data - JSON, YAML, TOML, CSV, NDJSON, text lines - using Python-like expressions powered by Pydantic Monty, a Python runtime written in Rust.

No system Python is required. No pip install. No custom mini-language to learn if you already know basic Python. The standard binary is only a few megabytes - about 3.2 MB on my current Linux install - and still includes parsing, serialization, HTTP input, and a set of Rust-powered data helpers.

The goal is not to replace Python. The goal is to remove the boilerplate around small data transformations in CI, scripts, and config workflows.

📚 Website/docs: https://pytgaen.github.io/fimod/

⭐ Repository: https://github.com/pytgaen/fimod

In short:

  • 🐍 write small transforms with Python-like expressions;
  • 🦀 ship them as a small Rust binary;
  • 📦 read/write JSON, YAML, TOML, CSV, NDJSON and text;
  • 🔁 keep CI data plumbing explicit and shell-friendly.

🧪 A tiny example

Suppose a CI job has a JSON file like this:

[
  {"name": "api", "active": true},
  {"name": "worker", "active": false},
  {"name": "web", "active": true}
]
Enter fullscreen mode Exit fullscreen mode

With a regular Python one-liner, the useful logic is often hidden inside JSON loading, stdin handling, output formatting, and shell escaping.

With fimod, the parsed input is already available as data:

fimod s -i services.json \
  -e '[svc for svc in data if svc["active"]]' \
  -o active-services.json
Enter fullscreen mode Exit fullscreen mode

The expression is just the transformation. fimod handles reading, parsing, serializing, and writing.

✅ Validate config in CI

A common CI task is checking that a config file contains the fields a deployment
expects.

fimod s -i deploy.yaml \
  -e 'all(k in data for k in ["image", "replicas", "port"])' \
  --check
Enter fullscreen mode Exit fullscreen mode

--check suppresses stdout and uses the truthiness of the result as the process
exit code:

  • exit 0 if the result is truthy;
  • exit 1 if the result is falsy.

That makes it easy to use in shell scripts and CI jobs:

fimod s -i deploy.yaml \
  -e 'all(k in data for k in ["image", "replicas", "port"])' \
  --check || {
    echo "Invalid deploy config" >&2
    exit 1
  }
Enter fullscreen mode Exit fullscreen mode

For more explicit error messages, fimod also provides validation helpers such as
gk_assert:

fimod s -i deploy.yaml -e '
def transform(data, **_):
    gk_assert("image" in data, "missing image")
    gk_assert("replicas" in data, "missing replicas")
    gk_assert("port" in data, "missing port")
    return True
' --check
Enter fullscreen mode Exit fullscreen mode

gk_assert reports each missing field on stderr and sets a non-zero exit code on failure. The final return True makes the successful case truthy for --check, while --check keeps stdout silent.

🔀 Convert formats without changing the logic

Because parsing and serialization are handled by fimod, the same expression can work across different input formats.

YAML to TOML:

fimod s -i config.yaml -e 'data' -o config.toml
Enter fullscreen mode Exit fullscreen mode

CSV to JSON, with an explicit cast for age:

fimod s -i users.csv \
  -e '[{**row, "age": int(row["age"])} for row in data]' \
  -o users.json
Enter fullscreen mode Exit fullscreen mode

CSV values arrive in fimod as strings, so int(row["age"]) is intentional.
Modern yq can do CSV-to-JSON very well too - for example
yq -p=csv -o=json '.' users.csv - so this is not meant as a "yq cannot do this" example. The point is that when the transformation grows, the logic stays in familiar Python-like syntax.

Extract one value for a shell variable:

VERSION=$(fimod s -i package.json \
  -e 'data["version"]' \
  --output-format txt)
Enter fullscreen mode Exit fullscreen mode

The point is not that these examples are impossible with other tools. They are not. The point is that the transformation stays focused on the data, while fimod handles the repetitive I/O details.

🧰 What about jq and yq?

jq and yq are excellent tools. If your team already knows them and they solve the problem cleanly, keep using them.

fimod is useful when you prefer Python-like expressions and want the same command shape across JSON, YAML, TOML, CSV, NDJSON, and text. It is not meant as a "yq cannot do this" argument; it is a different tradeoff.

🧱 Reusable transforms: molds

One-liners are convenient, but CI pipelines eventually grow shared logic.

In fimod, a reusable transform is called a mold. It is a Python-like script with a transform(data, **_) function:

# cleanup.py
def transform(data, **_):
    for row in data:
        row["name"] = row["name"].strip().title()
        row["email"] = row["email"].strip().lower()
    return data
Enter fullscreen mode Exit fullscreen mode

Run it with:

fimod s -i contacts.csv -m cleanup.py -o contacts.json
Enter fullscreen mode Exit fullscreen mode

Molds can also receive parameters:

# filter_by_field.py
def transform(data, args, **_):
    field = args["field"]
    value = args["value"]
    return [row for row in data if row.get(field) == value]
Enter fullscreen mode Exit fullscreen mode 
fimod s -i users.json \
  -m filter_by_field.py \
  --arg field=role \
  --arg value=admin
Enter fullscreen mode Exit fullscreen mode

The really practical part is that molds are not just local scripts.
They can be shared through registries and referenced by name with @.

For example, a CI job can point fimod at a local shared mold directory:

FIMOD_REGISTRY=./molds \
  fimod s -i contacts.csv -m @cleanup -o contacts.json
Enter fullscreen mode Exit fullscreen mode

Or at the official fimod example registry on GitHub, and call a real published mold by name:

FIMOD_REGISTRY=https://github.com/pytgaen/fimod/tree/main/molds \
  fimod s -i contacts.csv \
    -m @pick_fields \
    --arg fields=name,email \
    -o contacts-public.json
Enter fullscreen mode Exit fullscreen mode

In a real team setup, that registry can be your own Git repository. The important bit is that every project can call the same reviewed transform by name instead of copying another tiny script into every repo.

That is where molds become more than a convenience: they become shared data recipes for CI. I will cover registries more deeply in a separate article, but even the basic @name pattern is already useful.

🐍 Why not just use Python?

For many tasks, you should use Python.

But plain Python is not a unified structured-data CLI out of the box. JSON and CSV are in the standard library, but YAML usually means an extra dependency, TOML support depends on the Python version, and NDJSON, format detection, stdin/stdout plumbing, shell-friendly output, and cross-format conversion all add glue code.

That glue is fine in a real application. In a CI job, it often turns a small transformation into dependency management, quoting, and boilerplate.

fimod is useful when the script is small enough that a full Python file feels heavy, but important enough that a quoted python3 -c command is painful.

✨ What I am not covering here

fimod also has HTTP input, MiniJinja-powered templating for data-to-text outputs, and built-in helpers for regex, nested fields, grouping, hashing, validation, diagnostics, and environment substitution. The regex helpers are backed by Rust's fancy-regex crate, with PCRE-like features such as lookahead, lookbehind, backreferences, and named captures.

That means later articles can cover things like fetching an API response, extracting fields, hashing sensitive values, or rendering .env, Markdown, Dockerfile, and config files from structured data.

Those deserve separate articles. This one is intentionally focused on the first use case: replacing fragile CI one-liners and sharing transforms as molds.

⚠️ Important limitations

fimod is early-stage software.

A few things are intentionally not hidden:

  • fimod runs on Monty, not CPython;
  • it does not support arbitrary PyPI packages;
  • molds cannot freely access the filesystem or the network;
  • the project is maintained part-time;
  • the API and behavior may still evolve as Monty and fimod mature.

Those constraints are part of the design. fimod is not trying to be a general Python runtime. It is trying to be a practical data-shaping CLI with explicit inputs and controlled execution.

🚀 Try it

Install:

curl -fsSL https://raw.githubusercontent.com/pytgaen/fimod/main/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Run a first transform:

echo '[{"name":"Alice"},{"name":"Bob"}]' | fimod s -e 'len(data)'
Enter fullscreen mode Exit fullscreen mode

📚 Website/docs:

https://pytgaen.github.io/fimod/

⭐ Repository:

https://github.com/pytgaen/fimod

If you try fimod, I would start with one small, boring transform in a CI job or config workflow. 🧪

  • ⭐ If the idea looks useful, a GitHub star helps the project be discovered.
  • 💬 If you have a quick reaction, a comment here on dev.to is perfect.
  • 🐛 If something feels unclear - the install, the command shape, the docs, the error message, or a small improvement idea - a short GitHub issue with the you tried is more than enough.

Top comments (0)