DEV Community: Zephico Technologies

When Retool is the right answer (and when it isn't)

Zephico Technologies — Wed, 29 Jul 2026 00:02:20 +0000

We're Retool-certified and we build internal tools with it for clients — so you'd expect us to say Retool is always the answer. It isn't, and knowing where the line sits is most of the value of working with people who use it daily.

Where Retool wins, clearly

Ops consoles over existing data. Refund workflows, order lookups, inventory corrections, support tooling — anything where the data already lives in PostgreSQL, an API, or a warehouse, and the "app" is really forms, tables and buttons with permissions. A tool your ops team needs would take a product squad six weeks in React; in Retool it's days, and the six weeks were never going to be approved anyway. That's the honest comparison: Retool doesn't usually replace a custom build, it replaces nothing getting built.

Approval workflows with audit requirements. Role-based access, audit logs, and SSO come with the platform. For teams facing GDPR requests or SOC 2 audits, replacing "shared spreadsheet plus direct database edits" with an audited tool is often the entire business case.

Tools with fewer than a couple hundred internal users. Retool's pricing and performance are built for internal scale, not consumer scale.

Where we talk clients out of it

Customer-facing anything. The moment external users log in, you're fighting the platform on branding, latency, licensing and auth. Build customer-facing surfaces with a real framework.

Heavy client-side logic. If your "internal tool" is actually an interactive editor, a scheduling engine, or anything with complex client-side state, the visual builder becomes the bottleneck. You'll write thousands of lines of JavaScript in string fields and wish you had a repo. (Some of it can live in modules and version control now, but the gravity is still wrong.)

Core business workflows you'll iterate on for years. Retool is fast to build and fine to maintain, but a workflow that is the business — underwriting, dispatch, pricing — eventually deserves the testability, review culture and hiring pool of a conventional codebase. We've migrated tools out of Retool for exactly this reason, and that's fine: it earned its keep for two years first.

The pattern that works

Start the workflow in Retool while it's still changing weekly. When it stabilizes and hardens into the business, graduate it to a custom build with the requirements now fully known. The Retool version becomes your working spec — the cheapest, most accurate spec you'll ever write.

The wrong question is "Retool or real code?" The right one is "how much certainty do we have about this workflow?" Low certainty, move fast in Retool. High certainty and high stakes, invest in code. We're happy building either — which is exactly why you can trust the recommendation.

Originally published on the Zephico blog.

How to make contract engineers actually work: lessons from the other side

Zephico Technologies — Wed, 29 Jul 2026 00:01:59 +0000

We place contract engineers into teams across North America and Europe, which means we've watched the same engagement succeed at one company and stall at another — with the same engineer. The difference is almost never the engineer. It's a handful of structural choices the client makes in the first two weeks.

Treat week one as an investment, not a cost

The engagements that compound all look the same at the start: the contract engineer gets a working dev environment on day one, a real (small) ticket in the first week, and a named person to ask questions of. The ones that stall spend three weeks "getting access sorted" while the engineer bills hours reading a wiki.

If your security process takes two weeks to provision access, start it before the start date. This sounds obvious. It is skipped constantly.

Give outcomes, not tickets

A contract engineer who receives pre-chewed tickets performs like a junior no matter how senior they are, because all the judgment was spent by whoever wrote the ticket. The teams that get senior output hand over problems: "our nightly pipeline overruns into business hours — own it." Then the engineer's experience actually gets used, and the interesting decisions surface in review where your team can see the reasoning.

Timezone offset is a feature if you design for it

With a team in India and a client in New York, there are roughly four hours of overlap and twenty hours of relay. Teams that fight this — insisting on full-day synchronous presence — burn out the engineer and get the worst of both worlds. Teams that design for it get a genuine advantage: work specced in the client's afternoon is running by their next morning.

The design is simple: overlap hours are for decisions (standups, reviews, pairing on anything ambiguous), non-overlap hours are for execution, and everything decided in a call gets written down because someone will act on it eight hours later.

Measure integration, not utilization

The metric that predicts a successful engagement isn't hours billed — it's how quickly the contractor becomes indistinguishable from the team in your tools. Are they reviewing your engineers' PRs by week three? Are they in the incident channel? Did someone ask them a question this week? If after a month the contractor is still a ticket-taker on the outside of every discussion, the engagement is failing regardless of what the timesheet says.

Say the quiet part in the contract

Notice periods, IP assignment, who owns the laptop, what happens to access on the last day — none of this is awkward if it's written down at the start, and all of it is awkward later if it isn't. A staffing partner who resists clear contract terms is telling you something.

Most of this list is just "treat contract engineers like engineers." The companies that do get senior output within days of the start date — which is the entire reason they went with contractors instead of a six-month hiring pipeline.

Originally published on the Zephico blog.

Databricks Workflows vs Airflow vs Dagster: Picking an Orchestrator

Zephico Technologies — Tue, 28 Jul 2026 21:13:07 +0000

Every data team eventually asks the same question: what runs our pipelines, on what schedule, with what retry logic, and who gets paged when it fails. The answer used to default to Airflow because there wasn't a real alternative. Now there are three reasonable defaults, and they optimize for different things. Picking wrong doesn't break anything on day one — it shows up eighteen months later as either an operations team drowning in scheduler maintenance or an engineering team fighting a platform that won't do what they need it to.

Here's the actual tradeoff, not the vendor pitch version.

Databricks Workflows: the path of least resistance, if you're all-in on Databricks

Databricks Workflows is the orchestrator built into the platform. Jobs, clusters, Unity Catalog permissions, and Workflows all share the same control plane, which means you're not maintaining a separate scheduler, not managing a second set of credentials, and not debugging why an external system can't see a table that Unity Catalog says it can. Task dependencies, retries, cluster reuse across tasks, and job-level alerting all come for free.

The cost is exactly what you'd expect from a platform-native tool: it orchestrates Databricks well and everything else poorly. There's no first-class way to trigger a task in your orchestration DAG that waits on a Salesforce export, calls an internal API, or coordinates a dbt run against a warehouse that isn't Databricks SQL. You can bolt these in with webhooks and external scripts, but you're fighting the tool rather than using it. Workflows also doesn't give you the asset-lineage or testing story that Dagster does — it schedules tasks, not data assets.

If your data platform genuinely is Databricks end to end — ingestion, transformation, ML, serving — Workflows removes an entire category of operational overhead you'd otherwise be paying for nothing. Teams in this position who reach for Airflow anyway usually do it out of habit, not need, and end up running two schedulers where one would do.

Airflow: the incumbent, and the bill that comes with it

Airflow's argument is maturity and reach. It has the largest operator ecosystem of any orchestrator, a decade of production hardening, and a hiring pool that already knows it. If your pipelines touch a dozen systems — a couple of warehouses, three SaaS APIs, an on-prem database, a message queue, and Databricks somewhere in the mix — Airflow (or something with equivalent breadth) is close to mandatory, because it's the only one built from the ground up to sit above heterogeneous systems rather than inside one of them.

What Airflow charges you for that breadth is operational weight. Self-hosted, you're running a scheduler, a metadata database, a web server, and workers, and you own upgrades, DAG-parsing performance as the DAG count grows, and the sharp edges around task isolation. Managed options — MWAA on AWS, Astronomer, Google's Cloud Composer — remove the infrastructure burden but add real cost and still leave you managing DAG deployment and dependency versioning yourself. Airflow's execution model is also fundamentally task-centric: it knows a task ran, not what data it produced or whether that data is fresh, valid, or the same asset another DAG also writes to. Teams that want a data-asset-first mental model layer tools like dbt or add asset-check conventions on top to compensate.

Airflow 3 narrowed some of these gaps — better task isolation, some asset-aware scheduling — but the operational model is still: you run this, or you pay someone to run it for you.

Dagster: asset-based thinking, at the cost of ecosystem size

Dagster starts from a different premise: instead of orchestrating tasks, you declare the data assets your pipeline produces and the dependencies between them (software-defined assets). The scheduler then figures out the DAG from the asset graph. In practice this means significantly better local development — you can materialize a single asset, mock inputs, and test it in isolation without spinning up the whole pipeline — and lineage that's a natural byproduct of how you define pipelines rather than something you bolt on afterward. For teams that have been burned by Airflow DAGs that pass silently but produce garbage data, Dagster's typed I/O and asset-check model is a meaningful upgrade in confidence.

The tradeoff is ecosystem size. Dagster's integration library is smaller than Airflow's, the community and hiring pool are both younger, and if you need an obscure operator for a legacy system, you're more likely to be writing it yourself than pulling it off the shelf. Dagster also orchestrates across heterogeneous systems fine — it's not Databricks-specific — but you're trading Airflow's breadth for a better development model, not getting both for free.

The real decision framework

Ignore the marketing angle from each vendor and the decision comes down to two questions: how much of your stack lives outside Databricks, and how much do you value asset-centric thinking and local testability over ecosystem breadth.

Single-platform-on-Databricks shop, most pipelines are notebooks and Databricks jobs. Use Workflows. You'll be maintaining fewer systems and every feature you'd otherwise build (retries, alerting, cluster lifecycle) is already there.
Orchestrating across many heterogeneous systems — multiple warehouses, SaaS APIs, on-prem systems, Databricks as one node among several — Airflow's breadth usually wins, especially if your team already knows it or you're prepared to pay for a managed instance.
Same heterogeneous situation, but you're building new and value strong typing, asset lineage, and a real local dev/test loop over ecosystem size — Dagster deserves a serious evaluation, particularly for greenfield platforms where you're not migrating years of existing DAGs.

The honest answer really is "it depends on your existing stack," and any framework that skips that and hands you one universal answer is selling something. The one mistake we see repeatedly is teams running Airflow purely to orchestrate Databricks jobs, paying the full operational cost of a general-purpose scheduler to do a job Workflows does natively — that's worth a hard look before you commit to a migration or a managed Airflow contract.

If you're mid-evaluation and want a second opinion on which of these fits your actual pipeline mix, our Databricks data engineering team has built and migrated orchestration on all three. Get in touch and we'll walk through your stack with you.

Originally published on the Zephico blog.