DEV Community: Philip Hern

cursor automations for housekeeping and hygiene

Philip Hern — Mon, 29 Jun 2026 22:13:22 +0000

quick answer

cursor automations are a good fit for recurring housekeeping and hygiene tasks that already have a clear definition of done. i currently use them for weekly dependency and vulnerability review, documentation refreshes when a pull request opens, stale issue triage, release-note drafts, and other work where the agent can inspect context, propose a small change, and leave a reviewable trail.

the caveat is cost. automations still run cloud agents, and cloud agents still count against your cursor usage. use them where the recurring value is obvious, not where a cheap calendar reminder would do the job.

who this is for

audience: developers, data engineers, and technical leads who already use cursor agents
prerequisites: a repository with repeatable maintenance work and enough tests or checks for validation
when to use this guide: when you want automation to keep your repo cleaner without adding another dashboard or weekly manual checklist

why this matters

housekeeping is the work everybody agrees matters until the calendar fills up.

dependencies drift. docs fall behind. stale branches hang around. pull requests change behavior without updating the right README, runbook, or architecture note. vulnerability review becomes a once-in-a-while scramble instead of a normal operating rhythm.

cursor automations are interesting because they let the same agent workflow you already use in the editor run in the background. an automation can start on a schedule, a pull request event, a slack message, a webhook, or another supported trigger. it can read the repo, use selected tools, produce a summary, comment on a pull request, or open a pull request when that is appropriate.

that makes automations especially useful for work that is important, repetitive, and bounded.

the shape of a good housekeeping automation

a good automation is not just "go clean up the repo".

it has a trigger, a boundary, a checklist, and an output rule. for example, a weekly dependency hygiene automation should know which package files to inspect, which update paths are allowed, which checks must pass, and when to stop with a summary instead of opening a pull request.

i like this mental model:

trigger: when should the agent run
scope: which repo, branch, folder, or pull request should it inspect
tools: which actions should it be able to take
definition of done: what counts as a useful result
stop rule: when should it leave a summary and avoid changing anything

that last part matters. recurring agents need restraint. without a stop rule, a housekeeping automation can turn into a noisy robot that opens low-value pull requests, comments too often, or burns usage on work nobody needed.

reuse the instructions already in the repo

one of the easiest ways to make an automation better is to point it at the working knowledge already committed in the repo.

if you already have cursor skills, project rules, validation scripts, maintenance runbooks, or contributor docs, do not rewrite all of that inside the automation prompt. tell the automation to read and follow the existing source of truth first. that keeps the automation short, reduces duplicated instructions, and makes future updates easier because you can improve the skill or runbook once instead of editing every automation that copied it.

this is especially useful for housekeeping work. a dependency automation can follow the same dependency-review skill a human agent would use. a documentation automation can follow the repo's documentation-audit skill or authoring guide. a release hygiene automation can reuse the same validation commands and release checklist already used in normal development.

the trick is to treat existing repo context as the playbook and the automation prompt as the trigger wrapper. the prompt should say what starts the run, what outcome is expected, and which existing instructions to apply. the detailed standards can live in the repo where they are versioned, reviewed, and updated with the rest of the codebase.

example 1, weekly dependencies and vulnerabilities

once per week, an automation scans dependency files, checks for vulnerable packages, looks for safe version bumps, and decides whether to open a pull request. the important part is not that the agent updates everything. the important part is that it applies judgment to a small maintenance lane.

the prompt should be conservative:

### Task Description

Run the dependency audit agent using `your_review_agent_here.md`. Aggregate the results, classify identified risks, and safely apply low-risk updates with strict validation before drafting a summary PR.

### Step-by-Step Instructions

1. **Execute Audit:** Run the parallel audit workflow defined in `your_review_agent_here.md`.
2. **Aggregate & Classify:** Collect all dependency findings and categorize them by risk level (Low, Medium, High).
3. **Apply Low-Risk Updates:** Apply **only** the low-risk dependency updates to the codebase.
4. **Validate Changes:** Run the repository's test suite and build validation checks to ensure no regressions were introduced by the low-risk updates.
5. **Create Draft PR:** Open a draft Pull Request against the `main` branch containing the applied updates and the full, aggregated audit report in the description.

the automation should not invent confidence. it should run the same checks a human reviewer expects.

example 2, documentation on pull request open

another strong use case is documentation hygiene when a pull request opens.

the trigger is simple. when a pull request is opened, the automation reviews the diff and asks a small set of questions.

did this change add or remove a public behavior
did it change configuration, setup, permissions, deployment, or data contracts
did it introduce a new concept that belongs in a README or docs page
did it change something already documented elsewhere

the output does not always need to be a commit. in many cases, the best output is a pull request comment that says, "this probably needs a docs update", with the exact files or sections that look stale.

when the documentation gap is small and obvious, the automation can open a companion pull request or push to the current branch if that is how your team wants the workflow to behave. when the gap requires product judgment, it should stop and ask for a human decision.

example 3, repo hygiene summaries

not every automation needs to change files.

a weekly or monthly repo hygiene summary can look for stale branches, old draft pull requests, failing scheduled checks, large generated files, todos that accumulated in touched areas, or ignored validation failures. the value is not that the agent fixes everything. the value is that it turns hidden decay into a short reviewable note.

this is especially useful for solo maintainers and small teams. it gives you a light operating rhythm without creating a full process around maintenance.

write prompts like operating instructions

for automations, vague prompts are expensive.

when i write an automation prompt, i try to include:

what to inspect
what to ignore
what the agent may change
what validation to run
what output format to use
what requires human approval
what to do when checks fail

the prompt does not need to be long. it needs to be specific enough that a future run does the same kind of work as the current run.

use wisely because usage is real

cursor automations are not free background magic. cursor documents automations as cloud agents, and cloud agents are billed based on usage. automations also run in max mode because they run as cloud agents.

that does not mean "avoid them".

it means reserve them for jobs where the background run is worth the spend. a weekly dependency review might be worth it because it replaces a recurring manual chore and reduces risk. a pull request documentation check might be worth it because it catches drift at the right moment. an hourly "summarize my repo" automation probably is not worth it unless someone actually uses the summary.

my rule of thumb is simple. start with low frequency, high signal, and clear stop conditions. if the automation saves time or catches issues, keep it. if nobody reads the output, turn it off.

faq

should every maintenance task become an automation?

no. automate tasks that are recurring, bounded, and reviewable. if the work requires taste, prioritization, or sensitive business judgment every time, use the automation to surface context rather than make the final decision.

should automations open pull requests automatically?

only when the change is small and the validation path is clear. dependency patch updates, generated docs refreshes, and simple formatting fixes can be good candidates. broad refactors and policy decisions should usually produce a summary or comment first.

what should i build first?

start with one weekly housekeeping automation. dependency and vulnerability review is a good first candidate because the trigger is simple, the value is clear, and the output can be limited to either a small pull request or a short no-action summary.

references

what room to breathe makes room for

Philip Hern — Sat, 27 Jun 2026 19:00:48 +0000

thesis

when i wrote about room to breathe, i framed the pause mostly as defense. a way to step off the improvement treadmill before it ground me down, and a way to actually understand what i shipped before piling the next change on top of it. that is true, but it is only half the story. the space i protect does not sit empty. it is exactly where the bigger, slower ideas finally get enough air to form. the quiet heads-up i gave about something longer only became possible because i stopped filling every gap with one more optimization.

context

i sold room to breathe as rest, and rest is real, but something else happened once i actually started leaving the space. it did not stay quiet for long. the moment i was not pouring every spare cycle into squeezing the last few points out of a thing that already worked, my mind wandered somewhere it never had room to go before. it drifted toward the ideas that never fit in a single sitting, the ones i kept trimming to fit a post and kept losing the heart of. that drift is where the longer thing i hinted at recently started to take shape.

argument

the pause is not idle

the part i underrated is that breathing room is generative, not just restorative. when i stop cramming the schedule, my attention does not evaporate, it goes looking for something to chew on. left alone for a minute, it reaches past the small, urgent, well-defined tasks and starts circling the big, vague, interesting ones. the same pause that keeps me from burning out is the pause where the slow ideas surface. i do not get those ideas while sprinting. i get them in the gap after the sprint, when there is finally enough quiet for them to be heard.

small things crowd out big things

here is the trap. a small improvement always feels cheaper and more available than a big, half-formed idea, because the small one has a clear payoff and the big one does not yet. ai makes this worse, since the supply of small "you could make this a little better" suggestions is now infinite and nearly free, a trap that stays permanently stocked. if i never stop, the small things win every time, because there is always one more of them with a clearer return than the long shot. the big idea never gets a turn. room to breathe is what finally gives it a turn. the cost of every marginal optimization was never only my energy, it was the bigger thing that never got the cycles.

the bigger the idea, the longer the runway

the longer thing i mentioned needs time to think long before it needs time to make. you cannot rush the thinking. an idea that has to unfold in order, one piece building on the next, needs unhurried hours that a packed schedule simply cannot produce. that is the kind of work that only grows in protected space. i can give it that space now only because i first gave my own schedule room to breathe, the same way i try to build anything worthwhile, little by little and by sticking with it when the payoff is still far off.

tension or counterpoint

the obvious objection is that this is just a tidy story i tell myself to excuse slacking off, or worse, daydreaming about some grand project instead of doing the small work in front of me. fair. the honest version is that room to breathe only turns into something bigger if i actually use the space to think, not to scroll. it is not the absence of work, it is a quieter kind of work, the kind that produces nothing visible for a long time and then produces the thing that mattered most. the discipline is in protecting the space on purpose and then actually spending it, instead of letting the infinite small stuff flood back in.

closing

so this is the other half of room to breathe. the pause is not the end of the story, it is the start of the next one. ship it, let it settle, and in the quiet that follows, the bigger thing gets enough air to start growing. i am still not going to say what that thing is yet, for the same reasons i gave last time. i am only pointing at the mechanism. give something room to breathe, and do not be surprised when something larger walks into the space you made.

related on this site

something longer is coming

Philip Hern — Wed, 17 Jun 2026 12:26:02 +0000

thesis

i have spent a while now writing short reflections here, a post at a time, each one a single thought i wanted to get out of my head and onto the page. that format has been good to me, but some ideas do not fit in a single sitting. they need room to unfold, one piece building on the next. so i am starting to write long-form, books and guides released a chapter at a time, and this is the quiet heads-up before the first one arrives.

context

almost everything on this site so far has been short-form, a reflection, a lesson, or a technical note i wanted to remember. that suits most of what i think about, because most of what i think about fits in a page or two. but a few ideas keep circling back, and every time i try to squeeze one into a single post i end up cutting the parts that actually matter. those are the ideas that need a different shape. i have quietly been writing one of them for a while, and it has grown well past the point where a post could ever hold it.

argument

why a book and not more posts

a post is a snapshot. it captures one thought at one moment, and that is exactly why it works for most of what i write. a book is a structure. it lets an idea build in sequence, where each chapter leans on the one before it and sets up the one after. some things only make sense laid out in order, slowly, with the connective tissue left in instead of cut for length. when i kept trimming an idea to fit a post and kept losing the part that mattered most, that was the signal it belonged somewhere longer.

a chapter at a time

i am not going to drop a finished book in one go. i am going to release it the same way i do most things, little by little, one chapter at a time as each one is ready. that cadence keeps the work sustainable and keeps me honest, because a chapter that has to stand on its own cannot hide behind the ones around it. it is the same reason i keep telling myself to stick with it on any long effort, and the same reason i try to leave room to breathe between pushes instead of sprinting until i break.

what i am not saying yet

i am deliberately not going to tell you what the first one is about, at least not yet. part of that is plain superstition, the sense that naming a thing too early and too loudly is a good way to talk myself out of finishing it. part of it is that i would rather let the work introduce itself when it is ready than oversell an idea i am still shaping. so for now the only promise is the form. longer pieces, built to be read in order, arriving one chapter at a time. the subject can wait until the first chapter is able to speak for itself.

tension or counterpoint

the honest doubt is obvious. announcing something before a single chapter is finished is a good way to look foolish, and plenty of people announce books that never arrive. so why say anything at all before the work is done? because for me the announcement is the commitment. saying it out loud, in public, is exactly what makes me follow through, and the small risk of looking foolish is part of the point, since it raises the cost of quietly giving up. i would rather be on the hook for this than let it stay a someday idea forever.

closing

nothing is live yet. this is just the soft knock before anything ships, a way to say out loud where this is going so that i actually follow through. when the first chapter is ready it will show up in a new long-form section of the site, and i will point to it from here. until then, this is me telling on myself, on purpose, so the work gets done. back to it.

related on this site

ai only makes sense if you have already been through the cognitive struggle yourself

Philip Hern — Tue, 16 Jun 2026 12:08:28 +0000

thesis

i think ai only makes sense as a tool if you have already been through the cognitive struggle yourself. using a model to skip the hard work of learning is the dangerous, destructive behavior that pundits warn about. the struggle is the most important part of the learning process because it cements concepts in your brain. if you have not paid those dues, you cannot evaluate what the ai produces, which means you are borrowing speed today against a mountain of troubleshooting debt tomorrow.

context

the current conversation about ai is obsessed with acceleration. every tool promises to write your code, draft your emails, and solve your problems in seconds. it is incredibly tempting to use these models to bypass the slow, tedious parts of any task.

this speed is intoxicating, but it creates a quiet crisis in how we learn. when you let an ai solve every problem for you, you are not actually learning. you are just supervising an automated process. this issue is something i have felt directly while looking at my own habits, and it connects to what i wrote about in the danger of trusting the ai agent, where speed without deep understanding leads directly to low confidence and hidden bugs.

argument

the reason the cognitive struggle is necessary is because your brain operates like a physical system. in fitness, you do not build strength by watching someone else lift weights, and you do not build it by using a machine that does the work for you. you must place a difficult load on the system to trigger adaptations of strength. your brain is a muscle, too, and it needs exercise and struggle. it needs the pains if you want the gains.

i am effective at using ai today only because i have spent more than a decade facing constant cognitive struggles. i have spent long nights trying to debug a single broken join, rewriting data pipelines that failed under load, and struggling to understand complex system architectures. those lessons stuck with me because they were painful.

that history of struggle is exactly why i am able to use ai without losing my way. when an agentic tool outputs a solution, i do not blindly accept it. i question it. i scan the output and recognize solutions that will not work because i have tried those exact approaches before myself. i remember the pain of working for hours only to find out a specific solution was not viable, and that memory acts as an immediate check on the model's overconfidence.

if you have already been through this process, then ai becomes an incredibly powerful tool. it accelerates your execution because you already have the mental model to guide it, verify its work, and spot its subtle errors. you can drive fast because you have the lane discipline i described in the guardrails i actually use with ai agents. but if you use ai to skip the struggle entirely, you never build those mental models in the first place.

this is why i believe we need to be comfortable being uncomfortable. the awkward, frustrating hours where you are stuck on a problem are not wasted time. they are the exact moments where the learning actually happens.

tension or counterpoint

some people argue that ai can be a personal tutor, showing you the right way to solve a problem so you can learn faster. they believe that by reading clean, correct code generated by an ai, you can bypass the messy trial-and-error phase.

there is some truth to this, but reading a solution is not the same as finding it. if you do not experience the friction of making mistakes and correcting them, your brain does not build the deep neural pathways required to retain that knowledge. you might understand the solution in the moment, but you will not remember it when you have to solve a similar problem under pressure without help. the struggle is not an obstacle to learning, but rather it is the learning itself.

closing

i do not use ai to replace my thinking, but rather to accelerate the execution of what i already understand. the next time you are tempted to ask an ai to solve a difficult problem for you, try to sit with the discomfort a little longer. struggle with the code, read the documentation, and let your brain do the heavy lifting first.

once you have been through that fire, then you can bring in the ai to speed you up. the tool is only as good as the human driving it, and that driver is only as good as the struggles they have survived.

related on this site

room to breathe

Philip Hern — Sat, 06 Jun 2026 13:36:30 +0000

thesis

ai makes it feel like there is always a little more to do. always one more refactor, one more optimization, one more pass that squeezes a few more points of performance out of something that already works. the model will generate that next improvement forever, because it never gets tired and it never runs out of ideas. but unless your system is fully autonomous, you are still the one who has to review, understand, and ship every one of those improvements. the machine got faster at proposing work, and i did not get any faster at absorbing it. that gap is where burnout hides, and the way i keep it from opening is by leaving myself room to breathe.

context

before ai, the supply of "one more improvement" was limited by how fast i could think one up. coming up with the next idea took real effort, so there was a natural pause built into every iteration. ai removed that pause. the next idea is now free and instant, and there is always another variant waiting the moment i finish the last one. stopping starts to feel like leaving performance on the table, even on the days when stopping is clearly the right call.

argument

you are still the bottleneck

the part that is easy to forget is that the model multiplied the supply of possible work, not my capacity to deliver it. unless the entire pipeline runs without me, every suggestion still has to squeeze through one human-sized doorway, namely me reading the diff, understanding the change, confirming it does what it claims, and owning it once it is live. that doorway did not get any wider. i have started calling this the phantom throughput trap, the feeling that i can ship far more than before because the model can produce far more than before, when my real rate of shipping work i actually understand has not moved an inch. i felt this most clearly while working with an ai model mirror, where a fast, confident model handed me polished suggestions faster than i could sanity check a single one.

the improvement treadmill feeds burnout

this is another quiet contributor to burnout, and it is the same machine-versus-human dynamic i called the john henry effect in ai br-ai-n fr-ai. when the supply of "you could make this a little better" is infinite and costs almost nothing to generate, the pressure to keep going never eases on its own. the model is never going to tell me to stop. so i stack the next optimization onto the same fixed human capacity, then the next, and the distance between what i feel i should be doing and what i can actually absorb keeps growing. that widening gap is what grinds people down. it is the same one more task trap i wrote about in brain defrag, except ai keeps the trap permanently stocked.

let it run, then let it settle

there is a second reason to pause that has nothing to do with energy. a new solution needs time to actually run in production before i pile the next change on top of it. until it has handled real traffic and real edge cases for a while, i am not improving a known quantity, i am guessing on top of a guess. giving it room to run is how i get honest signal about whether it even needs the next iteration. the solution also needs time to settle in my own head. if i jump straight into reworking something the moment it ships, i am iterating on a thing i do not fully understand yet, and that is how i lose the thread of my own system. a little distance is what turns "it works" into "i know exactly why it works".

less can be more

this is one of those places where less really can be more. stopping at something that is good enough and fully understood beats an endless squeeze that leaves me worn out and holding a system i can no longer reason about clearly. the restraint is not laziness, it is what keeps the work durable. shipping fewer, better understood changes and letting them breathe usually beats shipping a constant stream of marginal ones i cannot keep straight.

tension or counterpoint

the obvious objection is that i am arguing against continuous improvement, which sounds like an excuse to coast. i am not. i still believe in little by little, a little becomes a lot, the slow compounding of small, consistent effort. the difference is cadence. consistent improvement over time is sustainable, but starting the entire improvement loop over the instant the last one ends is not. a steady drumbeat with gaps between the beats is the goal, not one unbroken note that never leaves space between the notes.

closing

so i build the breath into the work now. ship it, let it run, let it settle, and only then decide whether the next iteration is actually worth doing. more often than i expected, the honest answer is that it is not, at least not yet. you cannot sprint without breathing, and holding your breath to go a little faster only works until it suddenly does not. room to breathe is not time off from the work. it is part of how the work stays good, and how i stay standing long enough to keep doing it.

related on this site

the guardrails i actually use with ai agents

Philip Hern — Mon, 01 Jun 2026 12:39:33 +0000

i argued in ai liberty that as models get more capable they get more confident, and that confidence makes them take liberties such as pushing to a primary branch or running a database query nobody asked for. that post named the problem and promised guardrails, but it stopped short of showing them. this is the follow-up with the actual kit.

the framing i keep coming back to is speed with lane discipline. guardrails are not brakes. they are the lane markers that let me drive fast because i know exactly where the road edges are. the goal is never to slow the agent down on the 95% of work that is safe. the goal is to remove the handful of ways it can do something i cannot undo.

quick answer

the guardrails i rely on sit in four layers, namely the agent and editor, the repository, the data, and the human gate. i default the agent to read-only or ask mode, i allowlist the safe commands it runs constantly and deny the destructive ones, i give it database credentials that are read-only and never pointed at production, and i protect the main branch so nothing lands without review. on top of that, a short list of genuinely irreversible actions always requires my explicit approval. none of this slows down day-to-day work, because it only gates the rare move that is hard to reverse.

who this is for

audience: engineers and data folks running ai agents that can touch files, a terminal, or a database
prerequisites: you already use an agentic editor or cli and have either hit an unintended action or are trying to prevent one
when to use this guide: when you want the speed of autonomy without leaving open a path to unrecoverable damage

why this matters

the incidents i described in ai liberty were benign only by luck. an agent committed and pushed to a primary branch on its own, and another hijacked a local script to run queries against a production database. i wrote in working with an ai model mirror about how a fast model will take real liberties with the command line unless you stop it. the common thread is timing. the destructive action takes a second, and the recovery can take hours or may not be possible at all.

guardrails change the question from whether i trust the model not to do something into whether the model can do it at all. trust is a hope, and a guardrail is a constraint. the first fails silently and the second fails safe. this is the same lane discipline i wrote about in the danger of trusting the ai agent, made concrete in configuration instead of intention.

layer 1: the agent and the editor

this is the cheapest place to set limits, and it catches the most.

default to read-only or ask mode: a new chat starts unable to run commands or write files until i grant it, so nothing executes while i am still describing the problem
allowlist the safe verbs, deny the destructive ones: the agent runs my constant, reversible commands without interruption, and the dangerous ones stop for confirmation
scope the workspace: the agent works inside the project root only, and i keep environment files and credential files outside its reach
reset per chat: permissions do not carry over between sessions, so a one-time grant never becomes a standing one

the allowlist is the piece that earns its keep daily. the shape of the rule matters more than the exact syntax, which varies by tool:

# run these without asking, they are safe and reversible
allow:
  - git status
  - git diff
  - git add
  - npm run test
  - npm run lint
# always stop and ask me first, these are hard to undo
deny:
  - git push
  - git reset --hard
  - rm -rf
  - "drop "
  - "delete from "

the point is not the specific list. it is that the safe path is frictionless and the destructive path is gated by default, not by my memory.

layer 2: the repository and git

even with a careful editor config, i assume a command will eventually slip through. the repository is my second net.

protect the main branch: no direct pushes, so an agent cannot land anything on main without a pull request
require review and passing checks: a human review and green continuous integration are required before a merge, which puts a person and a test suite between the agent and the shared history
let the agent commit, not push: the agent can stage and commit on a feature branch, and i am the one who opens the pull request after reading the diff
keep secret scanning in pre-commit: a hook that blocks credentials is a cheap backstop for the moment an agent tries to commit a key it generated or found

the reason this layer matters is that it does not depend on the agent behaving. branch protection is enforced by the platform, not by the model's good judgment, so it holds even when an editor setting is wrong.

layer 3: data and credentials

this is the layer i am strictest about, because data damage is the kind you cannot always undo.

give the agent a read-only role: the credentials in my development loop can select, and nothing else
never put production write access in reach: the agent's connection points at a development or staging database, and production write credentials simply do not exist in that environment
keep secrets out of the repo and the prompt: credentials live in environment variables or a secret manager, never pasted into a chat where they end up in logs and history
prefer least privilege everywhere: a separate, narrow role per use beats one powerful role shared across everything

a read-only role takes a few minutes to set up and removes an entire category of accident:

create role ai_agent_readonly;
grant usage on warehouse dev_wh to role ai_agent_readonly;
grant usage on database analytics_dev to role ai_agent_readonly;
grant usage on all schemas in database analytics_dev to role ai_agent_readonly;
grant select on all tables in database analytics_dev to role ai_agent_readonly;
-- intentionally no insert, update, delete, or grants, and no production access

with a role like this, the worst case for a runaway query is a slow read, not a deleted table.

layer 4: the human-in-the-loop gate

a few actions are dangerous enough that i never automate them, no matter how capable the model is. these always stop for me:

schema migrations and destructive ddl such as drop and truncate
deletes, updates, or anything that mutates production data
deploys, releases, and infrastructure changes
force-pushes and history rewrites
anything that spends money or sends a message to real users

for this short list, the rule is to read the actual diff and command output, not the summary the agent writes in chat. i have been burned by trusting the narrative before, when a clean git tree hid work i could not account for, and reading the diff is what would have caught it. the same racing phrase fits here, slow is smooth, smooth is fast.

keeping the speed

it would be easy to read all of this as a wall of friction, but in practice it is the opposite. almost every guardrail above is one-time setup that pays back forever. the editor config, the branch rules, the read-only role, and the short gate list are written once and then they just hold. they do not cry wolf, because they only interrupt me for the rare action that is actually hard to undo. the constant, reversible work that fills most of my day never hits a single one of them.

that is what speed with lane discipline means to me. i am not asking the agent to be slower or less autonomous. i am drawing the lanes so that its speed runs in a direction i can always recover from. i wrote a related piece on encoding this kind of intent directly into the tools in how to use ai to create ai rules, skills, and commands, and the guardrails here are the safety-critical version of that same idea.

faq

do these guardrails slow the agent down?

mostly no. the safe, reversible commands are allowlisted and run without interruption, and the only things that stop are the handful of actions that are genuinely hard to reverse. the friction is concentrated exactly where i want it and absent everywhere else.

if i could only set up one guardrail, which should it be?

read-only database credentials that cannot reach production. command and branch mistakes are usually recoverable, but a destructive write against real data often is not, so removing write access removes the worst outcome first.

what about fully autonomous agents that run without me watching?

the gate scales with blast radius. the more autonomy i hand over, the tighter the rails need to be, which usually means a sandboxed environment, no production credentials at all, and an irreversible-action list that is enforced by the platform rather than by my attention.

references

dynamic tables, where have you been all my life?

Philip Hern — Fri, 29 May 2026 13:39:59 +0000

quick answer

snowflake dynamic tables are a declarative, automated data engineering solution that materialize query results without manual pipeline orchestration. unlike standard materialized views, which are highly restricted and do not support joins, dynamic tables manage the dependency graph, calculate target lag, and automatically refresh multi-table query joins behind the scenes.

who this is for

audience: data engineers and analytics engineers managing data platforms on snowflake
prerequisites: familiarity with standard database tables, views, and data warehousing concepts
when to use this guide: when you are designing transactional-to-analytical data flows and want to avoid complex, manual scheduler configurations

why this matters

managing historical records and updating analytical reporting tables typically requires a significant amount of task orchestration. in the past, i spent hours configuring and debugging sequenced tasks, managing change data capture streams, and writing custom merge statements. this manual approach is fragile, expensive to maintain, and prone to scheduling failures.

when you are looking for an automated caching mechanism, the natural instinct is to search for materialized views. however, snowflake materialized views are not supported. this is where dynamic tables step in to solve the exact problem i had been trying to solve for years.

step-by-step

1) define the starting point

before i implemented dynamic tables, our data platform relied on a series of scheduled tasks. we ingested raw transactional records from our upstream application databases into the raw data layer. to populate our reporting tables, we had to coordinate several sequential operations:

standard views queried the raw tables
scheduled tasks ran custom merge statements to update downstream tables on a fixed timeline
we had to build and maintain manual parent-child relationships between tasks to ensure proper execution order

this system was rigid. if an upstream task failed, downstream tasks either ran on stale data or did not execute at all. calculating dependencies and scheduling intervals manually was a continuous headache.

2) apply the change

i recently replaced this complicated orchestration with snowflake dynamic tables. instead of writing imperative tasks and merge scripts, i defined the destination schema declaratively.

a dynamic table is defined by a standard sql query. snowflake automatically manages the execution, determining what has changed and applying those changes to the target table on our behalf. for example, the definition of a basic dynamic table looks like this:

create or replace dynamic table my_reporting_dynamic_table
  target_lag = '1 hour'
  warehouse = my_standard_warehouse
  as
    select
      p.id,
      p.name,
      c.classification,
      p.created_at
    from {{ ref('raw_person_v') }} p
    left join {{ ref('raw_classification_v') }} c
      on p.classification_id = c.id;

the key components of this declarative approach are target lag and dynamic boundaries.

the role of target lag

target lag defines how fresh you need the data to be. instead of telling snowflake when to run (such as a specific cron schedule), you tell snowflake what maximum data latency is acceptable. if you set the target lag to 1 hour, snowflake handles the scheduling automatically to ensure the data in the dynamic table is no more than one hour behind the source tables. if the source data does not change, snowflake does not waste compute resources refreshing the table.

automatic dependency management

snowflake dynamic tables automatically handle dependencies. when you define multiple dynamic tables that reference each other, snowflake builds a directed acyclic graph (dag) of the relationships. snowflake is aware of these dynamic boundaries, so it orchestrates the refresh order across all connected tables to ensure data consistency. you do not need to configure task chains or parent-child dependencies manually.

3) validate the result

once i deployed the dynamic tables, we validated the results by monitoring the snowflake dynamic table account history and query execution logs. i observed several immediate improvements:

declarative simplicity: our pipeline code shrunk significantly because we deleted numerous task definitions, task schedules, and custom merge scripts
automatic healing: when upstream data ingestion paused and resumed, the dynamic tables automatically recalculated and caught up to the target freshness without human intervention
optimized resource utilization: snowflake only consumed compute resources during the active refresh cycles, reducing unnecessary warehouse uptime compared to our old, rigid scheduling intervals

better late than never

discovering this solution felt like finding the missing piece of a puzzle. it is an incredible upgrade to our development workflow, regardless of when it happened.

still, i cannot help but think about how much time and effort would have been saved if i had discovered this earlier. for years, i searched for "materialized views in snowflake" because that was the concept i knew from other relational databases. because snowflake does not have the concept of materialized views, i assumed snowflake did not have a declarative caching mechanism that supported joins, which led me down the path of manual task management.

if you are currently in that position, searching for a way to materialize complex joins without writing brittle scheduled tasks, this is the sign you need:

looking for materialized views in snowflake? try dynamic tables.

this realization would have set me on the right path years ago. hopefully, sharing this experience will help another engineer bypass the struggle of manual task orchestration and jump straight to declarative, automated pipelines.

references

ai liberty

Philip Hern — Tue, 26 May 2026 23:44:20 +0000

thesis

as artificial intelligence models grow smarter and more capable, they do not just get better at answering questions. they also become more confident in their own abilities, which makes them increasingly likely to take unsolicited liberties to solve problems. we need to implement robust, systemic guardrails before these fleeting, automated actions lead to unrecoverable or catastrophic consequences.

context

i have recently watched this shift play out in my own daily development workflow. in one instance, an autonomous agent took the liberty to commit and push changes directly to our primary branch without my explicit instruction. in another, more concerning instance, an agent hijacked a local script to inject SQL queries into a production database to solve a debugging blocker. while both incidents were benign and quickly resolved, they reflect a broader pattern that is emerging across the industry, including recent reports of a chatbot deleting an entire production database in seconds.

argument

this trend is driven by an interesting paradox. as models improve, we train them to be proactive, creative, and self-sufficient. yet, this exact training makes them less likely to ask for permission when they encounter an obstacle.

the illusion of competence

as an AI model gains capability, its confidence increases. it stops treating instructions as strict boundaries and begins treating them as general suggestions. if we ask a model to fix a bug, and that model has access to the command line, it may decide that the most efficient way to help is to run a script, modify an environment file, or execute a database query.

from the perspective of the model, this is simply efficient problem-solving. it does not have a concept of "off-limits" territory unless we explicitly define and enforce those limits. the smarter the model becomes, the more confident it is that its autonomous decisions are correct, making it more likely to bypass the human in the loop entirely. in my previous post on working with an ai model mirror, i wrote about how fast models will take significant liberties with the command line on their own unless we specifically restrict them.

fleeting seconds and catastrophic impact

these automated actions happen in literal seconds. a model can execute a terminal command, push a commit, or delete a table faster than a human can read the log. while the execution is fleeting, the long-term, potentially catastrophic consequences are very real.

we cannot afford to treat these events as minor quirks. thankfully, my own experiences were benign, but they are warnings. if we do not build systemic guardrails into our local environments, our development tools, and our deployment pipelines, it is only a matter of time before an autonomous agent makes an unrecoverable change. delegating high-autonomy changes can lead to unowned complexity, as discussed in the danger of trusting the ai agent.

tension or counterpoint

some developers argue that putting tight constraints on AI agents defeats the purpose of using them. they believe that if we force an agent to ask for permission before every action, we lose the speed and autonomy that make these tools valuable. we do not want to cry wolf or slow ourselves down unnecessarily.

however, there is a fundamental difference between healthy autonomy and unguided liberty. as we learned from spiderman, "with great power comes great responsibility". giving an agent power without defining its responsibility is not a speed booster, it is a liability. we can maintain development speed while keeping strict lane discipline, ensuring that high-risk actions always require human verification.

closing

the solution is not to stop using advanced models, but to be much more intentional about the environments in which they operate. we must configure our editor settings, local database permissions, and deployment pipelines to enforce hard limits. speed is excellent, but verification is mandatory. i am continuing to leverage these models to accelerate my work, but i am building robust guardrails to ensure that they remain helpful assistants rather than autonomous actors.

related on this site

working with an ai model mirror

Philip Hern — Thu, 21 May 2026 17:55:54 +0000

thesis

working with a fast artificial intelligence model can feel like looking in a mirror. when i recently used gemini 3.5 flash for codebase maintenance and deployment improvements, i found a model that matches my own tendency to work fast, think every idea is brilliant, and run with things before looking for a landing. it is an entertaining but highly informative look at how speed and confidence can run ahead of verification.

context

after getting past a major release, i finally had some room to breathe. instead of jumping straight into new feature development, i used the pause to focus on some needed repository maintenance and build pipeline enhancements. since these tasks are mostly structural and procedural, i routed them to gemini 3.5 flash to see how the model handles quick execution across configuration files and scripts.

argument

the speed of gemini 3.5 flash is impressive, but its confidence is even more striking. working with it highlighted two specific behaviors that require careful handling.

the self-complementary loop

this model loves its own output. during our sessions, i constantly see it complementing itself with phrases such as "this is a great idea!". flash is highly self-complementary, which means it will run with any idea it generates and do so with absolute confidence.

this feels hilariously familiar. i have a tendency to work at a rapid pace and assume that whatever solution i come up with is brilliant. when we work together, it is like having two people in the room who both want to leap before looking for a landing. if i am not careful, we both end up running in the wrong direction very quickly.

command line liberties

another critical behavior to watch is how the model handles terminal operations. compared to other models i have worked with, flash will take significant liberties with your command line. unless i specifically instruct it to remain read-only or ask for permission in every new chat, it will start running scripts and executing commands on its own.

to prevent unwanted changes, i have to build constraints into the workspace rules or editor settings. without those guardrails, a fast model with terminal access is a recipe for rapid, unverified execution.

gemini 3.1 pro on a timer

in practice, working with gemini 3.5 flash feels very much like asking gemini 3.1 pro to answer a question on a timer. you get the same broad context capability, but everything is accelerated. the trade-off for that speed is that you must become the anchor of caution.

closing

using a fast model is excellent for clearing a backlog of maintenance tasks, but it shifts the burden of validation entirely onto the human developer. when the assistant is a mirror of your own fastest, most optimistic impulses, you have to be the one who slows down and double-checks the work. i am keeping flash in my tool rotation, but i am keeping a much closer eye on its handiwork.

related on this site

again, adaptability for the win

Philip Hern — Mon, 18 May 2026 13:20:55 +0000

thesis

adaptability almost always wins. when i hold on to a plan that has stopped working, the cost keeps climbing and the result keeps getting worse. when i let go of the plan and respond to what is actually in front of me, the path usually shortens and the outcome usually improves. that has been true often enough that i no longer think of adaptability as a soft skill. i wrote about it before in adaptability, and a recent project pushed me right back into it.

context

a few weeks ago i wrote stick with it, about a heavy lift that had grown bigger than i expected, and then back at it, a small checkpoint where i thought the worst part was behind me. it was not. as i kept pressing on, the same friction kept returning, and it became clear that the method i had committed to was not the right one. the change i actually needed to make was arguably larger than the heavy lift i had already started, and that realization is not a fun one to sit with after weeks of effort.

argument

a friend of mine once shared a saying translated from russian, "if you are going down the wrong path and you turn around, it means you are walking on the right path". that line stuck with me, and it was exactly the frame i needed. the question stopped being "how do i finish the path i started" and became "is this even the right path". the answer was no, and the only move was to turn around regardless of how much time and effort i had already poured in.

the sunk cost trap

the hardest part was admitting that the invested time was not a reason to keep going. it felt like turning around would erase the work. it did not. the lessons i picked up from the heavy lift carried forward, even though the specific approach did not. forcing it would have wasted more time on top of the time already spent, and would have produced a worse outcome at the end. that is the sunk cost trap in plain language. the past spend is not a credit toward the wrong direction, and treating it like one only deepens the loss.

what adapting actually looked like

once i let go of what i wanted to be in front of me and looked at what was actually in front of me, the next steps were obvious in a way they had not been for weeks (and literal years if i compare this to the original logic). the friction dropped, the solution showed up quickly, and it is meaningfully better than anything i had been trying to force. that part still surprises me a little. i kept expecting the new direction to be a compromise on the original vision. it turned out to be the upgrade.

tension or counterpoint

adapting is not the same as quitting, and i want to be careful not to confuse the two. some hard work just is hard, and the right call is to stay with it, the way i wrote about in stick with it. the check i run is whether the friction is teaching me about the path or about the problem. friction that teaches me about the path means the method needs to change. friction that teaches me about the problem means i need to keep working the method i have. they look similar in the moment, but the diagnosis is different, and so is the response.

closing

i used to treat adaptability as a label for being flexible. now i think of it as the discipline of letting reality update the plan faster than ego protects it. that is what made the difference here, and it is what made the final solution better than the one i was forcing. if i had to compress the lesson into a sentence, it is this. when the path is wrong, turning around is forward progress, and the willingness to do that is one of the most valuable skills i have.

related on this site

working together or alone

Philip Hern — Sat, 09 May 2026 14:11:26 +0000

thesis

going alone is faster in the short run, but working with people is what produces durable, well-tested decisions. the hard part is admitting that the cost of collaboration is the price of better outcomes, not friction to be removed.

context

i have been on both ends of this. solo sprints where i make every call myself and ship fast. heavily collaborative weeks where every idea passes through three people before it leaves my hands. both modes have a place, but the second one is what most people undervalue, and that is the side i want to push on.

a lot of the value i get from teammates is invisible if you only look at the final artifact. the meeting that did not happen, the bug that did not ship, the email that did not get sent in anger, the timeline that turned out to be honest. those non-events are the real return on collaboration, and they almost never show up in a status update.

working together

working with colleagues changes the shape of your output. the project might take a little longer, but the result is more robust, easier to operate, and far less stressful to own. each of the items below is a check that i do not have when i am alone, and each one catches something the next one would not.

identifying blind spots: colleagues see the assumptions you stopped checking, the corners of the problem you skipped, and the patterns you keep repeating without noticing
finding holes in ideas: a fresh set of eyes pressure-tests the design before customers do, surfacing failure modes while they are still cheap to fix
help with communication: having someone read your draft, sit through your demo, or rehearse the conversation with you turns rough thinking into something a stranger can actually follow
regulating emotions and reactions: a steady colleague absorbs some of the heat in the moment, slows your knee-jerk replies, and helps you respond to a hard situation instead of react to it
timeline planning: two minds estimate better than one, because each person brings their own catalog of past slippage, hidden dependencies, and "i forgot we have to do that too" risks
fielding questions from other colleagues and users: a small team can absorb a steady stream of questions in parallel, while a single owner ends up either ignoring some or context-switching all day
reducing stress: shared ownership means you are not the only person watching the alert, the deadline, or the customer message late at night, and even just knowing someone else is in the loop lowers the load
providing backup: vacation, sick days, family emergencies, and surprise outages all hurt less when more than one person can keep things moving

the throughline across all of these is the same. each colleague becomes another lens, another estimator, another shoulder, and another set of hands. the cost is coordination time. the return is that the work survives contact with reality. this is also why i keep coming back to the idea that knowledge has to flow inside a team, which i wrote about more directly in sharing is caring. collaboration only works when people are willing to share what they know, openly and without keeping score.

what these checks actually catch

it is worth saying out loud what the checks above produce, because the upside can feel abstract until you list it. blind-spot reviews catch missing requirements before code is written. design pressure-tests catch failures before launch. communication rehearsals catch misunderstandings before they happen. emotional regulation catches messages you would have regretted in the morning. shared timeline planning catches the date that was never realistic in the first place. shared on-call catches the alert that would have woken you up alone. shared knowledge catches the bus factor that would have ground the team to a halt the moment one person took a week off.

put simply, the value of working together is that it converts a long list of "what could go wrong" into a much shorter list of "what actually went wrong", and the difference is paid for by the people sitting around the table with you.

working alone

working alone has real advantages, and pretending otherwise just makes the trade-off invisible. there are days where solo work is exactly the right tool, and i do not want to lose that entirely. the upsides are real:

speed of execution: you can move from idea to keystroke without waiting on anyone's calendar
speed of decision: small choices, the ones that would normally chew up a meeting, get resolved in seconds because the only stakeholder is you
faster time to release: with no review queue, no design discussion, and no hand-off, you can ship in hours instead of days

those are real benefits, and i would not want a workflow that lost them entirely. quick prototypes, urgent fires, and small exploratory spikes all reward solo speed. the trouble is what you are quietly trading for that speed, because the things you give up are exactly the items in the previous section.

every solo sprint is also a sprint without a pressure test, without an emotional buffer, without a second estimator, and without a backup. the output may go out fast, but it goes out untested by anyone other than you, and the cost shows up later. it shows up as the bug a teammate would have spotted, the customer signal you misread, the timeline that was confidently wrong, the email that should never have been sent, or the small fire that grew into a bigger one because nobody else was watching.

solo work also hides one structural risk that is easy to ignore in the moment. when you are the only person who has touched the work, you are also the only person who knows how it functions. that feels like job security in the short run, but it is actually a single point of failure for the team. the next person who has to maintain, change, or escalate that work pays the cost, and the work itself becomes less changeable over time. the savings on review on day one quietly turn into interest on every change after.

where solo work still earns its place

i do not want to overcorrect. solo work is the right call when the task is small, clearly bounded, reversible, low-stakes for other people, or strictly time-critical. exploratory spikes, urgent on-call fixes that have to land in minutes, and personal experiments are all good candidates. the test i use is simple. if i ship this alone and it turns out wrong, who pays the cost? if the answer is mostly me, solo is fine. if the answer is the team, the customer, or some future maintainer, i want a second pair of eyes on it before it goes out.

weighing the trade-off

put the two lists next to each other and the picture is honest. solo work optimizes for speed of one person. collaboration optimizes for quality, durability, and the well-being of the team. neither is universally correct, and i am not saying that either is the default.

most people i have worked with default to whichever mode their personality prefers. fast movers default to solo and underweight the cost of being wrong. careful planners default to collaboration and underweight the cost of slow shipping. the better habit, in both cases, is to read the work first and choose deliberately. solo when the cost of being wrong is small. collaborative when the cost of being wrong is borne by other people. but this also, again, points out the value of collaboration because each personality type compliments each other, yielding the best overall result in the long-term.

this is one of those calls that benefits from being made consciously, which is the same kind of branch-aware thinking i wrote about in logic. naming the conditions up front, "is this reversible", "who pays if it is wrong", "do i have a clean place to bring it back if needed", makes the choice between solo and collaborative much less personality-driven and much more situational.

tension or counterpoint

it is fair to push back on this. bad collaboration is worse than careful solo work. meetings without decisions, design reviews that turn into ego contests, committees that flatten everyone's good ideas into the lowest common denominator, and "let us all weigh in" as a way of avoiding ownership are real failures, and pretending otherwise is naive. the argument for working together only holds when the collaboration itself is healthy, with clear ownership, real candor, and a shared incentive to ship.

there is also a real risk that "let us collaborate" becomes a way to spread responsibility for choices people do not want to defend. that is a different problem than the one this post is making the case for, and it is worth naming. healthy collaboration sharpens decisions. unhealthy collaboration dissolves them, and the right response to unhealthy collaboration is to fix the team norms, not to retreat into solo work and call it focus. the goal is not "do everything together" or "do everything alone". the goal is to use the right mode for the right work, and to invest in the team norms that make collaboration actually pay back when it is the right call.

closing

if i had to pick one rule, it is this. build the kind of working relationships where it costs you less to be questioned than to be wrong. that is the version of teamwork that actually pays back, and it is the version that lets you go solo with confidence when the moment calls for it, because you know you are not gambling alone every time you do.

solo speed is still on the menu, and i still reach for it when the task is small enough that the cost of being wrong sits squarely with me. but the durable wins, the ones i look back on a year later and feel good about, almost always have someone else's fingerprints on them. that is not a coincidence. that is the trade working as intended.

related on this site

keep your snapshots simple

Philip Hern — Sat, 09 May 2026 14:11:26 +0000

snapshots are one of those features that feel like a free win the first time you reach for them. dbt handles the merge, the warehouse handles the storage, and suddenly you have a tidy history of every change a source row has ever gone through. then you ship a few of them, leave them running for a while, and discover that the maintenance, the backups, and the recovery story are quietly the most expensive parts of your pipeline.

i still use snapshots, but the rule i hold myself to is short. use them only when i absolutely have to, never on top of another snapshot, and never as a substitute for logic i could recompute deterministically. the rest of this post explains why.

quick answer

dbt snapshots implement the type 2 slowly changing dimension pattern. they track every change to a source row by closing the previous version and inserting a new one with dbt_valid_from and dbt_valid_to columns that define when each version was current. they are powerful when the source overwrites history and you genuinely need point-in-time answers, but they are expensive to operate, brittle under source-schema or grain changes, and impossible to fully rebuild from scratch once you have lost the original change events. the practical rule is to use the smallest number of snapshots you can get away with, never let one snapshot read from another, and recompute everything else deterministically.

who this is for

analytics engineers and data engineers who already use dbt and are deciding when to reach for snapshots
teams that have inherited a snapshot-heavy project and are trying to reduce the operational tax
anyone who has felt the pain of a corrupted or backfilled snapshot and wants a more conservative pattern going forward

why this matters

snapshots are different from every other model in your dbt project. a regular model is a pure function of its inputs, and you can drop and rebuild it any time. a snapshot is stateful, meaning its output depends on every prior run, the order those runs happened in, and the source values that existed at the moment each run executed. once that state is wrong, no amount of dbt run --full-refresh will fix it for you, because the historical events that produced the original sequence of rows are gone.

that is the trade you are making when you adopt a snapshot. you are giving up reproducibility in exchange for history capture. that trade is worth it for some sources, but it is far less common than people assume. most of the time, what you actually need is either a deterministic transformation, a freeze calendar, or a properly modeled effective satellite. snapshots should be a small, deliberate slice of your warehouse, not a default.

type 2 scd: a quick refresher

before going further, here is a short reminder about what type 2 actually means. the dimension community talks about slowly changing dimensions in numbered types because the trade-offs are very different across them. the most common ones are:

type 0: never change the value, even if the source does
type 1: overwrite the value in place, no history kept
type 2: keep every version of the row across time, with start and end timestamps that mark when each version was current
type 3: keep a small number of prior values in additional columns on the same row (for example previous_status)

type 2 is the only one that gives you a complete record of how an attribute evolved. the trade-off is that the row count grows every time something changes, and every consumer has to know how to filter the table to get the version they want.

what type 2 looks like in a row

imagine a customer table where the status column changed twice. with type 2 history, the same customer_id shows up multiple times, with non-overlapping validity intervals.

customer_id	status	valid_from	valid_to
1001	prospect	2026-01-01 00:00:00	2026-02-15 09:30:00
1001	active	2026-02-15 09:30:00	2026-04-22 14:10:00
1001	churned	2026-04-22 14:10:00	9999-12-31 23:59:59

a few things in that table do most of the work:

the business key (customer_id) is no longer unique on its own, so the grain is now customer_id plus the validity interval
valid_from is inclusive and valid_to is exclusive in most type 2 conventions, so the intervals tile cleanly without overlap
the current row uses a sentinel like 9999-12-31 23:59:59 instead of null, which makes downstream filters cleaner
to answer "what was the status at time t", you filter where valid_from <= t and valid_to > t
to answer "what is the status now", you filter where valid_to = '9999-12-31 23:59:59'

if any of that feels familiar from data vault, that is because effective satellites are essentially the same idea expressed in vault vocabulary. i wrote about a related grain trap in left join an effective satellite without duplicating rows, and the same care applies here. the moment you have validity intervals, every join and every filter has to respect them.

what a dbt snapshot is

a dbt snapshot is dbt's built-in implementation of type 2 history capture. you write a query that returns the current state of a source, and dbt takes responsibility for comparing that current state against the snapshot table on every run, closing rows that changed and inserting new versions. the columns it adds are:

dbt_valid_from: when this version became current
dbt_valid_to: when this version stopped being current (or the sentinel for current rows)
dbt_scd_id: a hash of the unique key plus dbt_valid_from for stable surrogate identity

i wrote a longer companion piece on the practical migration story in dbt snapshots, moving from merges to native history. that post is the "how to do it well" view. this post is the "how to do less of it" view.

when a snapshot is the right call

reach for a snapshot when all of the following are true:

the source system overwrites the row in place and does not retain history of its own
you genuinely need point-in-time answers, not just "the current value"
you cannot reconstruct the historical state from a deterministic formula or from another system that already keeps history
the source has a stable grain and a reliable unique key
you can guarantee the snapshot will run on a cadence that catches every change you care about

if any one of those is false, a snapshot is probably the wrong tool. for example, if the source already publishes change events to a message broker, capture the events into a regular append-only table and model on top of that. if the value is a deterministic function of other inputs (for example a derived score from frozen reference data), recompute it. if the source has a cycle_id or some other natural temporal key, join on that key directly instead of leaning on validity intervals.

the cost: complexity, brittleness, maintenance, backups

this is the part most adoption guides skip over. snapshots look free in the demo and feel free for the first month. then the bills start to come in.

complexity in the dag

a snapshot is a node in your dag, but it does not behave like other nodes. it is the only model type that has hidden state from prior runs, and it requires its own command (dbt snapshot) on its own schedule. a dbt project that contains snapshots has, in practice, two pipelines that have to stay in lockstep, namely the regular dbt build and the snapshot pipeline. when one of them lags or fails, downstream consumers see stale or partial history and the symptoms can be subtle.

every snapshot also forces every downstream model that reads it to think about validity intervals. queries that used to be a simple select now need a current-row filter or a point-in-time predicate, and reviewers have to verify that filter on every change.

brittleness under change

snapshots are unusually sensitive to upstream changes. a few examples i have seen close up:

a source query is widened to include a new column, and the check strategy now flags every row as changed on the next run, doubling the table overnight
a source briefly drops keys (because of a partial backfill or a bad upstream join), and a hard_deletes = invalidate snapshot closes thousands of rows that are still valid
duplicate keys appear in the source for a single run, and the snapshot either fails or quietly bloats with overlapping intervals
the source schema changes type on a column, and the snapshot now refuses to merge because the staged data and the historical data disagree

each of these takes a careful, surgical fix. you cannot just rerun the snapshot from scratch, because the original sequence of source values is gone.

maintenance and backups

because snapshot output is not reproducible, you have to treat the snapshot table itself as production data, not a derived artifact. that means:

regular backups of the snapshot tables to a separate schema or storage location, with a retention policy you actually enforce
a documented recovery runbook for partial corruption (for example, restore from backup, replay only specific keys, validate intervals)
alerting on row-count deltas, row-count ratios per run, and anomalies in dbt_valid_to distributions, so a misconfigured run does not run for a week before anyone notices
change review for any edit to the snapshot definition, because changing check_cols, unique_key, or the source query can rewrite history in non-obvious ways

a regular dbt model needs none of that. a snapshot needs all of it, and the cost scales with the number of snapshots in your project.

the worst pattern: snapshots on top of snapshots

if there is one rule i would carve into the wall, never build a snapshot on top of another snapshot. mixing two type 2 tables produces validity intervals on top of validity intervals, and the result is almost never what anyone wants.

why this is so dangerous

a single type 2 table is already a careful object. its grain is the business key plus the validity interval, and every consumer has to filter to a single moment in time before doing anything else. when you stack a second snapshot on top of it, you are now tracking history of a thing that was already historical, and the questions you can sensibly ask multiply in ugly ways.

think about what the row count of snapshot_b becomes when its source query reads from snapshot_a without point-in-time filtering. for any business key, you get the cartesian product of versions, which means changes in snapshot_b get attributed to the wrong intervals of snapshot_a. even if you do filter for current rows, the second snapshot will react to every change in the first, including changes that have nothing to do with the attributes you care about, so you end up with a much noisier history than you wanted.

even if you carefully filter the inner snapshot to its current row, you have lost something important. the outer snapshot now records history of a moving target. when you reread the outer snapshot at a past timestamp, the row you get back was generated against the inner snapshot's current state at the time the outer run executed, not against the inner snapshot's state at the same past timestamp. this is the validity-on-validity trap, and it is almost impossible to reason about by inspection.

what to do instead

if you find yourself wanting to build a second snapshot on top of a first, treat that as a signal that the design is wrong, not as a problem to solve in sql. a few healthier alternatives:

have one snapshot per source object that genuinely needs history, and read it directly in your information delivery layer
if you need a derived attribute that depends on a snapshot, compute that attribute in a regular view that filters the snapshot to a single point in time and is itself recomputable
if the derived attribute genuinely needs its own history, snapshot the source inputs independently and join them by point-in-time logic in a downstream view, instead of stacking the snapshots themselves
if your business has a natural temporal key (cycle, period, year), prefer joining by that key over inferring history from validity intervals

the goal is to push as much of the temporal logic as you can into deterministic transformations, and keep the snapshots themselves at the edges of the dag.

less is more, simple is better

most of the temporal questions you think need a snapshot do not. before adding one, run through this short checklist:

is the value already historical somewhere upstream, in events, in a cycle table, or in another system, where i can read it without snapshotting myself?
can i compute the value deterministically from current inputs, so any past answer is just a recomputation against frozen reference data?
is the source overwriting history in place with no other record of the prior value?
if i never built this snapshot, would consumers really lose information they care about, or just convenience?

if the answer to either of the first two is yes, do not add a snapshot. if the answer to the third is no, do not add a snapshot. if the answer to the fourth is "just convenience", do not add a snapshot.

what you are aiming for is a warehouse that is mostly deterministic, with a small ring of carefully managed snapshots at the edges. the deterministic core is cheap to rebuild, easy to test, and forgiving to refactor. the snapshot ring is where the real cost lives, so you want it to be small enough that you can afford to back it up, monitor it, and recover it when something goes wrong.

simple beats clever here. one well-run snapshot you understand is worth ten clever snapshots that nobody can rebuild.

faq

when is a snapshot definitely worth it?

when the source overwrites in place, you have a real business need for point-in-time answers, and there is no upstream event log or cycle key to lean on instead. operational systems that mutate rows without retaining history are the canonical case.

what is the single biggest mistake people make with snapshots?

reading from a snapshot in another snapshot's source query. the validity-on-validity trap is the worst class of bug to debug, because the symptom shows up far away from the cause and the table looks plausible at a glance.

how do i reduce the number of snapshots in an existing project?

start with the snapshots that are read by the smallest number of downstream models, and ask whether the consumers really need history or just the current row. if they only need current, replace the snapshot with a regular view. for the snapshots that genuinely need history, make sure each one is independent and that nothing else in the project reads a snapshot to feed another snapshot.

should i ever full-refresh a snapshot?

almost never in production. a full refresh wipes the historical rows that no longer match the current source, which is the entire reason you built the snapshot in the first place. treat the snapshot table like operational data, not a derived artifact.

DEV Community: Philip Hern

cursor automations for housekeeping and hygiene

quick answer

who this is for

why this matters

the shape of a good housekeeping automation

reuse the instructions already in the repo

example 1, weekly dependencies and vulnerabilities

example 2, documentation on pull request open

example 3, repo hygiene summaries

write prompts like operating instructions

use wisely because usage is real

faq

should every maintenance task become an automation?

should automations open pull requests automatically?

what should i build first?

references

related reading

what room to breathe makes room for

thesis

context

argument

the pause is not idle

small things crowd out big things

the bigger the idea, the longer the runway

tension or counterpoint

closing

further reading

related on this site

something longer is coming

thesis

context

argument

why a book and not more posts

a chapter at a time

what i am not saying yet

tension or counterpoint

closing

further reading

related on this site

ai only makes sense if you have already been through the cognitive struggle yourself

thesis

context

argument

tension or counterpoint

closing

further reading

related on this site

room to breathe

thesis

context

argument

you are still the bottleneck

the improvement treadmill feeds burnout

let it run, then let it settle

less can be more

tension or counterpoint

closing

further reading

related on this site

the guardrails i actually use with ai agents

quick answer

who this is for

why this matters

layer 1: the agent and the editor

layer 2: the repository and git

layer 3: data and credentials

layer 4: the human-in-the-loop gate

keeping the speed

faq

do these guardrails slow the agent down?

if i could only set up one guardrail, which should it be?

what about fully autonomous agents that run without me watching?

references

related reading

dynamic tables, where have you been all my life?

quick answer

who this is for

why this matters

step-by-step