DEV Community: Philip Hern

working together or alone

Philip Hern — Sat, 09 May 2026 14:11:26 +0000

thesis

going alone is faster in the short run, but working with people is what produces durable, well-tested decisions. the hard part is admitting that the cost of collaboration is the price of better outcomes, not friction to be removed.

context

i have been on both ends of this. solo sprints where i make every call myself and ship fast. heavily collaborative weeks where every idea passes through three people before it leaves my hands. both modes have a place, but the second one is what most people undervalue, and that is the side i want to push on.

a lot of the value i get from teammates is invisible if you only look at the final artifact. the meeting that did not happen, the bug that did not ship, the email that did not get sent in anger, the timeline that turned out to be honest. those non-events are the real return on collaboration, and they almost never show up in a status update.

working together

working with colleagues changes the shape of your output. the project might take a little longer, but the result is more robust, easier to operate, and far less stressful to own. each of the items below is a check that i do not have when i am alone, and each one catches something the next one would not.

identifying blind spots: colleagues see the assumptions you stopped checking, the corners of the problem you skipped, and the patterns you keep repeating without noticing
finding holes in ideas: a fresh set of eyes pressure-tests the design before customers do, surfacing failure modes while they are still cheap to fix
help with communication: having someone read your draft, sit through your demo, or rehearse the conversation with you turns rough thinking into something a stranger can actually follow
regulating emotions and reactions: a steady colleague absorbs some of the heat in the moment, slows your knee-jerk replies, and helps you respond to a hard situation instead of react to it
timeline planning: two minds estimate better than one, because each person brings their own catalog of past slippage, hidden dependencies, and "i forgot we have to do that too" risks
fielding questions from other colleagues and users: a small team can absorb a steady stream of questions in parallel, while a single owner ends up either ignoring some or context-switching all day
reducing stress: shared ownership means you are not the only person watching the alert, the deadline, or the customer message late at night, and even just knowing someone else is in the loop lowers the load
providing backup: vacation, sick days, family emergencies, and surprise outages all hurt less when more than one person can keep things moving

the throughline across all of these is the same. each colleague becomes another lens, another estimator, another shoulder, and another set of hands. the cost is coordination time. the return is that the work survives contact with reality. this is also why i keep coming back to the idea that knowledge has to flow inside a team, which i wrote about more directly in sharing is caring. collaboration only works when people are willing to share what they know, openly and without keeping score.

what these checks actually catch

it is worth saying out loud what the checks above produce, because the upside can feel abstract until you list it. blind-spot reviews catch missing requirements before code is written. design pressure-tests catch failures before launch. communication rehearsals catch misunderstandings before they happen. emotional regulation catches messages you would have regretted in the morning. shared timeline planning catches the date that was never realistic in the first place. shared on-call catches the alert that would have woken you up alone. shared knowledge catches the bus factor that would have ground the team to a halt the moment one person took a week off.

put simply, the value of working together is that it converts a long list of "what could go wrong" into a much shorter list of "what actually went wrong", and the difference is paid for by the people sitting around the table with you.

working alone

working alone has real advantages, and pretending otherwise just makes the trade-off invisible. there are days where solo work is exactly the right tool, and i do not want to lose that entirely. the upsides are real:

speed of execution: you can move from idea to keystroke without waiting on anyone's calendar
speed of decision: small choices, the ones that would normally chew up a meeting, get resolved in seconds because the only stakeholder is you
faster time to release: with no review queue, no design discussion, and no hand-off, you can ship in hours instead of days

those are real benefits, and i would not want a workflow that lost them entirely. quick prototypes, urgent fires, and small exploratory spikes all reward solo speed. the trouble is what you are quietly trading for that speed, because the things you give up are exactly the items in the previous section.

every solo sprint is also a sprint without a pressure test, without an emotional buffer, without a second estimator, and without a backup. the output may go out fast, but it goes out untested by anyone other than you, and the cost shows up later. it shows up as the bug a teammate would have spotted, the customer signal you misread, the timeline that was confidently wrong, the email that should never have been sent, or the small fire that grew into a bigger one because nobody else was watching.

solo work also hides one structural risk that is easy to ignore in the moment. when you are the only person who has touched the work, you are also the only person who knows how it functions. that feels like job security in the short run, but it is actually a single point of failure for the team. the next person who has to maintain, change, or escalate that work pays the cost, and the work itself becomes less changeable over time. the savings on review on day one quietly turn into interest on every change after.

where solo work still earns its place

i do not want to overcorrect. solo work is the right call when the task is small, clearly bounded, reversible, low-stakes for other people, or strictly time-critical. exploratory spikes, urgent on-call fixes that have to land in minutes, and personal experiments are all good candidates. the test i use is simple. if i ship this alone and it turns out wrong, who pays the cost? if the answer is mostly me, solo is fine. if the answer is the team, the customer, or some future maintainer, i want a second pair of eyes on it before it goes out.

weighing the trade-off

put the two lists next to each other and the picture is honest. solo work optimizes for speed of one person. collaboration optimizes for quality, durability, and the well-being of the team. neither is universally correct, and i am not saying that either is the default.

most people i have worked with default to whichever mode their personality prefers. fast movers default to solo and underweight the cost of being wrong. careful planners default to collaboration and underweight the cost of slow shipping. the better habit, in both cases, is to read the work first and choose deliberately. solo when the cost of being wrong is small. collaborative when the cost of being wrong is borne by other people. but this also, again, points out the value of collaboration because each personality type compliments each other, yielding the best overall result in the long-term.

this is one of those calls that benefits from being made consciously, which is the same kind of branch-aware thinking i wrote about in logic. naming the conditions up front, "is this reversible", "who pays if it is wrong", "do i have a clean place to bring it back if needed", makes the choice between solo and collaborative much less personality-driven and much more situational.

tension or counterpoint

it is fair to push back on this. bad collaboration is worse than careful solo work. meetings without decisions, design reviews that turn into ego contests, committees that flatten everyone's good ideas into the lowest common denominator, and "let us all weigh in" as a way of avoiding ownership are real failures, and pretending otherwise is naive. the argument for working together only holds when the collaboration itself is healthy, with clear ownership, real candor, and a shared incentive to ship.

there is also a real risk that "let us collaborate" becomes a way to spread responsibility for choices people do not want to defend. that is a different problem than the one this post is making the case for, and it is worth naming. healthy collaboration sharpens decisions. unhealthy collaboration dissolves them, and the right response to unhealthy collaboration is to fix the team norms, not to retreat into solo work and call it focus. the goal is not "do everything together" or "do everything alone". the goal is to use the right mode for the right work, and to invest in the team norms that make collaboration actually pay back when it is the right call.

closing

if i had to pick one rule, it is this. build the kind of working relationships where it costs you less to be questioned than to be wrong. that is the version of teamwork that actually pays back, and it is the version that lets you go solo with confidence when the moment calls for it, because you know you are not gambling alone every time you do.

solo speed is still on the menu, and i still reach for it when the task is small enough that the cost of being wrong sits squarely with me. but the durable wins, the ones i look back on a year later and feel good about, almost always have someone else's fingerprints on them. that is not a coincidence. that is the trade working as intended.

related on this site

keep your snapshots simple

Philip Hern — Sat, 09 May 2026 14:11:26 +0000

snapshots are one of those features that feel like a free win the first time you reach for them. dbt handles the merge, the warehouse handles the storage, and suddenly you have a tidy history of every change a source row has ever gone through. then you ship a few of them, leave them running for a while, and discover that the maintenance, the backups, and the recovery story are quietly the most expensive parts of your pipeline.

i still use snapshots, but the rule i hold myself to is short. use them only when i absolutely have to, never on top of another snapshot, and never as a substitute for logic i could recompute deterministically. the rest of this post explains why.

quick answer

dbt snapshots implement the type 2 slowly changing dimension pattern. they track every change to a source row by closing the previous version and inserting a new one with dbt_valid_from and dbt_valid_to columns that define when each version was current. they are powerful when the source overwrites history and you genuinely need point-in-time answers, but they are expensive to operate, brittle under source-schema or grain changes, and impossible to fully rebuild from scratch once you have lost the original change events. the practical rule is to use the smallest number of snapshots you can get away with, never let one snapshot read from another, and recompute everything else deterministically.

who this is for

analytics engineers and data engineers who already use dbt and are deciding when to reach for snapshots
teams that have inherited a snapshot-heavy project and are trying to reduce the operational tax
anyone who has felt the pain of a corrupted or backfilled snapshot and wants a more conservative pattern going forward

why this matters

snapshots are different from every other model in your dbt project. a regular model is a pure function of its inputs, and you can drop and rebuild it any time. a snapshot is stateful, meaning its output depends on every prior run, the order those runs happened in, and the source values that existed at the moment each run executed. once that state is wrong, no amount of dbt run --full-refresh will fix it for you, because the historical events that produced the original sequence of rows are gone.

that is the trade you are making when you adopt a snapshot. you are giving up reproducibility in exchange for history capture. that trade is worth it for some sources, but it is far less common than people assume. most of the time, what you actually need is either a deterministic transformation, a freeze calendar, or a properly modeled effective satellite. snapshots should be a small, deliberate slice of your warehouse, not a default.

type 2 scd: a quick refresher

before going further, here is a short reminder about what type 2 actually means. the dimension community talks about slowly changing dimensions in numbered types because the trade-offs are very different across them. the most common ones are:

type 0: never change the value, even if the source does
type 1: overwrite the value in place, no history kept
type 2: keep every version of the row across time, with start and end timestamps that mark when each version was current
type 3: keep a small number of prior values in additional columns on the same row (for example previous_status)

type 2 is the only one that gives you a complete record of how an attribute evolved. the trade-off is that the row count grows every time something changes, and every consumer has to know how to filter the table to get the version they want.

what type 2 looks like in a row

imagine a customer table where the status column changed twice. with type 2 history, the same customer_id shows up multiple times, with non-overlapping validity intervals.

customer_id	status	valid_from	valid_to
1001	prospect	2026-01-01 00:00:00	2026-02-15 09:30:00
1001	active	2026-02-15 09:30:00	2026-04-22 14:10:00
1001	churned	2026-04-22 14:10:00	9999-12-31 23:59:59

a few things in that table do most of the work:

the business key (customer_id) is no longer unique on its own, so the grain is now customer_id plus the validity interval
valid_from is inclusive and valid_to is exclusive in most type 2 conventions, so the intervals tile cleanly without overlap
the current row uses a sentinel like 9999-12-31 23:59:59 instead of null, which makes downstream filters cleaner
to answer "what was the status at time t", you filter where valid_from <= t and valid_to > t
to answer "what is the status now", you filter where valid_to = '9999-12-31 23:59:59'

if any of that feels familiar from data vault, that is because effective satellites are essentially the same idea expressed in vault vocabulary. i wrote about a related grain trap in left join an effective satellite without duplicating rows, and the same care applies here. the moment you have validity intervals, every join and every filter has to respect them.

what a dbt snapshot is

a dbt snapshot is dbt's built-in implementation of type 2 history capture. you write a query that returns the current state of a source, and dbt takes responsibility for comparing that current state against the snapshot table on every run, closing rows that changed and inserting new versions. the columns it adds are:

dbt_valid_from: when this version became current
dbt_valid_to: when this version stopped being current (or the sentinel for current rows)
dbt_scd_id: a hash of the unique key plus dbt_valid_from for stable surrogate identity

i wrote a longer companion piece on the practical migration story in dbt snapshots, moving from merges to native history. that post is the "how to do it well" view. this post is the "how to do less of it" view.

when a snapshot is the right call

reach for a snapshot when all of the following are true:

the source system overwrites the row in place and does not retain history of its own
you genuinely need point-in-time answers, not just "the current value"
you cannot reconstruct the historical state from a deterministic formula or from another system that already keeps history
the source has a stable grain and a reliable unique key
you can guarantee the snapshot will run on a cadence that catches every change you care about

if any one of those is false, a snapshot is probably the wrong tool. for example, if the source already publishes change events to a message broker, capture the events into a regular append-only table and model on top of that. if the value is a deterministic function of other inputs (for example a derived score from frozen reference data), recompute it. if the source has a cycle_id or some other natural temporal key, join on that key directly instead of leaning on validity intervals.

the cost: complexity, brittleness, maintenance, backups

this is the part most adoption guides skip over. snapshots look free in the demo and feel free for the first month. then the bills start to come in.

complexity in the dag

a snapshot is a node in your dag, but it does not behave like other nodes. it is the only model type that has hidden state from prior runs, and it requires its own command (dbt snapshot) on its own schedule. a dbt project that contains snapshots has, in practice, two pipelines that have to stay in lockstep, namely the regular dbt build and the snapshot pipeline. when one of them lags or fails, downstream consumers see stale or partial history and the symptoms can be subtle.

every snapshot also forces every downstream model that reads it to think about validity intervals. queries that used to be a simple select now need a current-row filter or a point-in-time predicate, and reviewers have to verify that filter on every change.

brittleness under change

snapshots are unusually sensitive to upstream changes. a few examples i have seen close up:

a source query is widened to include a new column, and the check strategy now flags every row as changed on the next run, doubling the table overnight
a source briefly drops keys (because of a partial backfill or a bad upstream join), and a hard_deletes = invalidate snapshot closes thousands of rows that are still valid
duplicate keys appear in the source for a single run, and the snapshot either fails or quietly bloats with overlapping intervals
the source schema changes type on a column, and the snapshot now refuses to merge because the staged data and the historical data disagree

each of these takes a careful, surgical fix. you cannot just rerun the snapshot from scratch, because the original sequence of source values is gone.

maintenance and backups

because snapshot output is not reproducible, you have to treat the snapshot table itself as production data, not a derived artifact. that means:

regular backups of the snapshot tables to a separate schema or storage location, with a retention policy you actually enforce
a documented recovery runbook for partial corruption (for example, restore from backup, replay only specific keys, validate intervals)
alerting on row-count deltas, row-count ratios per run, and anomalies in dbt_valid_to distributions, so a misconfigured run does not run for a week before anyone notices
change review for any edit to the snapshot definition, because changing check_cols, unique_key, or the source query can rewrite history in non-obvious ways

a regular dbt model needs none of that. a snapshot needs all of it, and the cost scales with the number of snapshots in your project.

the worst pattern: snapshots on top of snapshots

if there is one rule i would carve into the wall, never build a snapshot on top of another snapshot. mixing two type 2 tables produces validity intervals on top of validity intervals, and the result is almost never what anyone wants.

why this is so dangerous

a single type 2 table is already a careful object. its grain is the business key plus the validity interval, and every consumer has to filter to a single moment in time before doing anything else. when you stack a second snapshot on top of it, you are now tracking history of a thing that was already historical, and the questions you can sensibly ask multiply in ugly ways.

think about what the row count of snapshot_b becomes when its source query reads from snapshot_a without point-in-time filtering. for any business key, you get the cartesian product of versions, which means changes in snapshot_b get attributed to the wrong intervals of snapshot_a. even if you do filter for current rows, the second snapshot will react to every change in the first, including changes that have nothing to do with the attributes you care about, so you end up with a much noisier history than you wanted.

even if you carefully filter the inner snapshot to its current row, you have lost something important. the outer snapshot now records history of a moving target. when you reread the outer snapshot at a past timestamp, the row you get back was generated against the inner snapshot's current state at the time the outer run executed, not against the inner snapshot's state at the same past timestamp. this is the validity-on-validity trap, and it is almost impossible to reason about by inspection.

what to do instead

if you find yourself wanting to build a second snapshot on top of a first, treat that as a signal that the design is wrong, not as a problem to solve in sql. a few healthier alternatives:

have one snapshot per source object that genuinely needs history, and read it directly in your information delivery layer
if you need a derived attribute that depends on a snapshot, compute that attribute in a regular view that filters the snapshot to a single point in time and is itself recomputable
if the derived attribute genuinely needs its own history, snapshot the source inputs independently and join them by point-in-time logic in a downstream view, instead of stacking the snapshots themselves
if your business has a natural temporal key (cycle, period, year), prefer joining by that key over inferring history from validity intervals

the goal is to push as much of the temporal logic as you can into deterministic transformations, and keep the snapshots themselves at the edges of the dag.

less is more, simple is better

most of the temporal questions you think need a snapshot do not. before adding one, run through this short checklist:

is the value already historical somewhere upstream, in events, in a cycle table, or in another system, where i can read it without snapshotting myself?
can i compute the value deterministically from current inputs, so any past answer is just a recomputation against frozen reference data?
is the source overwriting history in place with no other record of the prior value?
if i never built this snapshot, would consumers really lose information they care about, or just convenience?

if the answer to either of the first two is yes, do not add a snapshot. if the answer to the third is no, do not add a snapshot. if the answer to the fourth is "just convenience", do not add a snapshot.

what you are aiming for is a warehouse that is mostly deterministic, with a small ring of carefully managed snapshots at the edges. the deterministic core is cheap to rebuild, easy to test, and forgiving to refactor. the snapshot ring is where the real cost lives, so you want it to be small enough that you can afford to back it up, monitor it, and recover it when something goes wrong.

simple beats clever here. one well-run snapshot you understand is worth ten clever snapshots that nobody can rebuild.

faq

when is a snapshot definitely worth it?

when the source overwrites in place, you have a real business need for point-in-time answers, and there is no upstream event log or cycle key to lean on instead. operational systems that mutate rows without retaining history are the canonical case.

what is the single biggest mistake people make with snapshots?

reading from a snapshot in another snapshot's source query. the validity-on-validity trap is the worst class of bug to debug, because the symptom shows up far away from the cause and the table looks plausible at a glance.

how do i reduce the number of snapshots in an existing project?

start with the snapshots that are read by the smallest number of downstream models, and ask whether the consumers really need history or just the current row. if they only need current, replace the snapshot with a regular view. for the snapshots that genuinely need history, make sure each one is independent and that nothing else in the project reads a snapshot to feed another snapshot.

should i ever full-refresh a snapshot?

almost never in production. a full refresh wipes the historical rows that no longer match the current source, which is the entire reason you built the snapshot in the first place. treat the snapshot table like operational data, not a derived artifact.

references

on the plane, again

Philip Hern — Sun, 26 Apr 2026 22:35:23 +0000

thesis

my opinion on using wifi on a plane has shifted. i do not think it is the right default for everyone in every situation, but when i am traveling alone, especially for work, i have started to treat a connected cabin as a feature. it takes hours that used to feel like pure waiting, time i was just trying to burn through, and turns them into a stretch where i can work with a surprisingly solid level of focus and relatively few distractions.

context

a few weeks ago i wrote about how torn i still was on this topic in plane wifi: when the cabin forced disconnect. that piece was an honest inventory of the tradeoffs. this one is an update from the other side of the choice, after i have spent more flights actually buying the pass and sitting down to work instead of debating it.

argument

when i am alone and the trip is for my job, the row stops feeling like a cage and starts feeling like a quiet room with bad legroom. i already have headphones in, so the cabin noise is under control. notifications are fewer than at my desk, nobody is making noise by my office door, and the margin of "things i could be doing instead" feels narrower. it is not peace and it is not deep rest, but it is a usable kind of concentration.

i am now using that block to write, to debug, to plan, and to close loops i would otherwise push to after i land. i go in knowing i will get a meaningful amount done, and that expectation makes the clock feel less stuck. the time still passes at the same speed, but it passes with output attached, and that changes how it feels in my body.

i also have a concrete proof point that this mode is not just talk. i completely stood up my personal website while airborne, end to end, in one of those sessions. right now i am on a plane again, getting ahead for an in-person meeting so i can walk in prepared instead of scrambling on the jet bridge.

tension or counterpoint

i am not arguing that every person should pay for wifi on every flight. shared trips, family logistics, the middle seat, motion sickness, or the simple need to be offline are all good reasons to skip it. economy is still a bad default office for anyone who needs space or quiet that the cabin cannot give. my point is narrower. for me, in the situations where it fits, the cost of the pass is cheaper than the opportunity cost of treating the whole flight as lost time.

closing

i will probably still sometimes want the cabin to be an excuse to be unreachable. when i do, i can leave the wifi off. when i do not, i am glad the option exists, and i am using it on purpose.

related on this site

logic

Philip Hern — Sun, 26 Apr 2026 22:35:21 +0000

thesis

learning basic logic is one of the most useful, durable skills i can recommend to anyone, regardless of profession. the english-language version of if/then/else is a thinking tool that works everywhere, never expires, and quietly compounds into better decisions over a lifetime.

context

most people associate logic with code, math, or a philosophy classroom. that framing is too narrow. logic is just structured cause-and-effect thinking, and the simplest version of it sounds exactly like english:

if this, then that, else that other thing

once you can hold that pattern in your head on purpose, it changes how you plan, diagnose, design, and interpret almost everything in front of you. you do not need a programming language to use it. you just need to be willing to slow down for a beat and think in branches instead of straight lines.

a few familiar gates

here a few small pictures to demonstrate simple examples of logic gates that you already use in daily life. this just illustrates it so you can literally follow along with the logic gates as each situation progresses.

and both must be true for the outcome to be true
or at least one true is enough
not you follow the opposite branch of the test

the flow is the same kind of small chart you would sketch on a napkin.

{{< mermaid >}}
flowchart TB
subgraph g_and [and, both need to be true]
direction LR
w[good weather?] --> and1{and}
f[afternoon free?] --> and1
and1 -->|yes| hike[go hiking]
and1 -->|no| home[stay in]
end
subgraph g_or [or, at least one is enough]
direction LR
car[friend can drive?] --> or1{or}
bus[transit is running?] --> or1
or1 -->|yes| go[you can get there]
or1 -->|no| stuck[you are stuck]
end
subgraph g_not [not, the opposite branch of the test]
direction LR
pow[power on?] -->|no| br[check the breaker]
pow -->|yes| next1[next check in the chain]
end
{{< /mermaid >}}

none of that requires a keyboard. it is the same branch habit as the if/then/else line in the last section, just drawn with a few boxes.

argument

logic is a thinking tool, not a coding tool

the if/then/else pattern is older than any programming language. when i write a small script, i am formalizing the same branching i already do when i pick what to wear, route around traffic, or decide how to respond when something at work breaks. the keyboard is incidental, the structure is the point.

this kind of structured thinking is what moves me from "i feel stuck" to "what is the next decision, and what are the branches under it". that small shift, from a vague feeling to a concrete branch point, is where most of the leverage comes from.

where it pays off in normal life

once you start noticing branches, you see them everywhere. planning a day with kids becomes a small logic tree. if the weather holds, we hike. if it does not, we move to the indoor option. if both fail, we cancel and reschedule. naming the branches up front means the day does not collapse when conditions change.

troubleshooting has the same shape. when something is not working, i walk a tree out loud. if the appliance has power, then check the next link. if not, then check the breaker. each step rules out a branch and shrinks the search space.

designing a process at work has the same bones. i list the conditions first, then the path each one takes. naming the branches early makes the design easier to explain and easier to fix later.

understanding behavior is harder, but the structure still helps. people are not perfectly logical, but their patterns often are. if my kid is tired, then certain tantrums become more likely. if a colleague is overloaded, then certain reactions track. recognizing the antecedent makes the response less personal and easier to handle.

reverse engineering is the same thing run backward. when i look at a result and want to understand how it got there, i walk the logic in reverse. if this output exists, then these inputs and conditions must have been true. if not, then the model i had in my head is wrong, and that gap is useful information on its own.

the tool never goes out of style

frameworks change. tools change. programming languages come and go. if/then/else does not. it is a structure of thought, not a piece of technology, which is why it keeps working in domains it was never designed for. cooking, parenting, negotiations, medical decision trees, customer support scripts, and legal arguments all lean on the same scaffolding.

i find real comfort in skills that age well. so much of what i learn in tech has a short half-life now. logic does not. once i have it, i have it for good.

applying it broadly is what makes it powerful

a tool that works in one place is useful. a tool that works everywhere is leverage. logic works everywhere because every domain has cause and effect, conditions, and outcomes. that generality is the multiplier, and it is the same kind of cross-domain value i wrote about with adaptability. the principle stays steady while the surface details swap.

learn it as early as you can

the earlier this gets internalized, the more downstream decisions inherit it. a kid who can think in branches asks better questions, accepts fewer "because i said so" answers, and gradually builds a habit of checking conditions before reacting. that habit then runs in the background for the rest of their life.

i think about this with my own kids, and i think about it for myself. every year i wait to make decisions more deliberately is a year of slightly noisier decisions stacked behind me.

the butterfly effect of better decisions

small improvements in single decisions do not look like much in isolation. two paths that differ by one degree at the start can end up far apart over a long enough timeline. better daily decisions, even by a thin margin, compound the same way a little becomes a lot does for habits. the quality of the inputs, repeated across years, becomes the quality of the life.

logic is one of the cheapest ways i know to nudge that compounding in a good direction.

tension or counterpoint

logic on its own is not the whole answer. real situations carry emotion, ambiguity, missing information, and people who do not behave according to clean rules. if i treat every interaction like a flowchart, i lose intuition, empathy, and the ability to sit with uncertainty.

the skill is to use logic as scaffolding, not as a replacement for judgment. i map the branches i can see, then i listen for the part that the branches do not capture. both layers matter, and the logical part actually helps the intuitive part by giving it a clean place to stand.

closing

this is one of the cheapest, most durable investments anyone can make. learn the english-language form of if/then/else. practice naming the conditions and the branches in your own life. apply it to planning, troubleshooting, designing, and understanding the people around you.

learn it once and you keep it forever. apply it everywhere and it compounds. not many skills pay back like that.

related on this site

back at it

Philip Hern — Thu, 23 Apr 2026 12:27:13 +0000

thesis

this is a small checkpoint post. the heavy lift is not finished, but i am out of the weeds for now, and that is worth naming out loud.

context

the same work i was carrying in stick with it kept growing in weight and surface area. for a while it felt like one endless tangle. i stayed with it anyway, and eventually i approached it the way i should have from the start, as smaller chunks that stack into the much larger change. each piece still had to be real, but the sequencing and scope finally matched how my head and the system can tolerate change.

argument

getting to a stable point did not erase the backlog. i still have more testing to run, more simulations to exercise, and real user acceptance testing ahead. the difference is that the foundation is no longer thrashing. errors and surprises have a place to land without undoing everything at once.

that stability is what gave me room to breathe. i can take a short break on purpose, look at the whole arc with a little distance, and come back to the tuning work with less panic and more optimism. the remaining work is still serious, but it is the kind of serious that fits a calendar instead of the kind that owns every waking hour.

tension or counterpoint

a stable checkpoint is not the same as done. if i confuse relief for completion, i will skip validation i still need. the discipline now is to rest without pretending the job is closed.

closing

so i am back at it in a different posture, not firefighting the whole shape at once, but finishing the test matrix, listening to users, and dialing things in with a clearer mind. sticking with it got me here. the next stretch is about proving it in the world, calmly.

related on this site

stick with it

Philip Hern — Fri, 17 Apr 2026 13:24:41 +0000

thesis

this one is for me as much as anyone reading. the single most important thing i can do on a long, hard project is keep showing up for it. motivation rises and falls, energy comes in waves, and neither of those things matter as much as continuity. if i stay with the work long enough, the payoff arrives, even when progress is invisible for stretches in the middle.

context

i am in the middle of a very heavy lift right now. it started as a change i thought i would finish quickly, and it has turned into something much bigger. the effort, concentration, and validation required are more than i am used to, and the timeline has stretched well past what a typical change would take. the stress is real. i feel it in how i think about the project before bed and how quickly i reach for my laptop in the morning.

i am still in it, though, because the value at the end is worth the cost. when this lands, i will have more stable and explainable historical data, which means my ongoing workload of troubleshooting data validity questions drops. less firefighting later is worth more pressure now, and that tradeoff is the only reason i would keep going through a change this heavy.

argument

continuity beats intensity

motivation is a wave, not a rope. it pulls me forward for a while, then it lets go, then it comes back later with a different shape. if i tie my progress to the wave, i stop whenever the wave stops. if i tie my progress to the habit of showing up, the wave cannot take the project down with it. that is the same pattern i wrote about in little by little, a little becomes a lot, just applied to a single long problem instead of a daily practice.

isolate your changes, even in your own playground

the hardest lesson from this round is about isolation. i have been testing work in an environment i consider my playground, and for a long time that has been fine. this time, my changes broke downstream consumers, and the pressure immediately escalated because other people were suddenly blocked. the takeaway is simple. if my changes can reach downstream consumers, i need to separate my testing from a shared test environment, regardless of how freely i am used to moving in that space. a playground still has neighbors.

do not try to lift several objects at once

i also tried to move multiple pieces of the system at the same time. i thought bundling them would be faster. what actually happened is that each piece depended on the others in a way that made every single one harder to validate, and the total stress grew faster than the total work. smaller, sequential chunks would have finished sooner and felt calmer. one object at a time, even if it feels slower on paper, is almost always faster in practice.

better preparation shrinks the stress

the last lesson is about preparation. i went in expecting a small change and i prepared like it was a small change. when the scope grew, my preparation did not grow with it, and that mismatch is where the break points appeared. better preparation up front, regardless of how small i thought the task was, would have reduced both the stress and the number of places things could go wrong. the cost of preparing for a bigger job than you need is tiny. the cost of not preparing for the job you actually have is not.

tension or counterpoint

persistence is not the same as refusing to reassess. sticking with every hard thing forever is just sunk cost fallacy wearing a motivational t-shirt. the honest check i keep running is whether the value at the end is still real and still mine. if the answer is yes, i keep going. if the answer turns into no, i stop, and that is not quitting, that is discernment.

there is also a stress cost to "push through" language. if the pressure is spilling into health, relationships, or judgment, that is a signal to change the pace, not a signal to try harder. pushing through is a tool, not a strategy, and it only works when i also rest and isolate the work properly. that is part of why i think it helps to get comfortable being uncomfortable without confusing discomfort for permission to keep grinding.

closing

so this is my note to myself. keep going. the work is real, the value is real, and the lessons i am collecting on the way are already paying off for the next change. next time i will isolate my testing better, break the work into one object at a time, and prepare like the task is bigger than i think it is, because it almost always is.

and if the wave of motivation dips again tomorrow, that is fine. waves dip. what matters is that i still show up, finish one more piece, and trust that continuity is the actual engine. stick with it.

related on this site

comfortable being uncomfortable

Philip Hern — Fri, 10 Apr 2026 19:29:32 +0000

thesis

i want to normalize a simple idea that still feels hard in practice. getting outside of your comfort zone is not a side quest. it is the main mechanism by which you stretch, learn, and see the world with more room in it for other people. yes, it is uncomfortable, and that is exactly the point.

context

most of us are trained to seek stability. stability is not bad, but it is also not where the adaptation happens. when everything feels familiar, your brain is mostly rehearsing what it already knows. the moment you step into something new, the cost shows up immediately as awkwardness, uncertainty, or fear of looking foolish. that friction is not a sign you chose wrong. it is often a sign you chose honestly.

argument

change is disruptive by definition. if it did not interrupt your default patterns, it would not be change. i think we should embrace that disruption more often, because it is where new experiences actually enter your life. without that interruption, you mostly get repetition with better packaging.

the growth part is not theoretical. discomfort is where skills get pressure-tested. you learn not only how to do things, but how not to do things, which is just as valuable and often faster feedback. mistakes in public or under stress are expensive emotionally, but they are also unusually clear. they show you boundaries, preferences, and limits in a way that a comfortable afternoon rarely will.

more experiences also broaden your worldview in a practical sense. when you have seen more contexts, constraints, and ways people solve problems, it becomes harder to treat your own habits as universal law. that widening tends to produce more tolerant and compassionate attitudes, not because tolerance is a slogan, but because you have more firsthand evidence that reasonable people can live and work in very different, equally valid ways.

so my encouragement is simple. expose yourself to new experiences on purpose. seek situations where the pressure is on you to perform, because that is where you rise to the occasion and discover how capable you can be. it is also where you might discover that this is not your thing, and you should move on. either outcome is a win, because both give you self-insight you cannot fake. you learn what energizes you, what drains you, and what you are willing to practice until it gets easier.

this connects to how i think about adaptability in general. comfort is a resting state. adaptation requires movement.

tension or counterpoint

there is a real downside to glorifying discomfort without boundaries. not every challenge is worth the cost, and not every "growth opportunity" is ethical or safe. pushing yourself is different from letting yourself be pushed past your values or health. the goal is not suffering for its own sake. the goal is chosen stretch, with recovery and discernment built in.

closing

i am not asking for constant chaos. i am asking for a bias toward the new when you can afford it, and toward the high-stakes try when you are ready. the uncomfortable path is where you find out who you are when the easy defaults are not available, and that knowledge is about as practical as it gets.

related on this site

dbt snapshots: moving from merges to native history

Philip Hern — Fri, 10 Apr 2026 19:29:31 +0000

quick answer

dbt snapshots provide a native way to track slowly changing dimensions over time. by migrating from custom merge statements to native dbt snapshots, you can simplify your codebase, rely on built-in history tracking, and ensure your downstream models always have access to point-in-time records.

who this is for

audience: data engineers and analytics engineers using dbt
prerequisites: basic knowledge of dbt models, sql, and data warehousing concepts
when to use this guide: when you need to track historical changes to mutable source records and want to move away from manual merge logic

why this matters

tracking historical changes is a common requirement in data warehousing. building custom merge logic to handle inserts, updates, and history tracking is error-prone and difficult to maintain. dbt snapshots handle the heavy lifting of history tracking out of the box. this ensures you do not lose historical context when source systems overwrite data.

moving from merge to snapshot

recently, i migrated several historical tables from a custom merge strategy to native dbt snapshots. the previous approach relied on complex merge statements that manually checked for changes and inserted or updated rows to maintain history. this was difficult to read and even harder to debug.

by adopting native dbt snapshots, the logic became declarative. instead of writing the exact update and insert commands, i only needed to define the source query and configure how dbt should detect changes. the downstream consumer views then filter the snapshot output to return the current row or a point-in-time record.

the core shift in thinking

when using snapshots, your snapshot definition should remain source-representative. do not apply business date-window filtering in the snapshot definition itself. instead, capture the raw history and apply your logic for which rows to return in downstream consumer views.

for example, to get the current row in a downstream model, you filter using the sentinel value:

select *
from {{ ref('my_snapshot_st') }}
where dbt_valid_to = '9999-12-31 23:59:59'

to get a freeze record for a specific point in time, you derive a freeze timestamp and filter:

select *
from {{ ref('my_snapshot_st') }}
where dbt_valid_from <= freeze_ts and dbt_valid_to > freeze_ts

basic example

here is a basic example of a dbt snapshot using the check strategy. this snapshot tracks changes to a practice affiliation table.

{% snapshot practice_affiliation_st %}

{{
    config(
        target_schema = 'snapshots',
        strategy = 'check',
        unique_key = ['fmno', 'cycle', 'committee', 'hierarchy'],
        check_cols = 'all',
        hard_deletes = 'invalidate',
        dbt_valid_to_current = "to_timestamp_ntz('9999-12-31 23:59:59')"
    )
}}

select
    fmno,
    cycle,
    committee,
    practice_name,
    type,
    hierarchy
from {{ ref('source_practice_affiliation_v') }}

{% endsnapshot %}

configuration options

dbt snapshots offer several configuration options that control how changes are detected and recorded. you can read more about these in the official dbt snapshot documentation.

here are the key options and what they control:

target_schema: the schema where the snapshot table will be built
strategy: determines how dbt detects changes, with the two main options being timestamp and check
unique_key: the primary key of the record, which can be a single column or a list of columns for a composite key
check_cols: used with the check strategy to specify which columns to monitor for changes, accepting a list of column names or the word all
updated_at: used with the timestamp strategy to specify the column that indicates when the source row was last modified
hard_deletes: controls how dbt handles rows that disappear from the source, such as setting it to invalidate to close the current row when a key is no longer present
dbt_valid_to_current: overrides the default null value for current records, allowing you to set a far-future date to make downstream filtering easier

timestamp vs check strategy

the choice between timestamp and check strategies is critical.

use the timestamp strategy when your source has a reliable updated column that changes whenever the row changes. dbt compares the source timestamp to the snapshot timestamp to decide if a new version is needed.

use the check strategy when you do not have a reliable updated timestamp, or when you want to detect any change in a specific set of columns. dbt compares the actual values of the check columns between the source and the current snapshot row. if any checked column differs, dbt closes the current row and inserts a new version.

in my recent work, i found that the check strategy with all columns checked and a composite unique key was the most robust approach for sources where the updated timestamp was synthetic or not authoritative.

gotchas and lessons learned

migrating to snapshots surfaced a few important lessons:

upstream scope gating: if your upstream source query includes filters that remove keys, and you have hard deletes configured to invalidate, dbt will intentionally close the current rows for those missing keys
composite keys: dbt fully supports composite unique keys, and passing a list of columns ensures that dbt tracks history at the correct grain
duplicate source rows: snapshots expect the source data to be unique at the unique key grain, so if your source contains duplicate keys, the snapshot will fail or bloat
defensive deduplication: in some cases, i had to add a defensive qualify row number guard in the snapshot definition to collapse known duplicate-key source rows before dbt processed them
sentinel values: using a sentinel value for current rows instead of null makes downstream queries much cleaner, allowing you to use an equals operator instead of checking for nulls

deployment and automation

snapshots are not updated automatically when you run your standard dbt build commands. they require a dedicated command: dbt snapshot.

if you do not automate this, your history tracking will be manual and prone to gaps. to ensure continuous history capture, you must schedule the snapshot command to run on a regular cadence.

in a production environment, this usually means setting up a continuous integration workflow or an orchestrator task. for example, you can use automated workflows to run snapshot tags on daily, hourly, or monthly schedules.

a typical automated workflow might look like this:

a scheduled trigger fires the workflow
the workflow checks out the repository and sets up the dbt environment
the workflow executes the snapshot command for specific tags
dbt connects to the warehouse, compares the source data to the existing snapshot tables, and applies any necessary inserts or updates

by decoupling the snapshot schedule from your standard model runs, you can capture history at the exact frequency your business logic requires.

references

little by little, a little becomes a lot

Philip Hern — Tue, 07 Apr 2026 23:13:24 +0000

thesis

the importance of just trying to do a little each day cannot be overstated. we often overestimate what we can accomplish in a single afternoon, but we vastly underestimate what we can build over a year of sustained effort. incremental changes add up, and what seems like a drop in the bucket today becomes a reservoir over time.

context

whether it is work, fitness, or building new routines, habit forming often feels like it takes forever to take hold. we live in a world that expects immediate results, and we naturally get frustrated when the scale does not move or the project does not finish overnight. when that initial burst of motivation inevitably fades, the reality of the daily grind sets in, and that is exactly when most people decide to walk away.

argument

this process is a little easier when you understand you are working toward continuity and not perfection. it is not about executing flawlessly every single day, nor is it about never taking a break. it is about showing up and putting in the reps, even when the effort feels small or uninspired. missing one day is just a bump in the road, as long as you do not let it become two days in a row.

i am starting to see the rewards of that mindset now. little by little, my experience has added up into a foundation i can actually rely on. little by little, i have started sharing via my website, turning scattered thoughts into a structured body of work. little by little, i am starting to have a greater reach and help more people, simply because i chose to publish something small rather than waiting for the perfect masterpiece.

tension or counterpoint

the hardest part is trusting the process when the visible progress is zero. it is incredibly easy to quit when you do not see the immediate payoff of your daily effort. it feels like you are just watering dirt for weeks on end. but the compounding effect of showing up is real, even if it remains completely invisible in the short term.

closing

so right in theme, i will keep this short and keep focusing on the small, daily inputs rather than the distant outputs. the goal is simply to keep the chain going, trusting that a little becomes a lot when you give it enough time.

related on this site

how i automated dev.to and linkedin publishing so visibility stops depending on memory

Philip Hern — Sun, 05 Apr 2026 14:03:19 +0000

after i started writing more consistently, it became obvious that writing is only half the work; distribution is the other half. i wanted a system where i can publish from one canonical source and let automation push the same story to dev.to and linkedin.

quick answer

i set up two publish automations that watch my post changes and sync them to dev.to and linkedin. the first publish creates the post on each platform, and later edits update the same external post instead of creating duplicates. this gives me consistent visibility without adding manual publishing steps after every article.

who this is for

people who publish technical writing and keep forgetting cross-posting
creators who want one canonical source plus repeatable distribution
builders who care about discoverability as much as writing quality

why this matters

if distribution is manual, it eventually slips. then strong posts sit unread because i forgot to copy, paste, format, and re-share them across platforms. automation solves that by making visibility part of the same delivery path as the content itself.

this is the same pattern i described in a practical ai workflow: jira, github, and mcp, define one clear source of truth, then automate the handoff steps so i can spend more time on thinking and less time on clerical work.

step-by-step

1) define the starting point

i chose my site post as the only canonical source. every external platform receives content from that source, not from separate drafts. this keeps language, links, and updates aligned over time.

2) apply the change

i added automation for both targets:

trigger on post updates and support manual runs when i want a full backfill
create posts when no external mapping exists
update existing external posts when a mapping already exists
keep a small state map so each canonical url stays attached to one external post id

the practical result is that i can keep writing in one place and trust the sync layer to handle distribution. this complements the writing habits from my cursor setup, where reusable workflows remove repeated manual work.

3) validate the result

i test in three passes:

dry run to confirm detection and decisions without publishing
publish-all run to verify initial backfill behavior
normal change-trigger run to verify incremental updates on later edits

when all three pass, i know the pipeline is reliable enough for daily use.

faq

what was the biggest setup mistake?

token and redirect mismatches during oauth were the main failure point at first. once i aligned scopes, callback values, and secret placement, the automation became stable.

should i keep manual publishing as a fallback?

yes, especially while you are in early setup. after the workflow proves stable, manual publishing becomes a recovery path instead of a default habit.

references

the future of data engineering workflows with ai

Philip Hern — Fri, 03 Apr 2026 14:11:59 +0000

quick answer

the future of data engineering workflows with ai is about moving from manual coding to intelligent orchestration. ai agents will handle boilerplate code, pipeline generation, and data quality checks, allowing data engineers to focus on architecture, governance, and business value.

who this is for

audience: data engineers, analytics engineers, data architects, and technical leaders.
prerequisites: an understanding of modern data stack concepts and basic ai principles.
when to use this guide: when planning your data strategy and evaluating how to integrate ai into your engineering practices.

why this matters

the volume and complexity of data are growing faster than engineering teams can scale. relying solely on manual workflows leads to bottlenecks, technical debt, and delayed insights. embracing ai is not just about efficiency, it is a strategic imperative to remain competitive.

step-by-step

1) define the starting point

traditionally, data engineering has been a highly manual discipline. engineers spend countless hours writing sql, configuring orchestrators like airflow, and debugging failed pipelines. this approach is brittle and scales poorly as the organization grows.

2) apply the change

the integration of ai changes this paradigm. large language models can now generate complex sql queries, translate between dialects, and even suggest optimal data models based on source schemas. ai agents can monitor pipeline health, automatically retry transient failures, and alert engineers only when human intervention is necessary. this shift transforms the engineer from a coder into a system architect.

3) validate the result

the impact of this transformation is measurable. development cycles shorten, data quality improves through automated testing, and the overall reliability of the platform increases. engineers spend less time firefighting and more time building scalable, resilient architectures that drive business decisions.

faq

what is the most important caveat?

ai is a tool, not a replacement for fundamental engineering principles. you still need a strong understanding of data modeling, governance, and security to build a robust platform.

what should i do first?

start by identifying the most repetitive tasks in your workflow, such as writing documentation or basic transformations. experiment with ai tools to automate these specific areas before attempting to overhaul your entire architecture.

references

the modern data stack

how i use cursor and ai agents to write dbt tests and documentation

Philip Hern — Fri, 03 Apr 2026 14:07:49 +0000

quick answer

writing dbt tests and documentation is often the most neglected part of data engineering. i use cursor and custom ai agents to automate this process by reading my sql models, inferring the business logic, and generating the corresponding yaml files. this ensures high-quality data pipelines without the manual overhead.

who this is for

audience: data engineers, analytics engineers, and developers using dbt
prerequisites: basic knowledge of dbt, sql, and cursor
when to use this guide: when you want to scale your data engineering practices and reduce the time spent on writing boilerplate yaml

why this matters

documentation and testing are critical for data trust, but they are tedious to write manually. when these steps are skipped, data quality suffers and debugging becomes a nightmare. by automating this with ai, you get the benefits of rigorous testing and clear documentation while freeing up your time for higher-value architectural work.

step-by-step

1) define the starting point

most data engineers start with a raw sql model and a blank slate for their schema.yml file. the traditional approach requires manually typing out every column name, description, and test. this is prone to human error and inconsistency, plus almost always falls out of sync with current models with the first change.

2) apply the change

i use cursor to bridge this gap. by creating specific ai rules and skills, i can highlight a dbt model and ask the agent to generate the documentation. the agent reads the sql, understands the joins and transformations, and produces a complete yaml file with standard tests like not_null and unique. it can even infer complex relationships and suggest custom tests based on the data domain.

3) validate the result

once the ai generates the yaml, i review it for accuracy. i then run dbt test and dbt docs generate to ensure everything compiles correctly. the ai rarely makes syntax errors, so the validation step is mostly about confirming the business logic aligns with the documentation.

faq

what is the most important caveat?

you must still review the generated output. ai is excellent at scaffolding and inferring patterns, but it does not possess the full business context that you do.

what should i do first?

start by creating a simple cursor skill that defines your team's standards for dbt documentation. feed it a few examples of your best schema.yml files so it learns your preferred style.

references

dbt documentation

DEV Community: Philip Hern

working together or alone

thesis

context

working together

what these checks actually catch

working alone

where solo work still earns its place

weighing the trade-off

tension or counterpoint

closing

further reading

related on this site

keep your snapshots simple

quick answer

who this is for

why this matters

type 2 scd: a quick refresher

what type 2 looks like in a row

what a dbt snapshot is

when a snapshot is the right call

the cost: complexity, brittleness, maintenance, backups

complexity in the dag

brittleness under change

maintenance and backups

the worst pattern: snapshots on top of snapshots

why this is so dangerous

what to do instead

less is more, simple is better

faq

when is a snapshot definitely worth it?

what is the single biggest mistake people make with snapshots?

how do i reduce the number of snapshots in an existing project?

should i ever full-refresh a snapshot?

references

related reading

on the plane, again

thesis

context

argument

tension or counterpoint

closing

further reading

related on this site

logic

thesis

context

a few familiar gates

argument

logic is a thinking tool, not a coding tool

where it pays off in normal life

the tool never goes out of style

applying it broadly is what makes it powerful

learn it as early as you can

the butterfly effect of better decisions

tension or counterpoint

closing

further reading

related on this site

back at it

thesis

context

argument

tension or counterpoint

closing

further reading

related on this site

stick with it

thesis

context

argument

continuity beats intensity

isolate your changes, even in your own playground

do not try to lift several objects at once

better preparation shrinks the stress

tension or counterpoint

closing

further reading

related on this site

comfortable being uncomfortable