DEV Community: Philip Hern

chapter 4 of ai-assisted data engineering is live

Philip Hern — Mon, 20 Jul 2026 14:00:41 +0000

thesis

chapter 4 of ai-assisted data engineering is live. it is called the editor and the workspace, and it opens the setup part of the book, where judgment gets an environment that can carry context instead of starting from a blank prompt every time.

context

after three chapters about ownership, judgment, and production standards, chapter 4 gets practical. the fastest way to work with agents is to shape the workspace once so the agent sees the system the way the work actually crosses it. it follows chapter 3, continuing the weekly release cadence for the book.

argument

what chapter 4 covers

this chapter covers:

why a multi-repo workspace makes the agent more useful
which editor settings and shortcuts matter more than tool fashion
how one rule and one skill can raise the quality floor
why setup should be verified before serious work depends on it

why it belongs here

chapter 4 sits here because the book is building one layer at a time. the early chapters established judgment and ownership. each release after that adds another practical surface where those standards either hold or fail.

it is easy to dismiss setup as tinkering. the chapter argues the opposite. setup is where repeated judgment becomes reusable context, and that makes every later agent run less fragile.

closing

this is another monday release in the serialized run. chapter 5 is next in the queue, and it will publish the following monday if the schedule holds. the full book landing page stays at ai-assisted data engineering, and the companion channel stays at philliant on youtube for any longer explanation that works better out loud.

related on this site

capability is not a reason to act

Philip Hern — Mon, 20 Jul 2026 13:56:34 +0000

thesis

having an ai agent at my side can feel like having a genie with unlimited wishes. i can ask it to adopt the newest pattern, add another automation, reorganize a system, or squeeze one more improvement out of something that already works. the old cost of turning an idea into action has fallen dramatically, but that does not make every idea worth acting on. capability is not a reason to act. before i use another wish, i still need to decide whether i need what i am asking for, whether it fits my situation, and whether i am prepared to understand and own the result.

context

scarcity used to impose restraint for me. a change took enough time and effort that i naturally asked whether it was worth doing. ai weakens that filter. when an agent can research a trend, draft an implementation, update the documentation, and check the result in one sitting, trying the idea feels almost free.

almost free is not free. every generated change creates something for me to review, learn, maintain, and eventually troubleshoot. the agent can keep producing without rest, but i cannot keep absorbing without limit. the genie has unlimited wishes. i still have one human-sized mind.

argument

the ability to do something says nothing about its value

an agent can make an idea possible without making it useful. those are separate judgments. the model answers "can this be done?" very quickly, but only i can answer "does this need to be done here?"

that second question should come first. what problem am i solving? who benefits? what happens if i leave the current system alone? what new responsibility will the change create? if i cannot give a clear answer, implementation speed is irrelevant. i am using capability to manufacture work rather than solve a need.

unlimited wishes encourage careless wishing

the genie metaphor matters because unlimited wishes change how i value each wish. when implementation was expensive, i chose carefully. when implementation feels abundant, it is easy to ask for things simply because i can.

that is how useful assistance turns into an improvement treadmill. one more abstraction, one more automation, one more layer of agent configuration. each addition may be defensible on its own, but together they can leave me with a system that has grown faster than my understanding of it. the agent did not overextend itself. i overextended the amount of generated work i could responsibly own.

trends are options, not instructions

ai makes following a new trend unusually easy. i do not need to master the tool before experimenting with it because the agent can read the documentation, scaffold the integration, and explain the unfamiliar parts as it goes. that can be valuable, but it can also remove the pause where i would normally ask whether the trend belongs in my work at all.

new does not mean necessary. popular does not mean relevant. a pattern that helps another team at another scale may add nothing to my situation except complexity. i want to treat each trend as an option to evaluate, not an instruction to obey. the right question is not whether the agent can bring it into my system. the right question is whether my system has a real problem that the trend solves better than what i already have.

consideration must come before action

the faster action becomes, the more deliberate the decision before it needs to be. my filter is simple:

what specific need exists
what evidence says the current approach is insufficient
why this change fits my situation
what i will have to understand and maintain afterward
what would tell me not to proceed

if those questions do not produce a convincing case, i do not need another prompt. i need to leave the system alone.

sometimes the best use of capability is restraint

restraint is not a rejection of ai. it is how i keep ai useful. the point of an agent is not to maximize the number of things i ask it to change. the point is to help me act well when action makes sense.

i have already written about giving work room to breathe after it ships. this is the decision that comes before that. room to breathe says to stop squeezing once the useful work is done. capability is not a reason to act says not to begin merely because the squeezing has become easy.

tension or counterpoint

there is a risk in becoming so cautious that i use "consideration" as an excuse never to experiment. some useful ideas only reveal their value after i try them, and ai makes small experiments cheaper than they have ever been.

the distinction is intent and containment. an experiment should test a specific question, within boundaries i understand, with a clear way to stop. chasing a trend because everyone is discussing it is not the same as running a focused experiment to learn whether it solves one of my problems. consideration does not prohibit action. it gives action a reason.

closing

the agent beside me may never tire, run out of ideas, or ask me to stop. that does not mean i should match its appetite for action. i am still the person who has to understand the changes, carry their consequences, and decide whether they made anything better.

so i am learning to leave wishes unused. i do not need to follow every trend, automate every process, or improve every thing that could be improved. the genie can wait. when a real need appears and the change makes sense for my situation, the capability will still be there. until then, not acting is also a decision, and often it is the better one.

related on this site

an efficient .cursor directory: less context, better agents

Philip Hern — Thu, 16 Jul 2026 20:27:42 +0000

i spent time recently reorganizing how my repositories load cursor context. the old setup worked, but it was heavy. every agent turn started with a large bundle of rules, duplicated guidance, and inventories the model did not need for most tasks.

the new setup is deliberately smaller. agents still find what they need, but they pay for less context up front. that means lower token cost, less time waiting for the model to "read the room", and answers that stay closer to the task i actually asked for.

if you already have my cursor setup in place, think of this post as the next layer. that post covers the full toolkit. this one covers how to arrange the toolkit so it does not eat your context window before work begins.

quick answer

treat your .cursor directory like a library with a card catalog, not like one giant PDF pasted into every prompt. keep a thin always-on layer for guardrails, route everything else through links, scope detailed rules to the file types they apply to, and load skills on demand. the agent can always open more context when the task requires it, but the default should be the minimum useful baseline.

who this is for

people who already use cursor rules, skills, or AGENTS.md and notice agents getting slower or noisier over time
teams whose .cursor folder grew organically and now repeats the same instructions in three places
builders who care about token cost and want cleaner first responses without giving up deep project knowledge

why this matters

context is not free. every always-on rule, every paragraph in AGENTS.md, and every skill description the system injects up front consumes tokens before the agent writes a single useful line.

that cost shows up in three places:

money, because larger prompts cost more per turn
latency, because the model spends time absorbing context you may not have needed for this task
focus, because a bloated baseline makes it easier for the agent to wander, repeat itself, or "helpfully" expand scope

the counterintuitive part is that less default context often produces better results. when the baseline is tight, the agent has fewer conflicting instructions to reconcile. when it needs sql conventions, it loads sql conventions. when it needs a release checklist skill, it loads that skill. the fetch is intentional instead of accidental.

the problem with a "kitchen sink" .cursor folder

most .cursor directories start sensibly. you add one rule for coding standards. then a skill for your most common workflow. then AGENTS.md grows because you keep answering the same architecture questions. then another rule duplicates half of the first one because you wanted sql-specific guidance.

soon every chat begins like this:

a long always-on rule with architecture, naming, validation, git policy, and documentation philosophy
a second always-on rule with overlapping standards
an AGENTS.md file that restates the same material in prose
a skill inventory table with fifteen rows, even when you only needed one skill

none of this is wrong in isolation. together it is expensive noise.

the pattern i use now: route, do not restate

the organizing principle is one canonical document per topic, indexes that route rather than restate.

think in layers:

layer	job	load profile
`AGENTS.md`	entry point, hard constraints, route-by-task table	small, always present
`.cursor/README.md`	local index of rules, skills, and subagents	small, linked from entry
`.cursor/rules/*.mdc`	enforceable standards	thin always-on + glob-scoped detail
`.cursor/skills/*/SKILL.md`	procedural runbooks	on demand when task matches
`.cursor/reference/*.md`	catalogs, contracts, long inventories	only when maintaining or choosing

layer 1: a thin `AGENTS.md`

AGENTS.md should answer two questions quickly:

what must never be violated on this repo
where do i go next for the task at hand

a useful shape:

# my-repo agent entry

one-paragraph architecture invariant.

## hard constraints

- link to the always-on behavior rule
- link to task-scoped rules
- one line on validation expectations

## route by task

| task                 | canonical source             |
| -------------------- | ---------------------------- |
| local setup          | README                       |
| cursor asset layout  | .cursor/README.md            |
| full skill inventory | .cursor/reference/catalog.md |
| domain architecture  | docs/architecture note       |

what to leave out:

full skill tables (link to a catalog instead)
long prose explanations of every layer in your data platform
step-by-step workflows (those belong in skills)

layer 2: `.cursor/README.md` as the local index

this file is the map of cursor-native assets. keep it scannable with tables, not essays.

include:

which rules exist and when they apply
which skills exist and when to use them
which subagents wrap parallel work
where maintenance docs live

tell the agent explicitly when not to load the heavy stuff. a line like "do not open the full catalog unless maintaining assets or the user asks for an inventory" saves tokens repeatedly.

layer 3: split always-on rules from task-scoped rules

this is the highest-leverage change i made.

always-on should mean "behavior and safety", not "every convention in the repository".

my always-on behavior rule covers:

question before assuming
prevent scope drift
keep git user-controlled
match verification to risk
documentation philosophy in one short section
architecture invariants in a few lines

detailed conventions move to glob-scoped rules:

---
description: "sql and model conventions"
globs: "**/*.sql"
alwaysApply: false
---

and:

---
description: "yaml metadata conventions"
globs: "**/*.yml"
alwaysApply: false
---

now a markdown edit does not load sql rules. a yaml edit does not load manuscript rules. the agent gets the right lane without reading every lane at once.

for content-heavy repos, the same idea applies across zones. one rule scoped to website posts, another scoped to book chapters, each with its own glob pattern. the agent loads voice and punctuation rules for the files it is actually touching.

layer 4: skills as on-demand runbooks

skills are powerful because they are procedural and task-bound. that also makes them expensive if you treat them like always-on policy.

keep each skill focused:

frontmatter description that says when to use it
required inputs
ordered steps
done criteria
links to canonical reference docs instead of copying them

the agent discovers skills through descriptions and loads the file when the task matches. it should not need the full text of every skill before you say hello.

if you want a template for tightening skill structure, see starter templates for ai rules, skills, and commands.

layer 5: reference files for inventories and contracts

some material is too long to inline but still valuable. put it in .cursor/reference/ and link to it.

good candidates:

full skill and subagent catalogs
model or schema contracts
maintaining-assets runbooks
shared prose contracts used by multiple rules

each reference file should say at the top when to load it. catalogs especially should warn against opening them on every task.

duplication is the silent token tax

the fastest audit you can run is a duplication pass.

search for the same ideas repeated across:

AGENTS.md
always-on rules
skills
README files inside .cursor/

common duplicates:

git policy stated in three places
architecture overview copied into every rule
validation commands listed in both a skill and a rule
full directory trees pasted into prompts

pick one canonical home for each topic. everywhere else, replace the copy with a link.

this also makes maintenance easier. when validation changes, you update one file instead of hunting for stale copies the agent may still believe.

indexing hygiene still matters

token efficiency is not only about rules and skills. it is also about what cursor indexes from the repo.

keep using:

.gitignore for generated output and dependencies
.cursorignore for extra exclusions such as build artifacts, minified bundles, and local secrets

less junk in the index means less irrelevant material during semantic search and fewer wrong-file detours.

i covered the basics in my cursor setup under context hygiene. the .cursor directory optimization and indexing optimization work together.

let the agent fetch, but make fetching intentional

a lean baseline does not mean a ignorant agent. it means the agent starts focused and expands deliberately.

patterns that work well:

@Files and @Folders when you already know where the answer lives
task-scoped rules that attach automatically when the agent edits matching files
skills the agent opens because the task description matches the skill's description
reference catalogs opened for maintenance, inventory questions, or ambiguous workflow choice

patterns that waste tokens:

pasting entire directories into always-on instructions
writing "read all docs before doing anything" into a rule
creating five always-on rules because you could not decide which one was canonical

the model is good at retrieval when you give it a map. it is worse when you give it the whole encyclopedia and ask it to find page one.

a practical migration path

you do not need a weekend rewrite. this sequence worked for me:

1) inventory what loads every turn

list:

rules with alwaysApply: true
large sections of AGENTS.md
any skill or subagent list inlined in an always-on file

ask for each item: "did i need this on a generic question about yesterday's pull request?"

2) rewrite `AGENTS.md` as a router

keep hard constraints and a route-by-task table. move everything else behind links.

3) collapse always-on behavior into one thin rule

merge overlapping guardrails. link out to collaboration preferences or behavioral skills instead of copying them.

4) extract task-scoped rules with globs

sql, yaml, typescript, website markdown, book markdown, ci workflows. each gets its own scoped rule if the guidance is non-trivial.

5) move inventories to reference files

catalogs, contracts, and maintenance guides live under .cursor/reference/. add a "when to load" note at the top.

6) dedupe and delete

if two files say the same thing, pick a winner and remove the loser. stale duplicates are worse than missing docs because they look authoritative.

7) test with three prompts

run these after the refactor:

a generic task outside your specialty rules (should stay fast and focused)
a file-type-specific task (should pull the right scoped rule without you mentioning it)
a maintenance task such as "add a new skill" (should route to the catalog or maintaining doc)

what improved after the refactor

the differences are subtle per turn but add up quickly:

first responses arrive with less preamble and less repeated policy quoting
sql tasks stop inheriting markdown voice rules and vice versa
token spend on routine edits dropped because the baseline shrank
when an agent does expand scope, it is usually because the task actually required more context, not because the repo forced it

this is the same philosophy behind cursor automations for housekeeping and hygiene, where the automation prompt stays short because the repo already contains the playbook. efficient .cursor layout makes both interactive agents and background automations cheaper to run.

closing

you do not need a perfect .cursor directory on day one. you need one that grows without accumulating duplicate authority.

start small, route aggressively, scope rules to the files they govern, and treat catalogs as reference material rather than default context. the agent will still go look when it needs to. that is the point. you are just stopping it from carrying the whole library into every conversation.

faq

how small should my always-on layer be?

small enough that you can read it in under a minute and still trust the agent on safety and scope. if your always-on material needs scrolling, move detail into glob-scoped rules or skills.

will lean context make the agent miss project conventions?

only if you removed the canonical source entirely. routing fixes that. the conventions still exist, they just load when relevant. if you see misses, tighten the skill description, adjust globs, or add one line in AGENTS.md pointing to the canonical doc.

should i optimize tokens or optimize for "the agent always knows everything"?

optimize for intentional loading. "always knows everything" sounds safe but usually means "always reads everything", which is slower, costlier, and often muddier. give the agent a map and let retrieval do its job.

references

chapter 3 of ai-assisted data engineering is live

Philip Hern — Mon, 13 Jul 2026 15:43:49 +0000

thesis

chapter 3 of ai-assisted data engineering is live. it is called from prototype to production, and it moves the book from mindset into the production bar, where ai-assisted work has to be measured, verified, and designed for real blast radius.

context

this is the chapter where the book turns from warning into operating standard. prototypes can survive vibes, but production work needs evidence, tests, and reliability thinking before the output reaches anyone else. it follows chapter 2, continuing the weekly release cadence for the book.

argument

what chapter 3 covers

this chapter covers:

what changed when ai moved from side-task novelty into operational systems
why production mistakes cost more than prototype mistakes
how verification becomes evidence instead of vibes
why reliability is a design responsibility, not a final polish step

why it belongs here

chapter 3 sits here because the book is building one layer at a time. the early chapters established judgment and ownership. each release after that adds another practical surface where those standards either hold or fail.

the tempting argument is that a prototype mindset is good enough if the model output looks right. chapter 3 pushes back on that because looking right is exactly the easiest standard for a model to satisfy.

closing

this is another monday release in the serialized run. chapter 4 is next in the queue, and it will publish the following monday if the schedule holds. the full book landing page stays at ai-assisted data engineering, and the companion channel stays at philliant on youtube for any longer explanation that works better out loud.

related on this site

bronze, silver, and gold standard data vault 2.0

Philip Hern — Thu, 09 Jul 2026 15:04:51 +0000

i finally got a data vault 2.0 data product to actual gold standard, and yes, i am going to let myself enjoy that one.

this was not a clean build from day one. the warehouse started as a mishmash. layers blurred together, business logic lived in the wrong places, and every new consumer request turned into a debate about where the rule belonged. getting from that state to real gold took time, but the definitions were always the guide. bronze is the raw historical foundation. silver is the business-enriched layer. gold is the governed consumer layer. once those jobs were clear, the architecture could finally do what it was supposed to do.

quick answer

in a proper data vault 2.0 stack, bronze is the raw and standardized historical foundation, landing first and then taking data vault shape in the raw vault layer. silver is the business-enriched layer, where the business vault materializes reusable business logic. gold is the consumer layer, where the information delivery layer serves governed presentation views for apps, BI, and other consumers, preferably over precalculated business vault tables.

gold matters because it is where the warehouse becomes usable without giving up lineage. the raw vault preserves history. the business vault makes business meaning durable. the information delivery layer turns that meaning into a contract consumers can trust.

who this is for

data engineers building data vault 2.0 beyond the raw vault
analytics engineers trying to understand where the business vault ends and the information delivery layer begins
technical leads deciding whether their warehouse has reached a real consumer gold layer
anyone who inherited a layered warehouse that still feels like a mishmash

why this matters

data vault 2.0 is excellent at preserving change, but preserving change is not the same as serving a product. a raw vault can be modeled correctly and still be too awkward for an application, BI team, or downstream consumer to use directly.

that is why the bronze, silver, and gold language is useful when it is tied to real layers. it keeps each layer clear about its job. the raw vault should not become a dumping ground for business rules. the business vault should not be skipped because a dashboard wants a quick answer. the consumer layer should not become a hiding place for duplicated logic that belongs upstream.

when the layers do their jobs, the system gets easier to reason about. when they blur together, every new request becomes a debate about where the logic belongs.

where i started

the warehouse i am talking about did not begin as a textbook data vault 2.0 product. it began as a mishmash.

some history was modeled well. some business logic lived in presentation views. some calculations were copied into reports. some tables were safe to query and some only made sense if you already knew the backstory. the raw vault existed, but the business vault was thin or missing in places, and the consumer layer kept absorbing work that should have lived upstream.

that is common in the "real world" where getting the work done quickly often overpowers doing the work correctly. the team knows data vault 2.0 is the long-term direction, but delivery pressure keeps pulling logic into whatever layer is fastest to ship. over time the warehouse grows, but the architecture stops explaining itself.

i did not fix that by renaming folders or declaring victory early. i fixed it by making the layer definitions real and then moving the work to the right place.

bronze, preserving the facts

bronze is where the platform starts preserving what arrived.

in the stack i am talking about, that begins with raw ingestion and then becomes standardized raw history in the raw vault layer. this is where data vault 2.0 earns its keep as a non-destructive historical model. hubs represent stable business keys. links represent relationships. satellites preserve descriptive context. effective satellites capture relationship effectivity when the model needs it.

bronze or raw silver is doing its job when:

source data lands without overwriting useful history
business keys are modeled consistently
relationships are separated from descriptive context
load metadata and record source are preserved
history can survive source changes and backfills
business presentation logic stays out of the raw vault

this layer is already a major win. it gives the rest of the platform a historical foundation that can be trusted. but it is still a foundation, not the final consumer product.

that distinction matters. the raw vault should be understandable to engineers, but it does not need to be the easiest thing for every consumer to query. its job is to preserve the truth cleanly enough that better serving layers can be built on top.

silver, making business meaning durable

silver is where the business vault earns its place.

the business vault is the business-enriched layer. it materializes the business rules, calculations, cleansing, and conformed logic that should not be repeated in every downstream view. in the architecture i am talking about, the business vault is built as dynamic tables, so the logic is not just a thin view someone hopes performs well enough. it is a materialized layer with refresh behavior, tests, documentation, and a clear purpose.

silver is doing its job when:

business rules live once instead of being pasted across reports
calculations are materialized where multiple consumers need them
source-specific rough edges are cleaned or conformed before serving
temporal logic is handled deliberately
reusable business tables sit between the raw vault and the information delivery layer
tests and documentation explain what the business layer guarantees

this is where the hard middle work lives. it is also the layer people are tempted to skip. the raw vault exists, the consumer wants a view, and it feels faster to jump straight to presentation.

that shortcut is how gold gets polluted. if the business vault does not hold the reusable business meaning, the information delivery layer has to carry it. then app views, BI views, extracts, and one-off models each grow their own copy of the same rules. that is not gold. that is a maintenance problem wearing a nice name.

gold, serving the consumer contract

gold is the information delivery layer.

that sentence is the center of the whole post. gold is not just "the prettiest table". gold is the governed presentation and semantic layer that apps, BI, and other consumers actually depend on.

in the gold standard architecture, the information delivery layer mostly serves views over precalculated business vault tables. that is the part i care about. the expensive and reusable business meaning has already been shaped in the business vault, so the information delivery layer can focus on naming, presentation, access, governance, and consumer contract.

gold is doing its job when:

app and BI consumers get stable governed views
column names and shapes match the consumer contract
business logic mostly comes from the business vault instead of being reinvented in the information delivery layer
access is applied at the consumer boundary
lineage stays readable back through the business vault and raw vault
presentation views are useful without pretending the raw vault does not exist

this is why i am comfortable calling the result gold standard. not because every model is perfect forever, but because the architecture has stopped fighting itself. each layer has a job, and the jobs are finally lined up.

how i got from mishmash to gold

the move from mishmash to gold was not one heroic refactor. it was a sequence of smaller decisions that all pointed the same direction.

first, i stopped treating the raw vault like a shortcut to consumer answers. history stayed in the raw vault. presentation logic moved out. that alone reduced a lot of confusion about which table was "the real one".

second, i stopped letting the consumer layer absorb business rules just because a view was the fastest place to ship them. when a calculation needed to be reused, it went into the business vault. when a rule only mattered to one report, i still asked whether it would matter again before leaving it downstream.

third, i materialized the business vault on purpose. reusable business meaning should not live in fragile view chains that every consumer inherits differently. dynamic tables gave the silver layer a real home with refresh behavior, tests, and documentation.

fourth, i made the information delivery layer thinner on purpose. consumer views should mostly shape and govern what the business vault already proved. naming, access, presentation, and contract belong there. duplicated business logic does not.

fifth, i wrote the definitions down and treated them as architecture, not opinion. bronze means raw standardized history. silver means materialized business meaning. gold means governed consumer views over that business meaning. once that language was stable, every new model had a clearer destination.

that is how the mishmash became gold. not by pretending the warehouse was always clean, but by making the layer jobs real and then moving the work until the system matched the definitions.

what changed when i reached it

the biggest signal was that the model stopped needing a tour guide.

before gold, even a technically correct warehouse can feel like oral tradition. someone has to explain which view has the real logic, which report copied the rule slightly differently, which table is safe to query, and which one only makes sense if you know the backstory.

after gold, the path is much cleaner. the raw vault preserves the data vault 2.0 history. the business vault materializes the business meaning. the information delivery layer serves the consumer contract. if a new consumer needs a governed view, i know where it belongs. if a calculation changes, i know where it should be fixed. if an agent helps scaffold the next piece, it has a stronger pattern to follow.

what gold is not

gold is not permission to treat the consumer layer like a junk drawer.

that is the easiest way to ruin it. consumers see that layer first, so it is tempting to keep adding logic there until a view becomes the whole warehouse. the consumer layer should serve the contract. the business vault should hold reusable business meaning. the raw vault should preserve historical structure.

gold is also not an app-shaped serving layer.

copying operational table shapes can help when consumers need source-like access without querying production systems. that is still a different lane from governed presentation built over precalculated business tables.

legacy migration views are separate too. they may be necessary. they are not the bronze-to-silver-to-gold path.

why this feels worth bragging about

data vault 2.0 requires discipline for the payoff.

separate keys from relationships. preserve descriptive history. model effectivity carefully. keep business rules out of the raw vault. resist flattening everything just because the first report wants a convenient shape.

that discipline can feel expensive while the system is being built. then the gold layer arrives and the payoff becomes visible. consumers get usable views. engineering gets a clean place to maintain business logic. lineage stays readable. the next change has somewhere obvious to go.

that is the win. the raw vault gives me the historical foundation. the business vault gives me the business-enriched silver layer. the information delivery layer gives me consumer gold. getting all three to line up is the gold standard i was chasing, and i finally have it.

references

data vault 2.0 series

chapter 2 of ai-assisted data engineering is live

Philip Hern — Wed, 08 Jul 2026 12:16:16 +0000

thesis

chapter 2 of ai-assisted data engineering is live. it is called the danger of trusting the agent, and it is the darker companion to chapter 1, why ai rests on earned judgment. if chapter 1 is about the judgment that makes acceleration safe, chapter 2 is about what happens when you let acceleration run ahead of that judgment anyway.

context

i teased this back in something longer is coming, then named the book on philliant on youtube. chapter 1 shipped first because every other chapter depends on it. you cannot safely outsource judgment you have never built.

chapter 2 was already sitting in my head in a shorter form. i wrote about the same failure mode in march in the danger of trusting the ai agent, when an agent churned through files, left a clean git tree, and still left me with low confidence about what had actually happened. the book chapter takes that incident and turns it into a full argument about ownership, diffs, and the gap between a polished summary and the work that actually shipped.

argument

what chapter 2 covers

the chapter opens on a simple claim. speed without ownership becomes expensive very quickly. when an agent operates somewhere you do not understand deeply, you can inherit changes you cannot explain, verify, or recover from with any confidence.

from there it walks through four beats:

speed without ownership, where automation bias and a clean version-control tree can hide confidence drift
changes you cannot explain, including the net-zero diff incident that started as a post on this site
reading the diff, not the narrative, the habit that keeps the model's confidence from becoming your confidence by default
owning what ships, the standard that does not soften just because a model wrote the first draft

the chapter ends with a short practice list, the kind of checklist i actually use before i let high-autonomy work run in a system i am accountable for.

why it follows chapter 1

chapter 1 and chapter 2 are meant to be read as a pair. earned judgment is what makes ai useful. unowned acceleration is what makes it dangerous. i could have jumped straight into editors, models, rules, and dbt, but that would have skipped the part that actually determines whether any of the tooling helps or hurts.

if you already read chapter 1, chapter 2 is the warning label on the same bottle. if you have not read chapter 1 yet, start there. the book is written in order on purpose.

tension or counterpoint

the obvious pushback is that this sounds like fear of automation dressed up as philosophy. i do not think that is fair. i use agents constantly. the point is not to use them less. the point is to stop treating a tidy chat summary as proof that you understand what changed.

the other pushback is that chapter 2 retreads ground from the march post. partly true. the post was the first time i wrote about the failure. the chapter is where i finally gave it enough room for the habits, the downstream data risk, and the ownership standard that should sit behind every agent run.

closing

this is the second chapter in a sixteen-chapter book, released the same way as the rest of the project, little by little, one chapter at a time. chapter 3 is drafted and still in progress. when it is ready it will show up in the same writings section without any extra ceremony.

if you read chapter 2 and want to talk it through, the companion channel is philliant on youtube. the site stays the archive. the channel is where i can slow down and explain the argument out loud.

related on this site

model reviews coming soon to my youtube channel

Philip Hern — Mon, 06 Jul 2026 03:27:46 +0000

quick answer

i am planning a recurring model review series on philliant on youtube. the working idea is to talk through "models of the week", notable releases, and the models i am actually using for production work, with practical opinions based on use instead of launch hype.

the goal is simple. i want to give people a shortcut to understanding whether a model is likely to be useful to them, because i have already spent the time using it, grading it, judging it, and comparing it against the work i actually need to ship.

who this is for

people using ai models for real work, not only demos
engineers, data people, writers, and builders trying to choose between too many model options
anyone who wants practical model judgment from someone using these systems in production workflows
future me, when the model names change again and i need a record of what actually worked

why this matters

model announcements move faster than most people can evaluate them.

every week there is a new release, a new benchmark chart, a new context window, a new reasoning mode, a new coding claim, or a new tool-use demo. those announcements are useful, but they are not the same thing as living with a model inside real work.

my question is usually more practical. can this model help me ship something i understand and can stand behind? does it handle long context without losing the point? does it make good edits, or does it produce polished noise? does it improve my production workflow enough to earn a place in rotation?

that is the kind of review i want to make.

what i mean by model reviews

i am not trying to become a benchmark channel.

benchmarks matter, but they are not the whole story. a model can look impressive in a release post and still be awkward in a real workflow. another model can look less exciting on paper and still become useful because it is fast, predictable, good with tools, or strong in one narrow lane.

these reviews will be working reviews. i want to talk about how models feel after i have used them to write, reason, code, review, debug, plan, and move production work forward.

that means the reviews will probably cover things like:

what the model is good at in my workflow
where it breaks down
what kind of tasks i would give it again
what kind of tasks i would avoid giving it
how it compares with the models already in my rotation
whether the release changed my actual behavior

the model of the week idea

the phrase i keep coming back to is "model of the week".

some weeks that might mean a brand-new release. other weeks it might mean a model i finally spent enough time with to have a useful opinion. sometimes it might be a direct comparison between two models that are competing for the same slot in my workflow.

i am still deciding the cadence. weekly might be the cleanest format if releases keep moving this fast. monthly might be better if i want more time with each model before saying anything public. release-driven episodes might be the right answer when something meaningful ships and deserves a timely review.

the format may become a mix of all three. a regular rhythm when there is enough to say, deeper monthly roundups when the week-to-week noise is too thin, and immediate reviews when a release is important enough to change the conversation.

empirical evidence over launch hype

the part i care about most is empirical evidence from my own work.

i do not mean lab-grade measurement. i mean repeated use against the kinds of tasks i actually do. production code. data engineering decisions. writing. repo maintenance. documentation. debugging. model comparison. prompt repair. agent supervision.

that kind of evidence is not universal, but it is useful. if i tell you a model helped me with a hard refactor, i want to explain what made it useful. if i tell you a model fell apart, i want to explain where the failure showed up. if i tell you a model earned a place in my rotation, i want to explain what it replaced or what new capability it added.

that is the shortcut i want to provide. not a promise that my ranking should become your ranking, but a practical signal from someone already doing the work.

where this fits with the site

i already wrote a snapshot of the ai models i actually use in cursor. that post is a written roster, and it is useful as a point-in-time reference.

the youtube version should be more alive than that. models change too quickly for one static page to carry the whole conversation. the channel gives me a place to talk through what changed, what i tried, what surprised me, and what i would actually recommend based on current use.

this also fits the broader reason i launched philliant on youtube. the site stays the archive and source of truth. the channel becomes the place where i can talk through the work in a more conversational way.

closing

so that is the plan. model reviews are coming to the philliant youtube channel.

i will keep the focus practical. less hype, more use. less "this model is best", more "this model helped me with this kind of work, failed at this other kind of work, and here is what i would use it for now".

if the format works, it should help people spend less time guessing and more time choosing the right model for the work in front of them.

faq

will this replace written model posts?

no. the written posts will still be useful for reference, links, and slower thinking. the videos should add voice, timing, and current impressions that are harder to keep fresh in a static post.

will these be rankings?

sometimes, but rankings are not the main point. i care more about task fit than a single leaderboard. the better question is not "which model is best", but "which model would i trust for this kind of work right now".

will you only cover coding models?

no. coding and agentic work will be a major part of the series because that is where i spend a lot of time, but i also expect to cover writing, reasoning, long context, multimodal input, speed, cost, and general workflow fit.

references

philliant on youtube

anything i can imagine

Philip Hern — Wed, 01 Jul 2026 16:16:53 +0000

thesis

ai is great at helping me make anything i can imagine. if i can picture the answer, the process, or the steps ahead of time, the model can usually help me move faster. but that condition matters. ai is not replacing imagination. it is amplifying the parts of my imagination that i can explain clearly enough for a machine to work with.

context

the more i use ai, the more i notice that the best results do not come from asking the model to be creative in a vacuum. the best results come when i already have a shape in my head. i might not know every sentence, every line of code, or every implementation detail yet, but i know the direction. i know what the answer should feel like. i know the steps i would take if i had enough time and energy to do them manually.

that is when ai feels almost unfair. it takes the thing i can already imagine and turns it into a draft, a plan, a diff, a diagram, a script, or a set of options. this is related to what i wrote in ai only makes sense if you have already been through the cognitive struggle yourself. the struggle builds the mental model. once that mental model exists, ai gives it leverage.

argument

imagination is the spec

when i say ai can do anything i can imagine, i do not mean it can read my mind. i mean my imagination becomes the specification.

if i can describe the end state, the model has something to aim at. if i can describe the process, it has a path to follow. if i can describe the constraints, it has boundaries. if i can describe what would make the answer wrong, it has a way to avoid obvious failure.

this is why vague prompts produce vague work. the model can fill space forever, but empty space is not the same as direction. the value appears when i give it intent. i am still the one deciding what matters, what order the work should happen in, which trade-offs are acceptable, and when the output is good enough to keep.

process matters as much as the answer

the most useful ai work is often not the final answer. it is the process that gets me there.

if i can imagine the steps, the model can help me execute those steps without making me carry every detail in working memory. it can draft the first version, split a large task into smaller moves, compare options, run through edge cases, or turn a rough plan into something concrete. it is especially powerful when the work is procedural, repetitive, or easy to verify once the shape is clear.

but if i cannot imagine the process at all, the situation changes. then i am not leading the work. i am asking the model to invent both the path and the destination, and that is where the risk starts. i may get something polished, but i do not have enough of a mental model to know whether it is right. that is the same trap behind the danger of trusting the ai agent. confidence in the output is not the same as ownership of the outcome.

creativity is still the source material

this is the part that gets lost in the louder conversation about replacement. ai is good at doing what i can imagine, which means human creativity is still necessary.

models are built from patterns learned from human-created material, human feedback, human preferences, and human goals. even when synthetic data enters the loop, the useful shape is still downstream of human ideas and human judgment. the model can remix, extend, compress, transform, and accelerate. it can surprise me with combinations i would not have reached as quickly on my own. but it still needs a starting signal.

that signal is human. the taste is human. the reason for doing the work is human. the decision that a result is beautiful, useful, durable, funny, clear, or worth publishing is human.

ai can help me produce more of what i can imagine, but it does not remove the need to imagine. if anything, it raises the value of imagination because the person with the clearest picture can get the most leverage from the tool.

ai-assisted still means i lead

we are still living in an ai-assisted world. that phrase matters because assistance has direction. the assistant is helping the person who owns the goal.

i do not want to become the assistant to the ai. i do not want my role to shrink into approving whatever the model happens to generate next. the useful arrangement is the opposite. i set the intention. i define the boundaries. i choose the sequence. i review the work. i decide what ships. the ai helps me move through the work faster, but the work is still mine.

this is also why i need room to breathe. when the model can produce endless next steps, i have to remember that output is not the same as progress. progress is work i understand, accept, and can stand behind. ai can increase the supply of possible moves, but it cannot replace the leadership required to choose the right move.

tension or counterpoint

the obvious counterpoint is that models can generate ideas i did not explicitly imagine ahead of time. that is true, and it is one of the reasons these tools are useful. sometimes the model gives me a phrase, structure, implementation path, or connection that i would not have produced by myself in that moment.

but even then, i am still judging the result against something inside me. does it fit the goal? does it sound like me? does it solve the right problem? does it move the work forward? the model can offer possibilities, but i decide which possibilities have meaning.

that distinction keeps me grounded. ai can expand the field of options, but it does not eliminate the need for a human point of view. without that point of view, the model is just generating plausible shapes.

closing

the promise of ai is not that i no longer need to think, imagine, or create. the promise is that once i can see a thing clearly enough, i can bring it into the world faster than before.

that is a powerful shift, but it does not make the human less important. it makes the human more exposed. my imagination, taste, judgment, discipline, and leadership become the real bottlenecks. ai can assist all of them, but it cannot own them for me.

so the question i keep coming back to is simple. what can i imagine clearly enough to lead? because that is where ai becomes useful. not when it replaces me, but when it helps me build the thing i already know how to see.

references

philliant on youtube is the companion to the writing

Philip Hern — Tue, 30 Jun 2026 23:20:15 +0000

thesis

philliant.com is still the main home for my writing. the channel is philliant on youtube, and it is going to be the companion space where i talk through the posts, explain the chapters, and add the kind of spoken context that does not always fit cleanly on the page.

context

most of what i publish starts here on the site. the shorter posts live in the posts section, and they cover the mix of ideas i keep coming back to, including technology, data engineering, ai workflows, productivity, and the occasional personal reflection.

i am not going to turn the channel into a second version of the archive. there are already a lot of posts, and more are coming. the point is not to list every one of them out loud. the point is to give the writing another format, one that is easier to follow when an idea needs a voice, a little more context, or a direct explanation.

the book i am announcing

the long-form project i want to announce by name is ai-assisted data engineering.

the book is about using ai agents to build real data products without giving up judgment, ownership, or control. that distinction matters to me because ai can help with speed, but it cannot replace the responsibility of the person deciding what ships.

the argument starts with chapter 1, why ai rests on earned judgment. before getting into tools, prompts, agents, dbt, snowflake, tests, documentation, or workflows, the first chapter starts with the mindset. you cannot safely outsource judgment you have never built.

that is the foundation for the rest of the book. the technology can move fast, and the models can be useful, but the human part still matters. if an agent changes code, i still need to read the diff. if it generates tests, i still need to know whether those tests protect anything important. if it explains a system, i still need enough context to challenge the explanation.

what the channel will do

the channel gives me a place to talk through this material in a more conversational way.

some videos will follow the book chapter by chapter. others will respond to posts on the site, especially when a post raises a question that deserves more space than the original page gave it. i also expect some videos to become bridges between the two, where a short post points toward a bigger theme inside the book.

the website is where the chapters and posts live. the channel is where i can slow down, say the idea out loud, and make the thought easier to follow for anyone who prefers listening or watching before reading.

where the discussion moves

right now the website does not have comments. that is intentional enough for now, but it leaves one thing missing. if something resonates, if something is unclear, or if you disagree with part of the argument, there should be a place for that conversation to happen.

for now, that place is youtube.

if you read a chapter and have a question, bring it there. if you want a breakdown of a specific post, say which one. if the first chapter of ai-assisted data engineering raises a concern, question, or counterpoint, that is exactly the kind of response i want to hear.

closing

this is the plan. the site remains the archive and the source of truth. the channel becomes the companion, the place where i talk through the writing and give readers another way into the material.

the first breakdown will start with why ai rests on earned judgment, because that is the beginning of the book and the foundation of the whole project.

related on this site

cursor automations for housekeeping and hygiene

Philip Hern — Mon, 29 Jun 2026 22:13:22 +0000

quick answer

cursor automations are a good fit for recurring housekeeping and hygiene tasks that already have a clear definition of done. i currently use them for weekly dependency and vulnerability review, documentation refreshes when a pull request opens, stale issue triage, release-note drafts, and other work where the agent can inspect context, propose a small change, and leave a reviewable trail.

the caveat is cost. automations still run cloud agents, and cloud agents still count against your cursor usage. use them where the recurring value is obvious, not where a cheap calendar reminder would do the job.

who this is for

audience: developers, data engineers, and technical leads who already use cursor agents
prerequisites: a repository with repeatable maintenance work and enough tests or checks for validation
when to use this guide: when you want automation to keep your repo cleaner without adding another dashboard or weekly manual checklist

why this matters

housekeeping is the work everybody agrees matters until the calendar fills up.

dependencies drift. docs fall behind. stale branches hang around. pull requests change behavior without updating the right README, runbook, or architecture note. vulnerability review becomes a once-in-a-while scramble instead of a normal operating rhythm.

cursor automations are interesting because they let the same agent workflow you already use in the editor run in the background. an automation can start on a schedule, a pull request event, a slack message, a webhook, or another supported trigger. it can read the repo, use selected tools, produce a summary, comment on a pull request, or open a pull request when that is appropriate.

that makes automations especially useful for work that is important, repetitive, and bounded.

the shape of a good housekeeping automation

a good automation is not just "go clean up the repo".

it has a trigger, a boundary, a checklist, and an output rule. for example, a weekly dependency hygiene automation should know which package files to inspect, which update paths are allowed, which checks must pass, and when to stop with a summary instead of opening a pull request.

i like this mental model:

trigger: when should the agent run
scope: which repo, branch, folder, or pull request should it inspect
tools: which actions should it be able to take
definition of done: what counts as a useful result
stop rule: when should it leave a summary and avoid changing anything

that last part matters. recurring agents need restraint. without a stop rule, a housekeeping automation can turn into a noisy robot that opens low-value pull requests, comments too often, or burns usage on work nobody needed.

reuse the instructions already in the repo

one of the easiest ways to make an automation better is to point it at the working knowledge already committed in the repo.

if you already have cursor skills, project rules, validation scripts, maintenance runbooks, or contributor docs, do not rewrite all of that inside the automation prompt. tell the automation to read and follow the existing source of truth first. that keeps the automation short, reduces duplicated instructions, and makes future updates easier because you can improve the skill or runbook once instead of editing every automation that copied it.

this is especially useful for housekeeping work. a dependency automation can follow the same dependency-review skill a human agent would use. a documentation automation can follow the repo's documentation-audit skill or authoring guide. a release hygiene automation can reuse the same validation commands and release checklist already used in normal development.

the trick is to treat existing repo context as the playbook and the automation prompt as the trigger wrapper. the prompt should say what starts the run, what outcome is expected, and which existing instructions to apply. the detailed standards can live in the repo where they are versioned, reviewed, and updated with the rest of the codebase.

example 1, weekly dependencies and vulnerabilities

once per week, an automation scans dependency files, checks for vulnerable packages, looks for safe version bumps, and decides whether to open a pull request. the important part is not that the agent updates everything. the important part is that it applies judgment to a small maintenance lane.

the prompt should be conservative:

### Task Description

Run the dependency audit agent using `your_review_agent_here.md`. Aggregate the results, classify identified risks, and safely apply low-risk updates with strict validation before drafting a summary PR.

### Step-by-Step Instructions

1. **Execute Audit:** Run the parallel audit workflow defined in `your_review_agent_here.md`.
2. **Aggregate & Classify:** Collect all dependency findings and categorize them by risk level (Low, Medium, High).
3. **Apply Low-Risk Updates:** Apply **only** the low-risk dependency updates to the codebase.
4. **Validate Changes:** Run the repository's test suite and build validation checks to ensure no regressions were introduced by the low-risk updates.
5. **Create Draft PR:** Open a draft Pull Request against the `main` branch containing the applied updates and the full, aggregated audit report in the description.

the automation should not invent confidence. it should run the same checks a human reviewer expects.

example 2, documentation on pull request open

another strong use case is documentation hygiene when a pull request opens.

the trigger is simple. when a pull request is opened, the automation reviews the diff and asks a small set of questions.

did this change add or remove a public behavior
did it change configuration, setup, permissions, deployment, or data contracts
did it introduce a new concept that belongs in a README or docs page
did it change something already documented elsewhere

the output does not always need to be a commit. in many cases, the best output is a pull request comment that says, "this probably needs a docs update", with the exact files or sections that look stale.

when the documentation gap is small and obvious, the automation can open a companion pull request or push to the current branch if that is how your team wants the workflow to behave. when the gap requires product judgment, it should stop and ask for a human decision.

example 3, repo hygiene summaries

not every automation needs to change files.

a weekly or monthly repo hygiene summary can look for stale branches, old draft pull requests, failing scheduled checks, large generated files, todos that accumulated in touched areas, or ignored validation failures. the value is not that the agent fixes everything. the value is that it turns hidden decay into a short reviewable note.

this is especially useful for solo maintainers and small teams. it gives you a light operating rhythm without creating a full process around maintenance.

write prompts like operating instructions

for automations, vague prompts are expensive.

when i write an automation prompt, i try to include:

what to inspect
what to ignore
what the agent may change
what validation to run
what output format to use
what requires human approval
what to do when checks fail

the prompt does not need to be long. it needs to be specific enough that a future run does the same kind of work as the current run.

use wisely because usage is real

cursor automations are not free background magic. cursor documents automations as cloud agents, and cloud agents are billed based on usage. automations also run in max mode because they run as cloud agents.

that does not mean "avoid them".

it means reserve them for jobs where the background run is worth the spend. a weekly dependency review might be worth it because it replaces a recurring manual chore and reduces risk. a pull request documentation check might be worth it because it catches drift at the right moment. an hourly "summarize my repo" automation probably is not worth it unless someone actually uses the summary.

my rule of thumb is simple. start with low frequency, high signal, and clear stop conditions. if the automation saves time or catches issues, keep it. if nobody reads the output, turn it off.

faq

should every maintenance task become an automation?

no. automate tasks that are recurring, bounded, and reviewable. if the work requires taste, prioritization, or sensitive business judgment every time, use the automation to surface context rather than make the final decision.

should automations open pull requests automatically?

only when the change is small and the validation path is clear. dependency patch updates, generated docs refreshes, and simple formatting fixes can be good candidates. broad refactors and policy decisions should usually produce a summary or comment first.

what should i build first?

start with one weekly housekeeping automation. dependency and vulnerability review is a good first candidate because the trigger is simple, the value is clear, and the output can be limited to either a small pull request or a short no-action summary.

references

what room to breathe makes room for

Philip Hern — Sat, 27 Jun 2026 19:00:48 +0000

thesis

when i wrote about room to breathe, i framed the pause mostly as defense. a way to step off the improvement treadmill before it ground me down, and a way to actually understand what i shipped before piling the next change on top of it. that is true, but it is only half the story. the space i protect does not sit empty. it is exactly where the bigger, slower ideas finally get enough air to form. the quiet heads-up i gave about something longer only became possible because i stopped filling every gap with one more optimization.

context

i sold room to breathe as rest, and rest is real, but something else happened once i actually started leaving the space. it did not stay quiet for long. the moment i was not pouring every spare cycle into squeezing the last few points out of a thing that already worked, my mind wandered somewhere it never had room to go before. it drifted toward the ideas that never fit in a single sitting, the ones i kept trimming to fit a post and kept losing the heart of. that drift is where the longer thing i hinted at recently started to take shape.

argument

the pause is not idle

the part i underrated is that breathing room is generative, not just restorative. when i stop cramming the schedule, my attention does not evaporate, it goes looking for something to chew on. left alone for a minute, it reaches past the small, urgent, well-defined tasks and starts circling the big, vague, interesting ones. the same pause that keeps me from burning out is the pause where the slow ideas surface. i do not get those ideas while sprinting. i get them in the gap after the sprint, when there is finally enough quiet for them to be heard.

small things crowd out big things

here is the trap. a small improvement always feels cheaper and more available than a big, half-formed idea, because the small one has a clear payoff and the big one does not yet. ai makes this worse, since the supply of small "you could make this a little better" suggestions is now infinite and nearly free, a trap that stays permanently stocked. if i never stop, the small things win every time, because there is always one more of them with a clearer return than the long shot. the big idea never gets a turn. room to breathe is what finally gives it a turn. the cost of every marginal optimization was never only my energy, it was the bigger thing that never got the cycles.

the bigger the idea, the longer the runway

the longer thing i mentioned needs time to think long before it needs time to make. you cannot rush the thinking. an idea that has to unfold in order, one piece building on the next, needs unhurried hours that a packed schedule simply cannot produce. that is the kind of work that only grows in protected space. i can give it that space now only because i first gave my own schedule room to breathe, the same way i try to build anything worthwhile, little by little and by sticking with it when the payoff is still far off.

tension or counterpoint

the obvious objection is that this is just a tidy story i tell myself to excuse slacking off, or worse, daydreaming about some grand project instead of doing the small work in front of me. fair. the honest version is that room to breathe only turns into something bigger if i actually use the space to think, not to scroll. it is not the absence of work, it is a quieter kind of work, the kind that produces nothing visible for a long time and then produces the thing that mattered most. the discipline is in protecting the space on purpose and then actually spending it, instead of letting the infinite small stuff flood back in.

closing

so this is the other half of room to breathe. the pause is not the end of the story, it is the start of the next one. ship it, let it settle, and in the quiet that follows, the bigger thing gets enough air to start growing. i am still not going to say what that thing is yet, for the same reasons i gave last time. i am only pointing at the mechanism. give something room to breathe, and do not be surprised when something larger walks into the space you made.

related on this site

something longer is coming

Philip Hern — Wed, 17 Jun 2026 12:26:02 +0000

thesis

i have spent a while now writing short reflections here, a post at a time, each one a single thought i wanted to get out of my head and onto the page. that format has been good to me, but some ideas do not fit in a single sitting. they need room to unfold, one piece building on the next. so i am starting to write long-form, books and guides released a chapter at a time, and this is the quiet heads-up before the first one arrives.

context

almost everything on this site so far has been short-form, a reflection, a lesson, or a technical note i wanted to remember. that suits most of what i think about, because most of what i think about fits in a page or two. but a few ideas keep circling back, and every time i try to squeeze one into a single post i end up cutting the parts that actually matter. those are the ideas that need a different shape. i have quietly been writing one of them for a while, and it has grown well past the point where a post could ever hold it.

argument

why a book and not more posts

a post is a snapshot. it captures one thought at one moment, and that is exactly why it works for most of what i write. a book is a structure. it lets an idea build in sequence, where each chapter leans on the one before it and sets up the one after. some things only make sense laid out in order, slowly, with the connective tissue left in instead of cut for length. when i kept trimming an idea to fit a post and kept losing the part that mattered most, that was the signal it belonged somewhere longer.

a chapter at a time

i am not going to drop a finished book in one go. i am going to release it the same way i do most things, little by little, one chapter at a time as each one is ready. that cadence keeps the work sustainable and keeps me honest, because a chapter that has to stand on its own cannot hide behind the ones around it. it is the same reason i keep telling myself to stick with it on any long effort, and the same reason i try to leave room to breathe between pushes instead of sprinting until i break.

what i am not saying yet

i am deliberately not going to tell you what the first one is about, at least not yet. part of that is plain superstition, the sense that naming a thing too early and too loudly is a good way to talk myself out of finishing it. part of it is that i would rather let the work introduce itself when it is ready than oversell an idea i am still shaping. so for now the only promise is the form. longer pieces, built to be read in order, arriving one chapter at a time. the subject can wait until the first chapter is able to speak for itself.

tension or counterpoint

the honest doubt is obvious. announcing something before a single chapter is finished is a good way to look foolish, and plenty of people announce books that never arrive. so why say anything at all before the work is done? because for me the announcement is the commitment. saying it out loud, in public, is exactly what makes me follow through, and the small risk of looking foolish is part of the point, since it raises the cost of quietly giving up. i would rather be on the hook for this than let it stay a someday idea forever.

closing

nothing is live yet. this is just the soft knock before anything ships, a way to say out loud where this is going so that i actually follow through. when the first chapter is ready it will show up in a new long-form section of the site, and i will point to it from here. until then, this is me telling on myself, on purpose, so the work gets done. back to it.

DEV Community: Philip Hern

chapter 4 of ai-assisted data engineering is live

thesis

context

argument

what chapter 4 covers

why it belongs here

closing

further reading

related on this site

capability is not a reason to act

thesis

context

argument

the ability to do something says nothing about its value

unlimited wishes encourage careless wishing

trends are options, not instructions

consideration must come before action

sometimes the best use of capability is restraint

tension or counterpoint

closing

further reading

related on this site

an efficient .cursor directory: less context, better agents

quick answer

who this is for

why this matters

the problem with a "kitchen sink" .cursor folder

the pattern i use now: route, do not restate

layer 1: a thin AGENTS.md

layer 2: .cursor/README.md as the local index

layer 3: split always-on rules from task-scoped rules

layer 4: skills as on-demand runbooks

layer 5: reference files for inventories and contracts

duplication is the silent token tax

indexing hygiene still matters

let the agent fetch, but make fetching intentional

a practical migration path

1) inventory what loads every turn

2) rewrite AGENTS.md as a router

3) collapse always-on behavior into one thin rule

4) extract task-scoped rules with globs

5) move inventories to reference files

6) dedupe and delete

7) test with three prompts

what improved after the refactor

closing

faq

how small should my always-on layer be?

will lean context make the agent miss project conventions?

should i optimize tokens or optimize for "the agent always knows everything"?

references

related reading

chapter 3 of ai-assisted data engineering is live

thesis

context

argument

what chapter 3 covers

why it belongs here

closing

further reading

related on this site

bronze, silver, and gold standard data vault 2.0

quick answer

who this is for

why this matters

where i started

bronze, preserving the facts

silver, making business meaning durable

gold, serving the consumer contract

how i got from mishmash to gold

what changed when i reached it

what gold is not

why this feels worth bragging about

references

related reading

chapter 2 of ai-assisted data engineering is live

thesis

context

argument

layer 1: a thin `AGENTS.md`

layer 2: `.cursor/README.md` as the local index

2) rewrite `AGENTS.md` as a router