thesis#
chapter 2 of ai-assisted data engineering is live. it is called the danger of trusting the agent, and it is the darker companion to chapter 1, why ai rests on earned judgment. if chapter 1 is about the judgment that makes acceleration safe, chapter 2 is about what happens when you let acceleration run ahead of that judgment anyway.
context#
i teased this back in something longer is coming, then named the book on philliant on youtube. chapter 1 shipped first because every other chapter depends on it. you cannot safely outsource judgment you have never built.
chapter 2 was already sitting in my head in a shorter form. i wrote about the same failure mode in march in the danger of trusting the ai agent, when an agent churned through files, left a clean git tree, and still left me with low confidence about what had actually happened. the book chapter takes that incident and turns it into a full argument about ownership, diffs, and the gap between a polished summary and the work that actually shipped.
argument#
what chapter 2 covers#
the chapter opens on a simple claim. speed without ownership becomes expensive very quickly. when an agent operates somewhere you do not understand deeply, you can inherit changes you cannot explain, verify, or recover from with any confidence.
from there it walks through four beats:
- speed without ownership, where automation bias and a clean version-control tree can hide confidence drift
- changes you cannot explain, including the net-zero diff incident that started as a post on this site
- reading the diff, not the narrative, the habit that keeps the model’s confidence from becoming your confidence by default
- owning what ships, the standard that does not soften just because a model wrote the first draft
the chapter ends with a short practice list, the kind of checklist i actually use before i let high-autonomy work run in a system i am accountable for.
why it follows chapter 1#
chapter 1 and chapter 2 are meant to be read as a pair. earned judgment is what makes ai useful. unowned acceleration is what makes it dangerous. i could have jumped straight into editors, models, rules, and dbt, but that would have skipped the part that actually determines whether any of the tooling helps or hurts.
if you already read chapter 1, chapter 2 is the warning label on the same bottle. if you have not read chapter 1 yet, start there. the book is written in order on purpose.
tension or counterpoint#
the obvious pushback is that this sounds like fear of automation dressed up as philosophy. i do not think that is fair. i use agents constantly. the point is not to use them less. the point is to stop treating a tidy chat summary as proof that you understand what changed.
the other pushback is that chapter 2 retreads ground from the march post. partly true. the post was the first time i wrote about the failure. the chapter is where i finally gave it enough room for the habits, the downstream data risk, and the ownership standard that should sit behind every agent run.
closing#
this is the second chapter in a sixteen-chapter book, released the same way as the rest of the project, little by little, one chapter at a time. chapter 3 is drafted and still in progress. when it is ready it will show up in the same writings section without any extra ceremony.
if you read chapter 2 and want to talk it through, the companion channel is philliant on youtube. the site stays the archive. the channel is where i can slow down and explain the argument out loud.
further reading#
- automation bias, the tendency to trust automated output before you have verified it yourself
