Generating Tests and Documentation

Introduction

#

Tests and documentation are the most neglected parts of data engineering because they are tedious to write by hand. An agent that reads a model, infers the business logic, and generates the matching definitions gives you the rigor without the manual overhead, as long as you still review the result.

This is the chapter where the whole book starts paying off in concrete data work. The principles from the earlier parts converge here. Earned judgment lets you tell a good test from a useless one, the setup gives the agent enough context to draft the right files, and the guardrails keep it safe near your warehouse. Now I point all of it at the work everyone agrees they should do and almost nobody does. The examples use the common transformation and warehouse stack I work in, but the lesson holds wherever you write tests and docs.

The Most Neglected Work

#

I will confess, because nearly everyone does this. You build the model, it works, you promise yourself you will add the tests and the documentation later, and you move on to the next urgent request. Later rarely comes. The model ships naked, the next one does too, and the project slowly accumulates a backlog of undocumented, untested logic that everyone is a little afraid to touch.

The reason is not that engineers do not value testing and documentation. It is that both are tedious to produce by hand, and tedium loses to urgency every single time. Writing out every column description and every assertion is exactly the kind of careful, repetitive work that feels like a tax in the moment, even though the absence of it is what makes data untrustworthy and debugging miserable later.

The cost of skipping is real and it is delayed, which is the worst combination. A broken metric can sit in production for days before anyone notices, and by then it has already shaped a decision. Undocumented models force every downstream consumer to rely on tribal knowledge and guesswork, which does not scale past the few people who happened to be in the room when the model was built. Tests and documentation are how you protect trust in the data, and that trust is the entire product.

Reading the Model to Infer Intent

#

This is precisely the kind of work an agent is built to accelerate, because the source material is right there in the SQL. The agent can read a model, follow the joins and transformations, understand the grain, and infer what the columns mean and what assumptions must hold. From that reading it proposes a complete set of descriptions and tests, including the obvious structural ones and often some genuinely useful custom checks based on the structure of the data.

The structural tests are the easy, high-value baseline, and the agent rarely gets their syntax wrong. A small block covers most of the real risk in a heavily consumed model.

models:
  - name: fct_orders
    columns:
      - name: order_id
        data_tests:
          - not_null
          - unique
      - name: customer_id
        data_tests:
          - not_null
          - relationships:
              arguments:
                to: ref('dim_customers')
                field: customer_id
      - name: order_status
        data_tests:
          - accepted_values:
              arguments:
                values: ["placed", "shipped", "cancelled", "returned"]

Each line guards something specific. Not-null protects required fields, uniqueness protects grain, relationships protect referential integrity across joins, and accepted-values protects the finite set of business states a column is allowed to hold. Beyond these, the agent can scaffold a singular test for a rule the structural tests cannot express, such as an amount that must never be negative, by writing a query that returns only the offending rows. The agent’s strength is breadth and speed across this whole surface, generating in seconds what would take an hour of careful typing.

Documentation That Stays in Sync

#

The deeper win is that generating from the model keeps the documentation accurate. The chronic disease of data documentation is drift, where a description written once slowly diverges from the model it describes until it is actively misleading. A wiki page about your data goes stale the moment someone adds a column. Documentation generated from the model and regenerated as the model changes does not drift, because its source of truth is the code itself.

When the agent writes descriptions, it grounds them in the actual SQL, so the first version is already aligned with reality rather than with someone’s memory of reality from six months ago. The descriptions then feed the generated documentation site and, if you choose, the warehouse catalog itself, so the meaning of a column is available to a downstream analyst querying the database directly, not just to whoever opens the repository. The same metadata also powers the lineage graph that answers where a column comes from and what breaks if a source changes, which replaces the hand-drawn architecture diagram that always goes stale.

The principle generalizes beyond any one tool. Documentation that is generated from the source of truth and refreshed alongside it stays correct for free, while documentation maintained by hand in a separate place decays the instant the code moves. The agent makes the generated approach cheap enough that there is no longer a good excuse for the hand-maintained one. You write the model, the agent drafts the description from it, and the two stay tied together through every future change.

Reviewing What the Agent Wrote

#

None of this removes the review step, and the review step is where your judgment earns its keep. The agent is excellent at scaffolding and inferring patterns, but it does not hold the full business context that you do. It can see that a column is called status and contains four values. It cannot always know that one of those values is deprecated, that another should never co-occur with a particular state, or that the real grain is subtler than the keys suggest. That knowledge is yours, and the review is where you supply it.

Because the agent rarely makes syntax errors, the review is mostly about meaning rather than mechanics. You are confirming that the inferred business logic matches the real business logic, that the descriptions say something useful rather than restating the column name, and that the tests assert the things that actually matter instead of a pile of checks that pass without protecting anything. A test that cannot fail in a way you care about is just noise, and pruning that noise is part of the job.

When time is limited, prioritize the review the same way you prioritize the tests. Start where breakage is most expensive, not where the modeling is most elegant. Put key integrity, meaning uniqueness and not-null, on your most heavily consumed models first. Then add referential checks on the joins that power dashboards, then the controlled-value checks where business logic depends on a finite set, then one custom test for the highest-risk rule. The agent can generate all of it in a single pass. You decide which of it deserves to exist, and that decision is the part nobody can automate.

Putting It Into Practice

#

1. Stop promising to add tests and docs later, and generate them in the same pass that builds the model.
2. Have the agent read the model and propose descriptions and tests grounded in the actual SQL.
3. Cover the structural baseline first, namely uniqueness, not-null, referential, and accepted-value checks on what matters.
4. Add a singular test for any business rule the structural tests cannot express cleanly.
5. Generate documentation from the model so it stays in sync, instead of maintaining it by hand somewhere it will rot.
6. Review for meaning, not syntax, and supply the business context the agent cannot infer.
7. Prioritize by where breakage is most expensive, and prune any test that cannot fail in a way you care about.