Artificial Intelligence
Tobby Hagler, VP, Engineering Evolution
November 10, 2026
Give an AI agent the same ticket on Monday and again on Tuesday, and you'll get two different answers. They could both work, or one might work while the other might confidently pull in a Drupal module that doesn't exist. For engineers who were drawn to this work partly because the same code always did the same thing, that's a real loss.
A lot of what gets called "AI slop" comes from this variance. Code can compile and look reasonable and still be built on something the agent made up.
That shift isn't slowing down. Tools, models, and ways of working change every few months, and there's no waiting for things to settle. Some engineers and agencies thrive in that, some struggle, and some doubt it's worth the trouble. The ones who thrive work out how the current moves and steer with it, by experimenting and sharing what they learn.
They also focus on the two things they can control: what the agent knows going in, and how closely they check what comes out.
What comes next won’t make the output predictable, but it’s what we’ve found helps us manage it.
You can't make the model give the same answer twice. What you can control is what it knows going in.
In our experience at Phase2, most bad output traces back to bad or missing context rather than a bad model or improper instruction. When there's a lack of details, the agent fills the gap with something that sounds right.
So we treat context as an engineering input, the same way we'd treat config or test data. Somebody owns it, it gets reviewed, and it gets fixed when it's wrong or out of date.
A human still has to judge whether the output is any good. Better context means fewer bad guesses reach that review.
Good context also means better consistency. An agent might perform the same operation multiple times, but with well-curated context, you can steer output variation in the same direction.
LLMs are bad at timelines. Give an agent a doc that mixes the old plan with the new one, with nothing dated or marked superseded, and it'll happily build against either one with little or no distinction.
So our docs capture only the current state. The current fact goes up front, and a doc shouldn't hint at what used to be true. A frequent directive we'll use is, "edit, not amend."
If history matters, it goes in a changelog or errata section in the margins. This helps people as much as agents, since a new engineer joining a project has the same problem the agent does.
On a recent platform build, the architecture doc was a static file that became stale as the project pivoted and evolved over time. The knowledge base in the repo kept up, because it was updated by agents as the work happened. When developers joined mid-project, they were productive right away, working from the knowledge base instead of trying to square an old doc with what was actually built.
Context hygiene is deciding what information carries forward from one stage of an engagement to the next, so agents don't treat aspirations as commitments.
Context piles up over an engagement. Sales captures the win themes and what the client hopes is possible, and discovery adds assumptions. Delivery inherits all of it.
Without context hygiene, an agent can't tell the difference between "Yeah, we could build that," and everyone agreeing, "Yes, this is the thing we have agreed to build."
Sales context explains why the deal looks the way it does, but it shouldn't become ground truth for how the work gets built. Historical context works the same way. It's good for explaining why a past decision was made, and harmful when it quietly steers current work in the wrong direction.
Organizations that thrive in variance decide what has to cross the handoff from sales to delivery, so engineers aren't left sorting promises from commitments on their own. For clients, it means the development team and their agents are all on the same page with what's been agreed to.
The most important part of the job hasn't changed: knowing what good looks like. Documentation should record what was decided and why. But a new architect can inherit complete project docs and still struggle to guide a novel solution, because judgment doesn't come with the docs.
What's shifting is where that judgment gets used. Say an agent runs a security review on a pull request that touches login. An engineer still has to know what that review needed to cover, and check that the agent actually covered it. Until an agent has a track record on that kind of review, someone reads what it checked line by line.
We call the amount of checking a piece of work needs its assurance level. Login code gets a lot, and a copy tweak on a landing page barely gets any. Organizations set what the levels are, so each engineer isn't deciding from scratch how much scrutiny login code needs.
A year ago, every one of our project teams was experimenting with its own way of feeding agents the right context. Context windows were small and tokens were expensive, so that was the right work for the time. It also meant every project ended up with its own system, each doing the exact same thing, just a little differently. Nobody did anything wrong there, but agentic work quickly amplifies the friction caused by inconsistent tooling across teams.
Models have advanced since then, and they’ll keep advancing at a rapid pace. A lot of what those experiments needed to work around are automatically managed by today's platforms. In many cases, what was once an important guardrail is now a heavy restraint.
So organizations need to standardize on a single way to manage a living knowledge base that agents and people can both consult, one that updates as the project, code and requirements change. They should also expect to heavily modify or throw that tool away as newer models solve the problems it was built for.
Experimentation doesn't stop, but it's more deliberate now. An experiment starts with a specific problem the standard doesn't handle well, and when it works, the results go back into the standard instead of living beside it.
The engineers who built those early tools have exactly the instinct this needs. They didn't wait for someone to hand them one. Now that energy goes into the shared tool, working on experiments with engineers on other projects, so the next person (or agent) starts from what the whole team already knows.
The engineers and agencies that thrive here are flexible. They walk into a messy situation, sort out what matters, and adjust as it changes, instead of waiting for a set of steps.
What we're building to support that is closer to a launch checklist than a playbook. Did you lower the DNS TTL a month out, or is this simply a CDN switch?
It's very much a "kitchen sink" of what goes into an engagement: ticket policy, QA approach, security posture, accessibility coverage, and what ready/right/done means on that project. Teams pick what's relevant to the project in front of them. We document the most common variations, and the odd bits and pieces are dealt with as we go.
Go back to that agent that pulled in a Drupal module that doesn't exist. The quick fix is to swap in the real module and move on.
An engineer who thrives here asks why the agent thought it existed. Maybe the project's knowledge base never says which contrib modules the site actually uses, so the agent guessed. They add that, so the next agent run and the next engineer on the project start from it, and they take a closer look at the agent's next few dependency changes before anyone trusts them.
Swapping in the right module fixes the bug in front of you, but it doesn't change what the agent does with the same prompt tomorrow. Fixing the root cause means the next time the agent reaches for a module, it works from the list the site actually uses.
If you miss the predictability of the old work, the rigor you brought to code still applies. It just goes into the context the agent works from and how closely you review what comes back.