Procedural Intelligence: When Systems Start Reasoning About Themselves
What changes when your system stops just making decisions and starts deciding how its decisions should evolve.
Last week I spent 5 days in a surge. It’s essentially a hackathon focused on scaling agentic procedures. What I came out with wasn’t more automation. It was a clearer view of a second reasoning layer that I hadn’t been writing about yet. That’s what I’m sharing today.
In a few weeks, I’ll turn toward how I’d fix the system issues I’ve called out — and start folding the new meta layer into the frameworks themselves.
If you want to follow how the next frontier of automated systems is actually evolving, that’s the upcoming arc!
I‘ve spent the last year writing about reasoning systems — how they should be structured, how they should recover from failure, how they should be observable enough to trust. Procedural Intelligence and the Discovery Layer were my way of mapping that architecture.
Lately I’ve been watching that architecture sprout a new floor.
One reasoning system, then two
Most agentic systems in production today are built around a single reasoning system: the runtime layer. Signals come in, it figures out what’s happening, it decides what to do. That’s the system most people picture when they say “AI automation.” It’s where almost all of the public conversation lives.
What’s starting to emerge above that is a different kind of system. Not one that makes decisions. One that reasons about the system that makes decisions. Where it’s incomplete. Where it’s wrong. Where the world has shifted underneath it. Where new logic needs to exist.
Call it the meta layer — the new one.
The runtime layer asks: what should I do right now?
The meta layer asks: what should exist that doesn’t yet?
These are not the same problem. They don’t even share most of their primitives. The runtime layer is about precision under constraint. The meta layer is about gap detection, evaluation, generation, and repair; running continuously, on top of a system that is itself running continuously.
Most of the field is still arguing about the runtime layer. The meta layer barely has a vocabulary yet.
What this did to my own frameworks
For the last year I've been describing Procedural Intelligence and the Discovery Layer as the architecture. What I’m starting to see is that I was describing one floor of it.
The runtime layer: the reasoning system that decides what to do.
The meta layer (the new one): the system that decides what should exist.
The frameworks I had aren’t wrong. They’re load-bearing for the runtime layer. But the meta layer needs frameworks of its own, and that’s the work for the next stretch of my year.
Where the labor question enters
What I watched compress during those 5 days wasn’t headcount. It was the manual loop — the part where humans expand a system’s logic by hand. A gap gets noticed, a human writes the fix, it ships. That cycle is being shortened. Dramatically.
Dario Amodei, CEO of Anthropic, has been arguing that AI will eliminate roughly half of entry-level white-collar jobs within 5 years. I think the shape is slightly off. Jobs don’t get replaced as a single event. What gets replaced is that loop. Not “humans are gone” but humans are no longer the mechanism by which the automation surface grows.
That changes what humans do next more than it changes whether they’re there.
1, 3, 5 years out (realistically)
From my POV, this is what automation will look like in big tech (at the very least):
1 year out: most of this still has humans heavily in the loop. The meta-system surfaces gaps and proposes fixes; humans review, approve, and ship. The win is speed and coverage with fewer manual cycles to expand the surface.
3 years out: the review step narrows. The meta-system isn’t just proposing, it’s running its own evaluation, scoring proposals, shadow-testing them against real traffic, and only escalating genuinely ambiguous cases. The team’s job becomes designing what “good” looks like, not approving each instance of it.
5 years out: if this works, the meta-system is where most of the procedural logic lives. The runtime system runs. The meta-system evolves the runtime system. Humans sit one floor above that designing the evaluation criteria, the safety bounds, the values the meta-system optimizes against. Three layers, not one. The role isn’t “operator.” It’s system designer for systems that design systems.
What I’m taking from this
A year ago I was writing about how AI systems should reason. Lately I’ve been watching this pattern start to reason about itself.
That’s not a contradiction of the work. It’s the next floor of it and it’s why my frameworks feel both more correct and more incomplete than they did a few months ago — they hold under the new layer, but the new layer needs its own scaffolding.
The interesting work for the next several years isn’t the runtime layer. It’s the meta layer. Most of the field hasn’t focused on it yet.
👋 I’m Sarah. I write about building AI systems that can reason, recover, and earn trust. These frameworks aren’t final — they’re in motion, like the systems we build. If you’re seeing similar gaps in your own work, I’d love to hear what’s surfacing for you.
→ sarahpayne.ai for frameworks, visuals, and what’s coming next.
I have a similar take on the 1,3 and 5 years. When systems decide on the decision frame itself (your meta layer), rather than deciding towards task execution - a big pivot from current human in the loop will occur. The question is what will the new shape look like.
If I understand you right, the real question is who gets to set the evaluation criteria at the top. That seems to be where the real power is now. The meta layer observes the runtime layer, and people move up to decide what the meta layer should optimize for, so the power keeps moving higher. Honestly, I find this both exciting and a bit scary. The next five years are going to be wild. Great article, Sarah.