Graph engineering picked up on X this week. The idea's been around for a few years, it's just nice to finally have a term people get excited about, one that can get more people to actually reach for it.
How did we get here?
Over the last three years, these buzzwords actually built on top of each other and got us here:
Prompt → Context → Harness → Loop → Graph
Prompt engineering was just telling the model what to do. Context engineering came next, giving it what it needed to know. Harness gave it room to actually act, tools, access, permission to touch things outside the conversation. Loop meant giving it a way to check its own output and try again if it didn't pass, so it kept working instead of stopping after one attempt. Graph is what sits on top of all of that: several of these loops, wired together, with real rules for how they hand off.
Most AI applications still happen as a single step. Ask a question, get an answer. A lot of our use cases have grown beyond that. That's what a graph is for.
In a graph, each node has one responsibility. One researches, one validates, one writes, one decides where to go next. Each node produces its own output. The edges between them define how information moves.
Formally: nodes do work, edges are transitions, and state travels through both. A node can be plain deterministic code, a single model call, a tool call, or an entire agent with its own internal loop. Edges are either fixed or conditional on what a node returned.
The building blocks
So here's five basic building blocks that can help you start building with graphs. These are from Anthropic, back in December 2024.1
Prompt chaining. One node's output feeds the next node's input, each with a narrow job. Extract the fields, then format what got extracted, then send it on. The real reason to split this into two nodes instead of one long prompt is context: the extraction step accumulates a lot of scratch reasoning getting to its answer, and the formatting step just needs the final answer. Carrying that forward can add noise to the next step.
Routing. One node classifies, and a conditional edge sends the work down one of several paths. A support ticket that's a password reset goes one way, a ticket alleging lost customer data goes another, with a stronger model and a human in the loop on that second path. Routing is one of the cheaper things you can add here, since it matches cost to difficulty instead of paying frontier-model prices for every request.
Parallelization. The graph fans out to independent nodes and joins their output. Three research agents pull from different sources at once, then a synthesis node combines what came back. This only pays off when the branches genuinely don't depend on each other. If they all need the same context to do their job, you're paying to process that same context three separate times just to save time you might not need saved.
Orchestrator-workers. One node is itself an agent that decides at runtime how much work there is and spins up workers to match. Told to update every call site of a deprecated function, the orchestrator has no way to know how many files are affected until it actually checks. You can't draw this graph in advance, since it only takes shape once the run starts. That's also why it needs a hard cap on worker count, otherwise it'll spawn as many as it thinks the job needs.
Evaluator-optimizer. Generate, check against a condition, loop back on failure, exit on success or a hard bound. Rewriting a function until the type checker passes clean is this pattern at its simplest. The easy part is the happy path, generate, check, pass. The part that actually takes design work is the failure case: what happens when the same fix keeps failing the same way, or the loop starts flipping between two answers that each break what the other one fixed.
These are some basic ideas that can help you design your own graphs. You might use some of these as is, or nest and combine them for your own use case. A router can send different requests into entirely different parallelization setups depending on what got classified. People also build what's sometimes called a diamond: fan out to parallel agents, verify each branch on its own, then collapse back into one synthesis step, so a bad source doesn't quietly ride along into the final answer. This is parallelization and evaluator-optimizer combined, and it's often used in research-focused systems. But more often than not, you'll end up customizing something for whatever you're actually building.
Every node has a trust level
Take a pipeline that turns a Slack message into a pull request. It's routing, parallelization, and an evaluator-optimizer loop stacked together, not any one pattern on its own. Four nodes, three different levels of trust in the model:
- Reading the thread and filing the ticket are API calls. Deterministic code.
- Deciding which kind of doc this is, and writing the final summary, are single model calls with no tools.
- The doc work in each codebase is a full agent run.
Ask this at every node: if this step gets it wrong, what breaks and who finds out. If the answer is a retry, let the model handle it. If it's a bad change shipped to a customer-facing doc, put a gate there.
The graph also lets you encode structure you already know, a support flow classifies before it answers, a compliance flow needs approval before touching anything external. Those can be fixed edges instead of decisions the model makes fresh each time.
The hard part is handoffs
One agent with a decent loop doesn't need much orchestration. Three agents do, and it's usually not a capability problem.
- Handoff format. Full transcript can be expensive and noisy. A summary might drop context you need later. A structured object means deciding the schema, which means deciding what actually matters. Most teams skip that and pass whole transcripts by default.
- Conflict resolution. Two agents return different answers. The tiebreak can be a validator node, a confidence ordering, fixed precedence, or a human. That choice usually works better as an explicit node in the graph than as an unstated assumption.
- Tool boundaries. If it's not clear to an engineer which tool applies in a given situation, it's often not clear to the agent either. A lot of first versions ship with too many tools and vague parameters.
- State. Decide early whether nodes share one state object or pass messages. Shared state is easier to debug and harder to parallelise. Message passing is the opposite.
Real graphs have cycles
The word graph makes people picture a clean left-to-right pipeline. Production ones loop constantly, retrying a failed tool call, going back to the user for a missing field, revising after a validator rejects, calling tools until there's enough context, pausing for approval and resuming.
A loop is a directed cyclic graph, so loop engineering is really just a small case of graph engineering.
You also can't always fix fan-out width in advance. Map-reduce is the standard case: split the input, send each piece to a worker, combine. Worker count depends on the input. LangGraph has a send primitive built for exactly this. If you're building your own orchestration you'll need something equivalent, or you'll end up with a hardcoded fan-out that breaks the moment you get more sources than you planned for.
Building one
You don't need a framework to start. A switch statement with a state object is already a graph, just an undeclared one. What you actually need is somewhere to persist state between nodes, a way to resume after a failure instead of restarting from node one, and a place to log which path a given run actually took.
LangGraph and similar orchestration frameworks earn their keep once you have real cycles, human-in-the-loop pauses, or dynamic fan-out, since hand-rolling checkpointing and resumability for those is real work you'd rather not repeat per project. For a mostly-linear graph with a couple of conditional branches, plain code with an explicit state object is often less overhead than adopting a framework for it.
A lot of the design work can happen before you write any of it, deciding the nodes, the trust level at each one, where the edges are conditional. Some of that can also get worked out alongside the build itself, but having even a rough version on paper first tends to save you some rework later.
When to skip the graph
Open-ended research is the clearest case. The agent has to plan, delegate, search, read, and synthesise in an order that depends on what it finds. LangChain and GPT Researcher both went through this exact move on their own deep research agents, dropping a predefined graph for a harness with a good loop instead.3
The test: can you name the stages before you see the input? If yes, graph it. If the stages depend on what the agent discovers, use a harness.
There's a study that actually backs this up. Google DeepMind and MIT ran 260 configurations across six benchmarks and found that once a single agent's baseline accuracy on a task already clears roughly 45%, adding more agents produces negative returns, the coordination overhead costs more than the extra agents contribute.2 Multi-agent setups gained close to 80% on tasks that decompose cleanly into parallel work, and lost up to 70% on tasks that require strict sequential planning, where splitting the work just fragments the context each agent needs. Putting that into a graph doesn't fix it, it just makes the fragmentation more visible.