A chatbot answers in one request. A durable agent starts work, waits for a human overnight, resumes after a deploy, retries a flaky tool, and finishes a job that spans hours. If your architecture is “stream tokens in a serverless function and hope,” you built a chatbot with makeup — not an agent platform. Founders feel this when demos work and production stalls: approval emails that resume nothing, memory that vanishes, duplicate side effects after retries. Durability is the difference.
State is the product substrate
Persist run state outside the LLM context window: current step, pending tool calls, approval IDs, checkpoints, and outputs written so far. The model proposes; the workflow engine decides what is committed. Context windows are for reasoning, not for being your database. On Next.js and similar stacks, that usually means a workflow or queue layer — not stuffing everything into a single route handler. Streaming UI can still reflect progress; the source of truth is durable storage keyed by run and tenant.
Memory: short, long, and selective
Short-term memory is the working set for the current run. Long-term memory is retrieved knowledge and prior decisions you intentionally store. Do not confuse “we embedded everything” with useful memory. Prefer structured memories — preferences, prior approvals, entity IDs — over dumping chat transcripts into a vector store and calling it done. Scope memory by tenant and by purpose. Cross-tenant leakage in memory layers is an incident, not a neat bug.
HITL delays are normal, not errors
Human approvals can take minutes or days. Your runtime must sleep without holding a hot compute session. Wake on webhook, queue, or poll; rehydrate state; continue. Timeouts should escalate or cancel with clear UX — not leave zombie runs.
Idempotency for tool side effects
Retries will happen. Every tool that creates or mutates must accept an idempotency key or natural unique constraint. Otherwise durable agents become duplicate-charge and duplicate-email machines. Design tools as if at-least-once delivery is guaranteed — because it is.
A minimal architecture path
Start with streaming chat for assistive UX. Add a run store and step machine. Add tools with idempotency. Add approval waits as first-class states. Add observability and eval hooks. Only then market “agents.” Skipping ahead produces demos that collapse under real waiting, real failures, and real multi-tenant load. Web teams already know how to build durable jobs for payments and imports. Apply that craft to agent loops. The model is new; the need for reliable state is not.
Failure modes unique to long-running agents
Partial completion: some tools succeeded, later ones failed. Design compensating actions or explicit “needs repair” states. Clock skew and duplicate webhooks: verify signatures and de-dupe events. Deploy mid-run: drain or checkpoint before killing workers; never assume in-memory state survives. Human abandonment: approvals that never come. Product policy must expire runs and notify stakeholders. Leaving runs open indefinitely creates zombie side effects when someone approves a stale proposal weeks later — so expire and require refresh for old proposals.
Testing durability
Chaos-test your agent runtime: kill workers mid-tool, restart after approval wait, replay webhooks twice, deploy during a run. If tests only cover streaming happy paths, production will invent the unhappy ones for you. Include multi-tenant load in tests. Durability bugs love to appear as cross-talk under concurrency — the worst kind of bug for trust.
What to tell customers
Be explicit about what “agent” means in your product: which jobs can run unattended, which always wait for humans, how long runs can live, and how to cancel. Customers building their own processes on top of your agents need those contracts. Vague autonomy marketing creates support burden and legal risk.
Memory hygiene over time
Long-running products accumulate stale memories: outdated preferences, obsolete entity IDs, withdrawn approvals. Build TTLs, user-visible memory settings, and admin clear tools. Memory without hygiene becomes a source of wrong actions that look mysterious because “the agent remembered something.” Prefer write-through of important decisions into your primary database with clear schema — then retrieve intentionally — over opaque vector sludge as the only memory. Vectors can help discovery; durable business state still belongs in systems you can query and correct. When you explain memory to customers, use precise language. “We store structured preferences and retrieve relevant documents with tenant isolation” beats “the agent learns about you,” which frightens security and confuses users.
Choose primitives you understand operationally: a battle-tested queue, a workflow engine, or durable execution framework — then wrap domain meaning around it. Chasing every new agent framework without durable state discipline recreates chatbot fragility with more moving parts. Expose run history in the product UI for end users and admins. People trust agents more when they can see steps, waits, and outcomes. Opacity invites fear; timelines invite accountability. That UX is part of durability, not a polish item for later.
In short: chatbots optimize for a single conversational response. Durable agents optimize for correct completion under delay, failure, and human time. If your stack cannot wait, resume, and idempotently finish, keep calling it chat — and invest in the state layer before the next logo asks for agents that outlive a browser tab.
