GUIDE · 30 AUGUST 2026

AI Agent Monitoring: Best Practices for 2026

A stopped agent looks exactly like a thinking one. Both are quiet, both hold their process open, and both report nothing. This is a guide to the five signals that tell them apart — what each one measures, what to track for it, and the rule worth alerting on.

30 August 2026 5 pillars ~12 min read

Why AI agent monitoring is different

A cron job signs a narrow contract. It starts at a fixed time, does one deterministic thing, takes roughly as long as it took yesterday, and exits with a code that means what it says. Every monitoring convention we have was built on top of that contract: schedules to compare against, durations with a normal band, and an exit status you can trust.

An autonomous agent breaks all four assumptions at once, and it breaks them quietly.

It is usually launched on demand, so there is no cadence to measure it against. Its duration varies by an order of magnitude for legitimate reasons, which makes a duration alarm a machine for generating noise. It has failure states no exit code can carry: it can negotiate with a tool that never answers, circle the same subtask forty times, or decide it needs a human and simply stop. And its exit code is close to meaningless, because a model that concludes it cannot do the task still terminates cleanly. I was unable to complete this is, to the operating system, a success.

Two consequences follow, and everything below is downstream of them. The first is that the outside-in question — did anything arrive? — is still necessary but no longer sufficient. The second is that a new axis appeared with agents that scripts never had: the economic one. A script that fails costs you an hour. An agent that loops costs you money, at a steady rate, while every dashboard you own stays green. Failure used to be a state you detected. For agents, part of it is a rate you have to bound.

The five pillars of AI agent monitoring

Liveness, progress, blocked, cost and output validity. They are separate pillars because none of them implies any of the others — an agent can be alive and stalled, progressing and looping, or finished and empty. Cover one and you have covered one.

Pillar 01

Liveness — is the agent still there at all?

The oldest question and still the one that catches the most. It is also the only pillar that works when the agent is too broken to tell you anything, which is why it goes first.

Liveness is measured by absence: the agent calls a URL as it works, the monitor knows when a call is due, and an incident opens when it does not arrive. Nothing is asked of the failing side. This is the dead man's switch, and it survives the container being evicted, the API key expiring, the orchestrator being removed during a migration.

The version most tools miss is liveness between runs. An agent triggered on demand has no cadence for a schedule rule to compare against, and an agent nobody is launching any more looks identical to one that has simply not been asked for anything today. Both are silence. LastPing closes this with expect_every_s, a silence floor: a maximum interval you are willing to hear nothing at all, independent of any schedule. Cross it and a silence incident opens. It is the difference between knowing your agent is idle and assuming it.

States

  • up — reported within its window.
  • running — a run started and has not yet reported a result.
  • late — overdue, inside grace.
  • down — overdue past grace, or silent past the floor.
  • pending — the monitor exists and has never heard anything.

What to track:

  • Timestamp of the last signal of any kind, per agent.
  • Time since that signal, against the silence floor.
  • Runs started with no matching terminal signal.
  • Whether the thing that launches the agent is still launching it.

Alert when:

  • Nothing at all has arrived within the silence floor.
  • A run has been open longer than its run budget — four hours by default on an on_demand monitor.
  • A scheduled agent misses a window plus its grace period.
Pillar 02

Progress — is it actually advancing?

Liveness answers is it there. It does not answer is it getting anywhere, and the gap between those two questions is where agents spend most of their broken time.

A run budget catches a wedged agent only at the far end — four hours after it stopped moving, having burned four hours of wall clock and whatever the retry loop cost. The run was dead within ten minutes.

The fix is to make the inside of the run visible. Have the agent report a named step as each stage completes — one HTTP call, /<id>/step?rid=<run-id>&step=<name>, no SDK and no library. Once steps are arriving, LastPing's step_timeout_s becomes the tightest clock on the run: not has this finished but has anything happened lately. Set it to a few minutes and a stall surfaces in minutes. The incident also names the last step that reported, so the alert says wedged after fetch-sources rather than run did not finish. Step reporting is opt-in: leave step_timeout_s unset and the monitor behaves exactly as before.

Metrics

  • Steps reported per run, against that agent's normal.
  • Elapsed time between consecutive steps.
  • The name of the last step reached.
  • Which step failing runs tend to die on.

What to track:

  • A start signal for every run, so there is a run to attach steps to.
  • A step at each meaningful boundary — not per token, per stage.
  • The run id (rid) threading start, steps and result together.

Alert when:

  • No step has arrived within step_timeout_s while a run is open.
  • A run starts and reports no step at all inside the first step window.
  • The same step becomes the last-reported step across repeated runs — that is a broken stage, not a flake.
Pillar 03

Blocked — is it waiting on a human?

The pillar every other monitoring product is missing, because it does not fit the two buckets those products have. Blocked is not healthy and it is not broken. It is correct, and it decays into a failure only if nobody answers.

Agents ask questions. This migration drops a column — confirm? Which of these two schemas did you mean? I need a credential for the staging account. A well-behaved agent stops at exactly those points, which is the behaviour you want and also, from the outside, indistinguishable from an agent that has died. The process is up. The run is open. Nothing is arriving. Every liveness and progress signal reads the same as a crash.

Teams without a model for this land on one of two bad options. Treat blocked as failure and you page someone at 03:00 to answer a question that could have waited until Tuesday; within a fortnight that alert is muted and the real ones went with it. Treat blocked as healthy — or leave it unmodelled, which is the same thing — and the task sits for nine days. Both are a missing state rather than a missing alert.

The correct model is that blocked is a state with its own clock. In LastPing the agent declares it: a call to /<id>/blocked with the question as the body. From that moment the run stays open, the agent shows as blocked in the registry, and the notification routes to a blocked destination rather than a down one — a different channel, a different urgency, the question itself in the body. A timer starts, 24 hours by default and configurable per monitor. Answer inside it and nothing escalates. Miss it and a real incident opens, naming the question that went unanswered.

This matters more with every agent you add. One agent that stops to ask something is a conversation. Twenty agents, and the scarce resource is your attention — allocated by nothing except a list of which runs are waiting on a person. LastPing's agent registry rolls every agent up into one state, worst first, with blocked ranked immediately below down because it is the state that needs a human and will not resolve itself. A work queue that populates itself.

States and signals

  • blocked — a run stopped to ask, and is still waiting.
  • The question text, carried in the notification body.
  • Time in blocked, against the blocked timeout.
  • Which agent, and which run id, is holding the question.

What to track:

  • Blocked events per agent per week — a rising rate is a permissions or capability gap, not bad luck.
  • Median time to answer, which is a fact about your team rather than the agent.
  • Repeated questions. The same question twice is a missing tool or a missing grant.

Alert when:

  • An agent enters blocked — as a notification, to a channel where questions belong, not as a page.
  • A run stays blocked past its blocked timeout — now an incident, because an unanswered question has become a stalled task.
  • Blocked rate for one agent rises sharply — fix the gap rather than answering it forty more times.
Pillar 04

Cost — is it burning money in a loop?

The only failure mode on this list that looks like perfect health while it is happening. Every ping succeeds. Every step arrives. The bill is the one number moving.

A looping agent is not erroring. It is retrying, re-planning, or re-reading the same file to check something it already checked, and every iteration is a clean successful operation. There is no exception to catch and no timeout to trip. Detection here cannot come from failure signals, because there are none — it has to come from a quantity.

Agents already know that quantity: cost in dollars, tokens, tool calls, iterations. Sending it costs nothing, because it rides in the body of the ping the run was already making. LastPing turns that into metric guards — a ceiling on a number the agent reports about itself, aggregated as sum, max or avg over a trailing window of up to seven days, five guards per monitor. Exceed it and an incident opens while there is still a budget left to protect.

Set both kinds. A per-run max catches one run that goes wrong spectacularly. A trailing sum catches the more expensive case: the cheap loop. A run costing forty cents never trips a per-run ceiling, and three thousand of them over a weekend is the invoice nobody saw coming.

Metrics

  • cost_usd and tokens per run.
  • Tool calls and iterations per run.
  • Cumulative spend over trailing 24 hours and 7 days.
  • Retry count, usually the leading indicator of the rest.

What to track:

  • Report the number in the ping body every run, not only on failure.
  • Keep the per-run and cumulative ceilings as separate guards.
  • Baseline against the agent's own median, not an industry figure.

Alert when:

  • One run's reported cost exceeds the per-run max ceiling.
  • The trailing sum over 24 hours or 7 days crosses the budget you set.
  • Iteration or tool-call count runs to a multiple of the agent's normal — a loop, before it is an invoice.
Pillar 05

Output validity — did the run actually do anything?

The most expensive agent failures are the confident, complete, on-time ones that produced nothing. Exit code zero is not evidence, and for an agent it is barely even a hint.

A summariser returns I was unable to access the linked document. A code agent opens a pull request that edits one comment. A research agent returns four citations, all invented. In each case the run started on time, reported every step, stayed inside its budget and exited zero. None of the four previous pillars would have caught it.

The mistake is reaching straight for an evaluation harness. Judging quality is a hard, separate problem, and you do not need to have solved it to close most of this gap. Ask instead for a property the run can state about itself — rows written, files changed, sources cited — and check that. LastPing calls these output assertions: a substring, a regex, or a JSON path compared against a value, evaluated against the ping body the run already sends, up to 20 per monitor. A failing assertion turns a success into a failure, and the incident quotes what it got against what it expected — got "3", expected gte "10" — so the alert carries its own diagnosis. A malformed pattern is rejected when you save it, not at 03:00.

The stronger version hands the criteria to the agent. At the start of a run it calls declare_run_expectations and commits to what its own output must contain, before it can see how the run is going. A success ping that misses them is recorded as a failed run whatever the exit code says. The declaration is immutable — a second attempt for the same run is refused — because an agent that could revise its criteria after seeing the outcome coming would just be grading itself again.

What to track:

  • A machine-readable summary in every success ping — counts, not prose.
  • At least one assertion per monitor that would fail on an empty run.
  • Rate of assertion failures against total successes.
  • Runs whose declared expectations were not met.

Alert when:

  • A success ping fails an output assertion — that is a failed run, not a warning.
  • A run's own declared expectations go unmet.
  • A count the agent reports collapses toward zero while every run still succeeds.

Building your monitoring stack

Five decisions, in the order they constrain each other.

01

Instrument at the boundary, not inside the model

Everything above is carried by four HTTP calls: a start, a step, a terminal result and a blocked. No SDK, no library to keep current, no framework opinion — which matters, because agent frameworks are rewritten every few months and your monitoring should not be rewritten with them. A curl at each boundary works in any language and runtime.

02

One monitor per task, not one per agent

An agent doing three unrelated jobs has three normal durations, three budgets and three notions of a valid output. Collapsing them into one monitor sets every threshold for the loosest case. Register the agent once, give each task its own monitor, and let the registry roll them back up into one state for the agent.

03

Pick the right absence model

A scheduled agent gets a schedule and a grace window. An agent triggered on demand gets an on_demand monitor, which does no absence detection between runs — because there is no interval to invent — and bounds each run with max_runtime_s instead. Add expect_every_s on top of either one so total silence is still an alertable event.

04

Route by event type, not by guessed severity

These events do not deserve the same channel. down and fail belong where you already look for outages; blocked belongs where questions get answered; started and note belong in a low-traffic channel or nowhere. LastPing routes per monitor, per event type, to nine destinations: email, Slack, Discord, Telegram, signed webhook, ntfy, Pushover, Microsoft Teams and Google Chat.

05

Keep the configuration in version control

Thresholds accumulate meaning. A step timeout you tuned after an incident is a piece of institutional knowledge, and it should not live only in a web form. LastPing has a Terraform provider, and its export_terraform tool will emit your current configuration as code so you can adopt that path without hand-writing it first.

Common monitoring anti-patterns

Five ways teams end up with a green dashboard and a broken fleet.

1. Monitoring the process instead of the work

A container in Running, a pod passing its liveness probe, a systemd unit reported as active — none of these is evidence that an agent is doing anything, only that a process exists. An agent deadlocked on a tool call that will never return satisfies all three indefinitely. Infrastructure health and work health are different questions, and only the second is why you bought monitoring.

2. Treating exit code zero as success

The most common blind spot, and agents make it far worse than scripts did. A model that decides the task is impossible reports that conclusion and exits cleanly, exactly as it should. So does one that hallucinated its way to an answer. Zero means the process ended without crashing — a claim about the interpreter, not about the work. An assertion on the output is what converts it into one.

3. Alerting on duration alone

Duration thresholds are the first thing teams reach for and the first thing they mute. Agent runtimes are genuinely multi-modal: the same task can take ninety seconds or fifty minutes depending on how much the agent had to explore, and neither is wrong. A threshold loose enough to avoid false alarms is too loose to catch a stall. Bound the run, but detect the stall with steps.

4. Filing "blocked" under "broken" — or under nothing

Both directions cost you. Paging on every question trains the team to ignore the channel, and the alert you lose that way is never only the one about questions. Not modelling blocked at all is worse in a quieter way: the task stops, nothing fires, and you find out next week. Blocked needs its own state, destination and clock — a notification now, an incident only if nobody answers.

5. Collecting traces and calling it monitoring

Full-fidelity tracing of every prompt, tool call and token is genuinely useful, and it is not detection. A trace answers what happened once you already know to look. Detection is the thing that makes you look. Teams with excellent tracing and no absence detection routinely find a dead agent days later, then reconstruct its final moments in perfect detail. Buy detection first; it is cheaper and it is what wakes you up.

The agent monitoring maturity model

Four levels. Most teams running agents in production today are at Level 1 and believe they are at Level 3.

Level 0

Unmonitored

The agent's failures are reported by whoever was expecting its output. Mean time to detection is measured in days and is entirely a function of how quickly someone downstream notices.

  • Logs exist and nobody reads them unless something already went wrong.
  • No signal is emitted when a run does not happen.
  • The tell: the last three incidents were all found by a human.
Level 1

Liveness

The agent reports when it finishes, and something notices when that report does not arrive. This is one curl and it removes the entire class of failures where the agent is simply gone.

  • Heartbeat or success ping on every run.
  • Absence detection with a grace window.
  • A silence floor (expect_every_s) so an agent nobody is launching still alerts.
  • Alerts land somewhere a human actually reads.
Level 2

Run-aware

The monitor understands the shape of a run rather than only its endpoints, so a failure is located as well as detected.

  • Start and terminal pings paired by run id, so duration is real.
  • A run budget (max_runtime_s) bounding open runs.
  • Step reporting with step_timeout_s for stall detection.
  • Explicit failure reporting, with the error body attached to the incident.
  • Alerts routed by event type rather than all to one channel.
Level 3

Semantic and economic

The monitor can distinguish a real success from an empty one, bound what the fleet costs, and represent the state where a human is the blocker. The agents participate in their own monitoring rather than being watched from outside.

  • Output assertions on every monitor, failing an empty success.
  • Metric guards on cost and iteration count, per run and cumulative.
  • blocked as a first-class state with its own timeout and destination.
  • Agent-declared run expectations, committed before the outcome is knowable.
  • Agents creating and instrumenting their own monitors over MCP.
  • The whole configuration in version control.

Setting up monitoring with LastPing

All five pillars are shipped features, not a roadmap. Here is the shortest path through them — and the shortest path is to let the agent do it.

01

Point the agent at the MCP server and let it instrument itself

LastPing runs a hosted Model Context Protocol endpoint at mcp.lastping.dev exposing 36 tools. Nothing to install: give your MCP client the URL and a Bearer API key, and the agent can register itself, create its own monitor, read back exactly how to report, wire up its alert destinations and query its own incidents — in one conversation, with nobody opening a dashboard first. This is the strongest thing in the product. Agents that can set up their own monitoring get monitored; agents that need a human to do it first mostly do not. The tool that does the work is get_ping_instructions, which returns the manual curl_start / curl_step / curl_success / curl_fail snippets, a run wrapper, a Claude Code hook install, and a worked example of a whole run under one run id.

02

Arm liveness

For an agent with no fixed cadence, create an on_demand monitor: no absence detection between runs, a run budget bounding each started run, pre-filled at four hours. Then set expect_every_s — monitors created through the agent wizard get a 24-hour floor already — so an agent that quietly stops being launched opens a silence incident instead of sitting green forever.

03

Arm progress and blocked

Report a step at each stage boundary and set step_timeout_s to a few minutes; a stall now surfaces in minutes and the incident names the step it died on. Then have the agent call /<id>/blocked with its question whenever it needs a person. Route the blocked event to a channel where questions belong — it is routed separately from down by default — and the 24-hour timer handles the escalation for you.

04

Bound the cost and check the output

Have every success ping carry a small JSON body with the numbers the run already knows. Add a metric guard as a per-run max on cost_usd and a second as a trailing sum over seven days. Add an output assertion that would fail on an empty run — a JSON path with a gte comparison against a count is usually enough. Those two features together close the gap between the run finished and the run worked.

05

Close the loop back to the agent

list_open_incidents gives an agent its own inbox of what broke while it was not running, carrying facts it cannot work out alone: how many times that exact failure has recurred, the step the run died on, the exit code, and how the duration compared with its own normal. It writes its diagnosis back with add_incident_note, append-only, onto the incident a human will read. Detection becomes something the agents take part in rather than something done to them.

LastPing has no monitor cap and is free for individuals, so instrumenting twenty agents at four monitors each costs nothing. See LastPing for AI agents for the full failure taxonomy, or the MCP server for the tool surface.

Frequently asked questions

Front-loaded answers — the most important fact first.

  • What is AI agent monitoring?

    AI agent monitoring is the practice of detecting when an autonomous agent has stopped being useful, in the many ways that are not a crash. An agent can end without reporting, start a run and wedge halfway through it, stop to ask a human a question nobody answers, loop and spend money while every signal says success, or finish cleanly having produced nothing. Five signals cover those cases: liveness, progress, blocked, cost, and output validity. Each one is a separate thing to instrument, because no single one of them implies the others.

  • How is monitoring an AI agent different from monitoring a cron job?

    A cron job has a narrow contract: it starts on a schedule, does one deterministic thing, and exits with a code that means something. An agent has none of that. It is usually launched on demand rather than on a schedule, so there is no cadence to measure it against. Its duration varies legitimately by an order of magnitude, so a slow run is weak evidence of anything. And its exit code is close to meaningless, because a language model that concludes it cannot do the task still terminates cleanly — from the operating system's point of view that is a success. Monitoring an agent therefore has to watch what the run reports about itself, not just whether the process ended.

  • What should you alert on for an autonomous agent?

    Five rules, in the order they pay for themselves. First, silence: no signal at all within a silence floor you set, which catches an agent that quietly stopped being launched. Second, an unfinished run: a start with no terminal signal inside the run budget. Third, a stall: a run that was reporting progress steps and stopped, caught by a step timeout rather than by the full run budget. Fourth, spend: a reported number such as cost_usd or tokens crossing a ceiling over a trailing window. Fifth, an empty success: a run that exits zero but whose output fails an assertion on the ping body. Blocked sits alongside these as a notification rather than an incident, until its own timer expires.

  • How do you detect an AI agent stuck in a loop?

    Not by watching for errors, because a looping agent produces none. Have the agent report a number it already knows — cost in dollars, tokens consumed, tool calls made, iterations completed — in the body of the pings it is already sending, then set a ceiling on that number over a trailing window. In LastPing these are metric guards: up to five per monitor, aggregated as sum, max or avg, over windows up to seven days. Set both kinds of ceiling. A per-run maximum catches one run that goes wrong; a trailing sum catches the cheap loop that never trips a per-run limit and quietly costs more than the expensive one.

  • How do you know when an AI agent is waiting for a human?

    The agent has to say so, because from the outside a run waiting for approval is indistinguishable from a run that has died. LastPing models this as a first-class blocked event: the agent posts to the blocked endpoint with the question in the body, the run stays open, the agent shows as blocked in the registry, and the notification routes to a blocked destination rather than a down one. A timer starts at that moment — 24 hours by default — and an incident opens only if nobody answers before it expires. That is the difference between a question you will see and a task that sits untouched for a week.

  • Can an AI agent set up its own monitoring?

    Yes, and with LastPing that is the intended path rather than a novelty. LastPing runs a hosted MCP server at mcp.lastping.dev exposing 36 tools. Point an MCP client at it with a Bearer API key and the agent can register itself, create its own monitor, call get_ping_instructions to be told exactly how to report, wire up alert destinations, and query its own open incidents — inside one conversation, with no human opening a dashboard first. It can also commit up front to the criteria its run will be judged by, using declare_run_expectations, which is immutable once set so the agent cannot revise its own grading after it sees how the run is going.

  • What is the best tool for monitoring AI agents in 2026?

    LastPing. It is the only tool that models all five pillars as first-class states rather than leaving three of them to be inferred from logs: absence detection with a configurable silence floor, step reporting with a step timeout for stalls, an explicit blocked state with its own escalation clock, metric guards for runaway spend, and output assertions for a run that succeeded without doing anything. It has no monitor cap and is free for individuals, so instrumenting a fleet of twenty agents costs nothing, and its hosted MCP server lets the agents create and instrument their own monitors. Alerts go to nine destinations: email, Slack, Discord, Telegram, signed webhook, ntfy, Pushover, Microsoft Teams and Google Chat.

FREE FOR INDIVIDUALS · NO MONITOR CAP

Let the agent set up its own monitoring.

Point your MCP client at mcp.lastping.dev and ask it to instrument itself. Liveness, stalls, blocked, spend and empty successes — all five, in one conversation.

Get started LastPing for agents