October 29, 2026

From On-Call to Always-On

Topic: Mobile, Video and Web

Sumeet Arora

TYPE: Articles

YEAR: 2026

Building a continuous product maintenance loop with AI agents

A product ships. The maintenance clock starts.

Each product’s life cycle spans design, development, testing, launch, and ongoing operation. Once it is live, teams monitor its health, respond to user feedback, fix failures, and improve performance, reliability, and usability. Each improvement goes through implementation, review, testing, rollout, and verification in production. This cycle repeats throughout the product’s life, requiring engineers to carry context and ownership from one stage to the next.

Dependencies change, traffic shifts, and new devices expose old assumptions. Some failures trigger incidents. Others appear as slower responses, accessibility gaps, or unreliable interactions that can persist quietly. Teams have to respond to these changes while continuing to build the product’s next set of capabilities.

The difficulty grows with the number of products, services, and teams involved. Engineers must reconstruct context, find the right owner, connect symptoms to changes, and carry work through verification. Those coordination requirements recur throughout the product’s life.

Coding agents can help with individual steps. Maintenance gives us a way to connect those steps around an outcome: keeping a product healthy as it evolves.

This post explains how we’re building Autopilot at Meta to connect reactive and proactive maintenance into a continuous loop. We’ll walk through how reusable agents, product context, persistent task state, evaluation, and permissions work together to keep maintenance moving while engineers retain oversight.

Maintenance has the feedback agents need

A symptom, an owning team, and a verification path give an agent a concrete objective. Code, telemetry, tests, documentation, and change history provide evidence it can use to investigate. A reproduction or a metric can help establish whether the intervention worked.

The most suitable tasks have five properties:

Bounded: A symptom on a known product surface defines the initial scope.
Evidence-rich: Code, telemetry, tests, ownership, documentation, and change history support the investigation.
Testable: A reproduction, metric, invariant, or automated test can establish whether the intervention worked.
Reversible: A proposed change can be reviewed, staged, monitored, or rolled back.
Owned: One team remains accountable for the product and its risk boundary.

These properties also define where automation should stop. Missing evidence may call for further investigation. Unclear ownership may require routing to a person. An action without a reliable verification path should not be treated as a completed repair.

With those boundaries in place, the next question is, what will trigger truly useful maintenance work?

Work enters through two doors

Some work arrives because a problem has already been reported. Other work begins when the system observes an opportunity to improve the product. Both require evidence, ownership, and a useful action, but they start at different points.

Reactive maintenance begins with the queue

Reactive work starts with a bug report, regression, alert, or incident. We began with the on-call queue because it brings together concrete demand and identifiable outcomes. The immediate objective is to understand the signal and restore product health.

A queue item is the start of an investigation. The system must validate that the report is current, reconstruct the relevant context, identify ownership, reproduce the problem where possible, and establish how recovery will be verified. A task can involve several agents or steps before it reaches a useful outcome.

Proactive maintenance starts before the queue

A finder agent observes a bounded product surface or quality signal and identifies a possible issue or optimization opportunity. It might surface a performance regression, an accessibility gap, or a reliability risk before an urgent report reaches the team.

The finder produces a candidate task with evidence, estimated impact, confidence, reproduction guidance, and likely ownership. Intake checks whether the observation is current, whether another task already covers it, and whether there is a useful action to take. This validation keeps every anomaly from turning into a proposed code change.

Both paths converge on the same loop

After intake, reactive and proactive work need the same capabilities: classify the task, investigate it, make an appropriate change, verify the outcome, and learn from the result (Figure 1).

*Figure 1. The same single, post-intake maintenance loop for both reactive and proactive work*

Consider a hypothetical notification-delivery delay. A finder could observe sustained latency and record the affected experience and supporting evidence. Or, the same task could arrive as an on-call bug report. Either way, the investigation must establish whether the delay is a regression, an expected behavior, or an observation that needs more evidence.

A shared loop lets improvements in routing, verification, and recovery benefit both sources of work. Proactive discovery can expand coverage while using the same execution and control infrastructure as the existing queue.

Introducing Autopilot

At Meta, we are building Autopilot to own this continuous maintenance loop. Reactive signals and finder discoveries enter the same system. Autopilot maintains task state, assembles context, routes work to the right expertise, and follows it through investigation, verification, review, or escalation.

This continuity matters because maintenance work rarely finishes in a single agent execution. An investigation can lead to a change, a test can expose a new problem, a review can require another iteration. The system keeps the goal and evidence connected across those transitions. Workflows track the steps and artifacts required to move the task forward.

A useful outcome can be a verified, reviewable change—an escalation with evidence and a recommendation—or a recorded, no-action decision. For the notification example, establishing that the delay is intentional may be the correct result. The system needs to preserve that reasoning as clearly as it would preserve a proposed fix.

Building this system across many products introduces a challenge immediately: An agent can know how to perform a class of work while still lacking the local knowledge needed to do it correctly for a particular product. Sharing expertise and preserving that local truth have to work together.

Shared capability needs local product truth

An agent provider may know how to investigate a class of failures well. The product team knows the architecture, ownership, invariants, and verification rules for its own system. Scaling maintenance automation requires both kinds of expertise.

Copying product knowledge into every specialist would create many versions to maintain. Asking a central agent team to keep every product’s context current would create a bottleneck. Autopilot separates those responsibilities, then combines them for a particular task.

That design includes four parts: encode local truth in a product pack, package reusable expertise as an agent definition, publish agents for other teams to install, and use task context to select an agent before binding the relevant product knowledge.

Keep local truth in a product pack

A product pack contains the context an on-call or product team owns. It lives in reviewed, versioned repository files, with each file serving a specific purpose (Figure 2).

Autopilot loads the relevant source when a decision requires it: triage rules for classification, product knowledge before changing ownership, code pointers before modifying code, and navigation guidance for reproduction. Test identity and setup information make that reproduction usable; routing metadata connects the work to the team’s task-management process.

Loading context at the point of use keeps the agent focused and makes its product assumptions easier to inspect. In the notification example, the pack could explain that delivery crosses a client surface, an API, and an asynchronous pipeline. It can identify the user-visible metric, known exceptions, likely code locations, owners, and correct test identity.

The team must keep this knowledge current. A stale ownership rule can send a sound investigation to the wrong team. A stale verification recipe can make an ineffective change look successful. Generated product packs provide a starting point; reviewing and maintaining them is part of operating the system.

Package expertise as an agent definition

The agent provider captures reusable expertise in an agent definition. It includes instructions, plugins, skills, tools, supported domains and task types, input and output contracts, failure and escalation behavior, ownership, and version information.

A contract makes the capability usable by the surrounding system. It describes what the agent needs, what it is expected to produce, and what happens when it cannot finish. The provider is responsible for that capability and its improvement loop. Product teams retain responsibility for their local context and operating policy.

The ecosystem can include starter agents for common domains, custom specialists built by product teams, and published agents that others reuse. A performance specialist can improve its investigation method without carrying a separate copy of every adopter’s architecture.

Publish agents for other teams to install

Agent providers publish their agents to the agent marketplace. Other product and on-call teams discover those agents and install a version for their own use (Figure 3). The same expertise can serve multiple teams.

Versioned publication and pinned installation give providers and adopters separate update decisions. A provider can improve an agent, while each team chooses when to take the new version. Installation makes the agent available to a product. The next decision is which installed agent fits an individual task.

Select from task context, then bind local truth

Routing starts with the task context: the goal, supporting evidence, domain, and constraints. Autopilot uses that context to select the right agent from the team’s installed agents, considering the required tools, risk, and product policy.

It then attaches the relevant product pack context. The provider supplies the reusable method; the product team supplies architecture, ownership, code pointers, reproduction guidance, and verification rules. Binding that local truth at dispatch produces a grounded run: an agent selected for the task, equipped with the product knowledge needed to carry it out (Figure 4).

For the notification task, latency evidence can point toward an installed performance specialist. The product pack then explains the delivery path and how this product measures success. Agent selection answers which expertise fits the work. Context binding supplies the local facts that expertise needs.

The shared agent definition remains reusable across products. Product-specific context arrives with the task, and its owners can update it independently. This division of responsibility is useful to scaling adoption without making the platform team responsible for every product’s details.

Suppose two performance agents are available for the same notification task. Both offer the relevant expertise, but a product team still needs evidence about which one is more likely to solve its problem. Publishing and installing agents makes that comparison necessary.

Agent quality must be measured

Teams need evidence about how often an agent succeeds, where it fails, and what it requires to run. Comparing agents requires a consistent method, and improving them requires enough detail to understand their mistakes. A marketplace makes reuse possible; evaluation helps teams decide what to reuse.

Use a controlled comparison

Agent Arena runs two marketplace-agent variants against the same fixed suite. The cases come from actual closed Autopilot bug-report tasks and the code changes that landed against them.

Both variants receive the same cases through the same execution and grading harness. Their models, instructions, plugins, skills, and overrides can differ. The scorecard includes solve rate, grader results, resources required, reliability, and per-case detail. Saved runs let teams compare a candidate with an earlier baseline.

Arena uses shared evaluation infrastructure for execution and grading, while Autopilot provides the interface for selecting agents, launching comparisons, and inspecting the history. Together these keep the evaluation method consistent while making it accessible from the same system where teams discover and use agents.

Per-case evidence is useful. A poor result on a latency task could come from missing product context, an incorrect diagnosis, a faulty change, or weak verification. Each calls for a different intervention. An aggregate improvement can also hide a regression on the tasks an adopting team cares about most.

Turn failures into a repeatable improvement loop

Providers can use those results for controlled iteration:

Establish a baseline on the fixed suite.
Inspect individual failures and choose one change, such as the prompt, model, plugin, or skill.
Rerun the same suite and compare both gains and regressions.
Publish deliberately, then monitor the adopted version in production.

The provider makes the improvement and publication decisions. Arena supplies evidence; it does not automatically publish agents, change routing, or grant them more authority.

This type of evaluation also has limits. A suite built from previously solved tasks cannot represent the entire incoming queue. Current results are point estimates without significance testing, so a small difference may not justify a strong conclusion. Production observations must test whether offline gains hold for real work, including tasks that never reached a successful fix.

Those results guide which agent a team installs. Dispatch introduces a further requirement: The chosen agent must be able to perform its work under an appropriate identity.

The right agent needs the right identity

Even the right agent with the right context can be blocked by access. A specialist may need systems the default service identity cannot reach. Giving every agent the union of those permissions would expand the authority of the entire system.

Autopilot supports three execution-identity patterns: the default Autopilot Bot service identity, a specialized Crewmate identity, or a custom service user for a narrowly scoped capability.

Specialized identities are configured per agent and validated before dispatch. The identity must be allowed for the product, and the execution path must have the required permission to act as it. Access becomes an explicit dependency of the capability, with a boundary the platform can check.

Keep consequential actions behind explicit checks

Autopilot workflows can pause at human checkpoints. Landing a code change has its own boundary: Auto-land is opt-in and requires two distinct human acceptances on the current revision, no later-blocking inline comment, and passing continuous integration checks.

These controls are enforced outside the model, across runtime validation, workflow execution, and landing infrastructure. Teams can also pause a pipeline, agent, or workflow. The ability to investigate a problem therefore does not silently become permission to take every subsequent action.

The design connects authority to the specific capability and action. Teams can delegate useful work while retaining checkpoints where the consequences require human judgment.

Putting it all together: the lead agent and the queue

Product context, evaluated specialists, and execution identities give us the pieces. To connect them into an ongoing maintenance loop, Autopilot uses a lead agent to coordinate each team’s on-call queue. It follows tasks across investigation, implementation, verification, and review, delegating bounded work to specialists along the way.

One architectural distinction is that a maintenance task outlives any one agent run. An investigation can finish while the task still needs a fix. A fix can be ready while tests or reviews are pending. The system needs to retain both what has happened and what must happen next across those boundaries.

Keep the queue moving

The lead agent triages incoming tasks, records findings and decisions, and routes work to an installed specialist or workflow. Each task carries a current status, a summary of the investigation, and the next steps. That record gives the lead agent a starting point when it returns to the task (Figure 5).

New work, completed runs, review activity, and scheduled rechecks can wake the lead agent. If a task is blocked, it can be snoozed with a specific condition to revisit, such as checking whether a code change has landed. When the task returns, the lead agent checks that condition and decides what to do next. Other tasks can progress while this one waits.

Let’s return to the hypothetical notification delay. Task context selects a performance specialist; the product pack supplies the delivery architecture and verification guidance; the configured identity provides access. The specialist produces a diagnosis. A later step may produce a candidate change, followed by verification and review. Those are separate results, each contributing evidence toward resolving the original task.

For work expressed as a workflow, the runtime persists the current step, run attempts, and output artifacts. Each step has an output contract: The evidence it must produce before downstream work can proceed. Valid completed steps normally advance automatically. The lead agent handles follow-through and exceptions that require another decision, such as a failed step or a request for more evidence.

Recover from recorded state

Recovery starts by inspecting the task, workflow, and run records. A timeout alone does not establish that a job failed; it may still be queued or running. Starting another run immediately could duplicate work. The recovery decision depends on what the system can establish has already happened (Table 1).

These checks keep waiting distinct from failure. External waits have recorded conditions and deadlines. Human checkpoints retain the proposed action and the required decision. Neither requires an agent execution to stay alive for the entire wait.

In the notification example, suppose an implementation step creates a candidate change and the workflow stops before verification begins. Recovery first checks that the step finished and its required outputs were saved. It can then continue with verification using that change. If verification fails, the failure evidence informs the next attempt. If review is pending, the task continues waiting for that decision.

For someone building a similar system, this is the state to make explicit: what completed, what is still running, what is waiting, and what evidence permits the next step. Persisting it lets a fresh execution continue the task without reconstructing the entire investigation. It also gives people a concrete place to inspect progress and intervene.

Feed the outcome back into the system

The lead agent updates the task with the outcome and any remaining decision for its owner. A completed run is one event in that history; resolving the maintenance task requires the relevant verification and review evidence.

The outcome can also improve future runs. A missing architecture fact belongs in the product pack. A weak investigation method belongs in the agent definition. A revealing failure may become an evaluation case. Teams make those updates deliberately, with each improvement going back to the component and owner responsible for it. This is what “learn” means in the maintenance loop.

Measure maintenance outcomes and human effort

Offline agent quality is one part of the picture. The operating model also needs to be assessed through teams using the system, maintenance tasks resolved, production changes landed, time to first useful action, and improvement in task resolution. Each measure needs a definition, a time window, and an appropriate comparison.

A useful first action might be a reproducible diagnosis or an escalation with evidence and a specific decision to make. In contrast, an acknowledgement says little about progress. Resolution should be tied to the task’s verification criteria, and a drafted change should remain distinct from one that was reviewed and landed.

The denominator matters just as much as the headline. Looking only at completed runs can hide failed dispatches, abandoned investigations, and repeated requests for human help. An assessment should describe the eligible task population, distinguish resolved work from escalations and justified no-action decisions, and account for changes in task mix.

Human attention is another requirement. More generated proposals can mean more review work. The useful question is whether engineers spend less time reconstructing context and coordinating routine steps while retaining the evidence needed to make good decisions. Measuring that burden alongside product outcomes tests the benefits of the complete loop.

What changes when the loop works

When the loop works, maintenance can begin closer to the signal. Finder agents can surface opportunities before they become urgent, and escalations can arrive with the evidence and a specific decision for a person to make. The system carries more of the routine coordination between those decisions.

Engineers remain accountable for architecture, priority, and risk. Their role includes maintaining product truth, selecting capabilities, reviewing consequential changes, and supervising the system’s behavior. Product ownership now includes caring for the context and controls that make delegated work dependable.

Expand from a bounded starting point

For teams building a similar system, a bounded maintenance queue is a practical starting point. Establish ownership, define a useful outcome and a verification path, and make escalation criteria explicit. Then inspect where tasks stop progressing: at missing context, weak agent behavior, failed tooling, or a decision that needs a person.

Those failures point to different improvements. Keeping context, capability, workflow state, evaluation, and authority separate makes it possible to improve the relevant part without redesigning the entire system.

Our direction is broader proactive coverage, richer product packs, and more reusable specialist agents. Product teams can contribute local knowledge, agent providers can publish expertise, and both can contribute representative evaluation cases. Responsibility can expand as evaluation and production evidence support it.