Operating model · managed AI agents
How managed AI agents work in production
A production agent is an operated system, not a prompt. It needs a reliable source of work, controlled context, narrowly scoped tools, action-level permissions, evaluation, traces, escalation, and a human owner who can change or stop it.
Cognate Labs operating definition
Cognate Labs defines a managed AI agent as an agentic system operated inside explicit work-state, role, context, tool, policy, evaluation, and observability controls—with human approval and escalation crossing the whole system.
The production agent loop
At its simplest, an agent receives work, assembles the permitted context, chooses an action, uses a tool, observes the result, and continues until it completes or escalates the task. Production design makes every part of that loop explicit.
01
Receive
A queue, event, message, or schedule creates a work item with an identifier.
02
Contextualise
The system retrieves only the instructions and records permitted for that role and case.
03
Decide
The model selects a next action from a bounded set or requests human input.
04
Act
A typed tool validates arguments, checks authority, performs the operation, and returns evidence.
05
Verify
The system checks completion criteria, records the trace, and either finishes, retries safely, or escalates.
The loop should have a step limit, time limit, cost limit, and termination conditions. An agent that can continue indefinitely is difficult to budget and unsafe to operate.
Cognate Agent Operating Model v1.0
| Component | Purpose | Failure if omitted |
|---|---|---|
| Work source | Creates a durable task and state | Requests disappear or run twice without detection |
| Role instructions | Define responsibility and constraints | Behaviour changes with vague prompts |
| Context layer | Retrieves allowed records and policies | The agent guesses or sees excessive data |
| Tool layer | Exposes typed, validated actions | The model acts through brittle free-form interfaces |
| Policy gate | Checks identity, scope, and action authority | A valid tool call becomes an unauthorised action |
| Evaluation | Tests expected and prohibited behaviour | Improvements and regressions are matters of opinion |
| Observability | Records decisions, calls, results, cost, and latency | Failures cannot be reconstructed or improved |
These seven components are Cognate Labs’ production model, not a universal standard. Human approval and escalation cross all seven. Approval is itself durable work: the reviewer needs the proposed action, relevant evidence, risk, and an unambiguous choice. A message that says only “approve?” moves the investigation burden back to the person.
Permissions belong to actions, not personalities
Avoid giving an agent a broad human account. Wrap each integration in narrow tools such as find_customer, draft_reply, or add_internal_tag. Validate inputs and enforce data scope outside the model.
| Action | Read | Propose | Execute | Prohibit |
|---|---|---|---|---|
| Customer record | Assigned tenant and approved fields | Correction note | Add reversible internal tag | Export full tenant or change access |
| External message | Approved thread and policy | Draft with cited evidence | None in initial release | Send legal, financial, or sensitive commitment |
| Financial record | Invoice metadata only | Possible duplicate flag | None | Post, refund, pay, or delete |
This illustrative permission matrix makes authority testable at the action and data-scope level. It also limits the effect of hostile instructions in retrieved content. Treat external text, attachments, webpages, and user messages as data—not as authority to redefine the role. OWASP LLM01:2025 explains why retrieved or user-supplied inputs cannot be assumed to preserve the intended instruction boundary.
Evaluate the work, not just the answer
A polished final message can hide a failed workflow. Agent evaluation must inspect the path: whether the correct sources were used, whether tool arguments were valid, whether prohibited actions were avoided, whether the task ended in the correct state, and whether escalation happened when it should.
- Routine cases that should complete automatically.
- Boundary cases that should request approval.
- Invalid or incomplete inputs that should be rejected.
- Conflicting instructions that should follow the approved policy.
- Tool failures, timeouts, duplicates, and partial completion.
- Adversarial content attempting to redirect or extract information.
Run the evaluation set before every meaningful change to instructions, models, tools, or retrieval. In production, sample completed work and feed confirmed failures back into the test set. The NIST AI RMF Core calls for pre-deployment and ongoing testing, documented metrics, monitoring, and risk response. It is a wider voluntary governance framework; the test design above is Cognate Labs’ application of those principles to agent workflows.
Observe business state as well as model state
Token use, latency, and model errors matter, but they do not prove the work succeeded. A managed agent also needs business-level signals: task age, completion rate, approval rate, correction rate, duplicate actions, unresolved exceptions, and the state of every external system it changed.
| Illustrative trace field | Example value | Operator use |
|---|---|---|
| Work item | invoice-intake/1042 · supplier email | Find the original trigger and current state |
| Context | invoice.pdf · supplier policy v3 | Reconstruct which evidence and policy were available |
| Tool decision | extract_fields → duplicate_check | Verify the allowed sequence and arguments |
| Policy result | posting denied · approval required | Confirm the authority boundary held |
| Outcome | draft record + possible duplicate flag | Inspect completion evidence and next action |
| Recovery | discard draft; preserve source and trace | Reverse the attempted change without losing evidence |
This synthetic trace is an illustration, not production telemetry or a client result. It shows the minimum chain an operator needs to reconstruct: trigger, permitted context, tool decision, policy result, changed state, evidence, and recovery path.
Alerts should lead to an actionable queue, not a stream of raw model logs. The human operator needs to see the affected task, its current state, the risk of waiting, and the available recovery action.
Release authority in stages
01
Evaluation only
Run representative historical cases and compare with accepted outcomes.
02
Shadow mode
Process live work without changing systems or contacting users.
03
Draft mode
Prepare changes and communications for explicit approval.
04
Bounded execution
Allow low-risk, reversible actions within limits.
05
Expanded operation
Increase scope only when measured performance and recovery controls justify it.
Each stage needs an entry criterion, rollback trigger, and owner. “The demo looked good” is not an entry criterion. Use a documented evaluation target and verify that the manual fallback still works.
What ‘managed’ should include
Managed operation continues after launch. Models change, upstream APIs change, policies change, and the distribution of real work drifts. Someone must own evaluation, incidents, costs, permissions, tool changes, and the decision to narrow or retire the agent.
- A named service owner and named client-side accountable owner.
- Versioned instructions, tools, models, and evaluation results.
- Routine performance and exception review.
- Incident response, rollback, and manual continuity procedures.
- Permission review and explicit approval for scope changes.
- Cost and value review against the workflow baseline.
This operating layer is what turns agent capability into an AI employee that a business can responsibly assign work to. It is also the centre of Cognate Labs’ managed AI agent service.