Multi-Agent Safety Evaluation: Beyond Single-Model Testing

You tested each agent individually. Every one passed. Then you deployed them together and an attacker compromised your entire pipeline through a single poisoned input that cascaded across four agents in under ninety seconds.

Single-model safety evaluation is necessary but insufficient. The attack surface of a multi-agent system is not the sum of its parts — it is the product of every interaction between them. Emergent behaviors, trust assumptions between agents, and cascading failures create risks that no individual agent test will reveal.

This post explains why multi-agent systems require fundamentally different evaluation approaches, what those approaches look like in practice, and how to build evaluation pipelines that catch the failures that matter.

Why Single-Agent Testing Fails for Multi-Agent Systems

Standard LLM safety evaluation follows a pattern: send adversarial inputs to a model, check if the outputs violate policy. Red-teaming, benchmark suites like HarmBench, automated prompt injection scanning — all operate on the same assumption: one input, one model, one output.

Multi-agent architectures break every part of that assumption.

The input is not singular. A downstream agent receives outputs from upstream agents, user inputs, tool results, and retrieved context — all simultaneously. An input that is harmless in isolation becomes dangerous when combined with context from another agent.

The model is not singular. Each agent in a pipeline has different capabilities, permissions, and system prompts. An attacker does not need to compromise the most hardened agent. They need to find the weakest one and use it as a bridge.

The output is not singular. Agent outputs become inputs to other agents, trigger tool calls, modify shared state, and influence downstream decisions. A subtle manipulation in one agent's output compounds through the pipeline.

The OWASP Top 10 for Agentic Applications (2026) identifies cascading failures (ASI08) as a distinct risk category: false signals that propagate through automated pipelines with escalating impact. A single compromised agent corrupts downstream agents' decision-making, and the damage amplifies at each hop.

The Three Failure Modes Single-Agent Tests Miss

1. Cross-Agent Injection Propagation

An attacker injects a malicious instruction into Agent A's input. Agent A does not act on it — its safety filters catch the injection attempt. But Agent A passes the instruction forward in its output summary. Agent B, which trusts Agent A's output as vetted internal data, executes the instruction without question.

This is the trust boundary problem. Within a multi-agent system, agents implicitly trust each other's outputs. That trust is rarely validated and almost never tested.

What this looks like in practice:

Agent A (Research): Receives web page containing hidden instruction
  → Summarizes content, injection payload survives in summary
Agent B (Analyst): Receives summary, treats it as trusted input
  → Incorporates malicious framing into analysis
Agent C (Executor): Receives analysis with embedded command
  → Executes tool call based on poisoned analysis

A single-agent test on Agent C would never surface this vulnerability. Agent C behaved correctly — it followed the analysis it received. The failure is in the pipeline's trust model, not in any individual agent.

2. Emergent Capability Escalation

Individual agents have restricted tool access. Agent A can read files. Agent B can make HTTP requests. Agent C can write to a database. No single agent has the full attack chain.

But when agents collaborate, their combined capabilities create an attack surface none of them has alone. An attacker who manipulates the orchestration logic — or poisons the shared context agents read from — can chain Agent A's file read into Agent B's HTTP request to exfiltrate the data Agent C retrieved from the database.

This is privilege composition: the effective privilege of a multi-agent system is the union of all agents' permissions, even though no single agent holds that full set. Standard penetration testing evaluates each agent's permissions in isolation and concludes each is appropriately scoped. The composition is never tested.

3. Cascading State Corruption

Multi-agent systems share state — memory stores, databases, message queues, shared context windows. When one agent writes corrupted data to shared state, every downstream agent that reads from that state is compromised.

Unlike single-agent memory poisoning, which affects one agent's future sessions, shared-state corruption is both immediate and lateral. It does not require waiting for the poisoned agent to be invoked again. Every agent that touches the corrupted state is affected in real time.

Evaluation that only tests agents as independent units will never detect shared-state vulnerabilities because the state itself is the attack vector, not any individual agent's behavior.

What Multi-Agent Safety Evaluation Requires

Pipeline-Level Red Teaming

Evaluation must operate at the pipeline level, not the agent level. This means:

1. Inject adversarial inputs at every entry point and trace propagation.

Do not just test the user-facing agent. Inject into tool results, retrieved documents, inter-agent messages, and shared memory. Track whether the injection survives, transforms, or amplifies as it moves through the pipeline.

2. Test trust boundaries explicitly.

For every agent-to-agent handoff, verify: Does the receiving agent validate the input? Does it distinguish between trusted internal data and potentially compromised upstream output? Most systems fail this test because they were not designed with inter-agent distrust in mind.

3. Evaluate composite privilege paths.

Map every combination of tool access across agent chains. Identify paths where chaining Agent A's read with Agent B's write with Agent C's network access creates an end-to-end capability that no single agent should have. Then test whether those paths are exploitable.

Interaction-Aware Benchmarking

Static benchmarks that evaluate agents on fixed input/output pairs miss interaction dynamics. Multi-agent evaluation needs:

• Multi-turn adversarial scenarios where the attack unfolds across several agent interactions, not a single prompt
• Shared-state mutation tracking that monitors how agent writes to shared resources affect downstream agent behavior
• Orchestration logic testing that verifies the system's routing, delegation, and fallback mechanisms cannot be manipulated to bypass safety controls
• Timing-sensitive tests that evaluate race conditions when multiple agents read and write shared state concurrently

Continuous Evaluation, Not Point-in-Time

Multi-agent systems are dynamic. Agents are updated independently. New agents are added. Tool permissions change. A system that passed evaluation last week has a different risk profile today because one agent's system prompt was modified.

Evaluation must run continuously, triggered by any change to any agent in the pipeline. This requires:

• Automated regression suites that execute pipeline-level safety scenarios on every deployment
• Drift detection that flags when an agent's behavior diverges from its evaluated baseline
• Composition analysis that re-evaluates privilege paths whenever agent permissions change

Building an Evaluation Pipeline

A practical multi-agent evaluation pipeline has four layers:

Layer 1: Individual Agent Screening. Standard prompt injection detection, output filtering, and safety benchmarks. This is table stakes — necessary but not sufficient.

Layer 2: Pairwise Interaction Testing. For every pair of agents that communicate, test whether Agent A can influence Agent B to violate its safety constraints. Test both direct instruction injection and subtle context manipulation.

Layer 3: End-to-End Pipeline Scenarios. Full pipeline tests with adversarial inputs injected at each entry point. Measure propagation distance (how many agents does a poisoned input reach?), amplification factor (does the impact grow at each hop?), and time-to-detection (how long before the system identifies the compromise?).

Layer 4: Continuous Monitoring. Runtime behavioral analysis that detects anomalous agent interactions in production. This catches emergent behaviors that no pre-deployment test anticipated.

The evaluation complexity scales with the number of agents. A 3-agent pipeline has 6 pairwise interactions. A 10-agent pipeline has 90. Automation is not optional — it is the only way to maintain coverage as the system grows.

How Parse Approaches Multi-Agent Evaluation

Parse for Agents provides pipeline-level safety evaluation designed for multi-agent architectures. Rather than scanning agents individually, Parse evaluates the interaction patterns between them.

The multi-agent safety evaluation API accepts a pipeline definition — the agents, their connections, their shared resources — and runs adversarial scenarios that target the interaction layer:

const evaluation = await fetch('https://www.parsethis.ai/api/v1/agents/pipeline-eval', {
  method: 'POST',
  headers: { 'Authorization': 'Bearer YOUR_API_KEY' },
  body: JSON.stringify({
    pipeline: {
      agents: ['research', 'analyst', 'executor'],
      connections: [
        { from: 'research', to: 'analyst' },
        { from: 'analyst', to: 'executor' }
      ],
      shared_state: ['context_store', 'task_queue']
    },
    scenarios: 'cross-agent-injection,privilege-escalation,state-corruption',
    depth: 'full'
  })
});

// Returns: {
//   risk_score: 0.72,
//   propagation_paths: [...],
//   trust_boundary_violations: 3,
//   composite_privilege_risks: ['read→exfiltrate via research→analyst chain'],
//   recommendations: [...]
// }

Parse's 12 analysis agents evaluate injection propagation across agent boundaries, identify composite privilege escalation paths, and test shared-state corruption scenarios — the three failure modes that single-agent testing misses entirely.

Actionable Steps

1. Map your agent interaction graph. Document every agent-to-agent communication path, shared resource, and trust assumption. You cannot evaluate what you have not mapped.

2. Add inter-agent input validation. Every agent should treat upstream agent output as potentially compromised. Apply the same input sanitization to inter-agent messages that you apply to user inputs.

3. Test privilege composition. Enumerate the combined tool access across every agent chain in your pipeline. Flag any chain where the composite capabilities exceed what any individual agent should have.

4. Run pipeline-level adversarial scenarios. Inject at every entry point — not just the user-facing interface. Use Parse's pipeline evaluation to automate cross-agent injection and state corruption testing.

5. Implement continuous evaluation. Trigger safety re-evaluation on every agent update, permission change, or new agent addition. A pipeline that was safe yesterday is not guaranteed safe today.

The Bottom Line

The security of a multi-agent system is determined by its weakest interaction, not its strongest agent. Single-model testing gives you false confidence by evaluating components that behave differently in composition than they do in isolation.

Evaluate the pipeline. Test the interactions. Monitor the composition.

Evaluate your multi-agent pipeline's safety posture. Try Parse for Agents free.