Deterministic-First Judge Panel

Agent action proposals pass first through a cascade of deterministic checks that block clear policy violations; only the ambiguous residual is referred to a panel of several LLM judges. In the reported security-operations domain only about a quarter of proposals reach the judges.

Properties

family: Evaluation & Ranking
maturity: frontier
diagram: layers
whyMultiAgent: The residual cases need several independent judges whose verdicts are combined, and the proposer must be separate from the checking layer it cannot influence; a single agent cannot audit its own proposals. Distinct from conditional-escalation (an executor consults one stronger advisor on low confidence) and from llm-as-judge (every output is scored): here a rule layer filters first and the judge panel only adjudicates what rules cannot decide.
mast: [Cat 3]
useCases: [Securing multi-agent systems against injected or malicious action proposals][Security-operations workflows where most decisions are rule-resolvable][Cost-bounded oversight that reserves LLM judges for the ambiguous tail]
source: Where Rules End and Judges Begin · arXiv:2610.07657

Provenance

Source: https://arxiv.org/abs/2610.07657

Deterministic-First Judge Panel

Properties

family: Evaluation & Ranking
maturity: frontier
diagram: layers
whyMultiAgent: The residual cases need several independent judges whose verdicts are combined, and the proposer must be separate from the checking layer it cannot influence; a single agent cannot audit its own proposals. Distinct from conditional-escalation (an executor consults one stronger advisor on low confidence) and from llm-as-judge (every output is scored): here a rule layer filters first and the judge panel only adjudicates what rules cannot decide.
mast: [Cat 3]
useCases: [Securing multi-agent systems against injected or malicious action proposals][Security-operations workflows where most decisions are rule-resolvable][Cost-bounded oversight that reserves LLM judges for the ambiguous tail]
source: Where Rules End and Judges Begin · arXiv:2610.07657

Provenance

Source: https://arxiv.org/abs/2610.07657