OpenMAS-GCom

Properties

axis: multi-agent
verification: mixed
whatItTests: Diagnostic attribution benchmark for graph-enhanced multi-agent LLM systems (G-MAS): compares 17 single-agent, ordinary multi-agent, and graph-enhanced configurations across 29 datasets in six domains, plus 400 new G-MAS-Complex multi-document tasks, using controlled interventions (rewiring communication edges, removing specialist or critic agents, corrupting intermediate messages, disabling workers) to attribute performance changes to specific communication structures and roles rather than confounding model, prompt, and cost differences.
discriminates: [role-specialization]
saturation: open
patternLabFit: Its controlled-ablation methodology (swap one component, hold everything else fixed) is a template for isolating which Pattern Lab topology or role choice actually drives a measured lift, rather than crediting the whole pipeline; the finding that removing a specialist hurts more than removing a critic is a concrete prior for prioritizing role-specialization experiments.
source: OpenMAS-GCom: A Diagnostic Benchmark for Graph-enhanced Multi-Agent Systems (arXiv:2609.21527)
notes: Submitted 2026-09-18, within the discovery window. Distinct from MultiAgentBench (milestone KPIs across fixed coordination topologies) and BattleAgentBench (staged difficulty ladder): OpenMAS-GCom's defining mechanism is a controlled-intervention diagnostic designed specifically to attribute performance to communication structure and role assignment rather than to compare topologies head to head. Verification marked mixed: the abstract states accuracy and accuracy-per-token metrics from controlled interventions but does not specify whether underlying task scoring is exact-match or judge-based across all 29 datasets.

Provenance

Source: https://arxiv.org/abs/2609.21527

Properties

axis

multi-agent

verification

mixed

whatItTests

Diagnostic attribution benchmark for graph-enhanced multi-agent LLM systems (G-MAS): compares 17 single-agent, ordinary multi-agent, and graph-enhanced configurations across 29 datasets in six domains, plus 400 new G-MAS-Complex multi-document tasks, using controlled interventions (rewiring communication edges, removing specialist or critic agents, corrupting intermediate messages, disabling workers) to attribute performance changes to specific communication structures and roles rather than confounding model, prompt, and cost differences.

discriminates

[role-specialization]

saturation

open

patternLabFit

Its controlled-ablation methodology (swap one component, hold everything else fixed) is a template for isolating which Pattern Lab topology or role choice actually drives a measured lift, rather than crediting the whole pipeline; the finding that removing a specialist hurts more than removing a critic is a concrete prior for prioritizing role-specialization experiments.

source

OpenMAS-GCom: A Diagnostic Benchmark for Graph-enhanced Multi-Agent Systems (arXiv:2609.21527)

notes

Submitted 2026-09-18, within the discovery window. Distinct from MultiAgentBench (milestone KPIs across fixed coordination topologies) and BattleAgentBench (staged difficulty ladder): OpenMAS-GCom's defining mechanism is a controlled-intervention diagnostic designed specifically to attribute performance to communication structure and role assignment rather than to compare topologies head to head. Verification marked mixed: the abstract states accuracy and accuracy-per-token metrics from controlled interventions but does not specify whether underlying task scoring is exact-match or judge-based across all 29 datasets.