- axis
- agentic
- verification
- mixed
- whatItTests
- A large-scale multilingual agent benchmark built via BabelFlow, a benchmark-general pipeline that adapts existing English-centric agent benchmarks to new languages through structure-preserving translation, runtime-dependency analysis, and multi-layer verification plus human review; comprises 16,146 instances derived from 702 canonical tasks across four benchmark families, 13 domains, and 23 languages.
- saturation
- open
- patternLabFit
- Its finding that no single model dominates across benchmark families, and that low-resource languages show more tool-use and control-flow errors rather than just worse answers, plus roughly double the token cost versus English on the same task, is a concrete caution for Pattern Lab experiments assuming a pattern's gain transfers cleanly across languages.
- notes
- Submitted 2026-09-20, within the discovery window. Distinct from tracked PolyWorkBench (67 hand-built multilingual workplace tasks across 5 domains): BabelArena instead systematically translates and re-verifies four existing English-centric agent benchmark families into 23 languages at much larger scale, isolating cross-language failure patterns rather than authoring new multilingual tasks from scratch. Verification marked mixed: BabelFlow's multi-layer verification plus human review preserves whatever scoring the original four benchmark families used, not one uniform method.
