This analysis organizes the evidence behind agent Infrastructure Has Entered the Benchmark Era, then explains the practical implications, trade-offs and current limits.
Read the evidence below as a decision trail: what changed, why it matters, which trade-offs shaped the result, and where the conclusion still depends on context.
Abstract: Artificial Analysis, an AI evaluation agency, launched the industry's first multi-vendor open agentic AI benchmark test AA-AgentPerf, which is measured using the execution trace of a real program agent. NVIDIA GB300 NVL72 can run 61.4K parallel agents per million watts of power, which is approximately 20 times that of H200 (2.6K). The point is not who is ahead, but that agents finally have a common standard against which they can be compared.
- Artificial Analysis launched AA-AgentPerf on June 12, 2026, claiming to be the industry's first multi-vendor open agentic AI benchmark test.
- It uses public libraries and real coding agent trajectories across more than 12 languages to measure how many parallel agents can be run per million watts of electricity.
- NVIDIA GB300 NVL72 runs 61.4K parallel agents per million watts, H200 runs 2.6K, a gap of about 20 times (DeepSeek-V4-Pro, under specific SLO conditions).
[Core Insight] The agent has moved from "whether it can run" to "how efficiently it can run"
In the past, AI agents were mostly compared to their abilities—whether they could complete tasks and whether they were done correctly. AA-AgentPerf asked another question: With the same power, how many agents can you run at the same time?
Artificial Analysis launched this benchmark on June 12, 2026, positioning it as "the industry's first multi-vendor open, profiling real AI agent coding task trajectory" test. It does not use artificial questions, but captures the execution trace of real coding agents in public libraries - reading files, writing and modifying programs, executing instructions, and then iterating based on the results - covering more than 12 programming languages.
【In-depth dismantling】
What exactly does AA-AgentPerf measure?
The core indicator it measures is "parallel agent capacity per million watts of electricity", and sets two service level objectives (SLO): 20 tokens and 60 tokens per second per agent. In other words, it translates the abstract "performance" into a unit closer to the bill - how many agents can you serve at the same time under a fixed power budget, and each one still runs fast enough.
61.4K vs 2.6K: The meaning behind the numbers
At first SLO setting, NVIDIA GB300 NVL72 runs per megawatt 61.4K parallel agents, an average of 57.5 per GPU; the previous generation H200 had 2.6K per megawatt and 1.4 per GPU. The difference is about 20 times. The test model uses DeepSeek-V4-Pro. NVIDIA attributes this gap to the 72-GPU rack-level NVLink connection of GB300 NVL72, as well as inference optimizations such as fused MoE and MXFP4/MXFP8 kernel.
Why is the unit "per million watts"?
The bottleneck of agentic workload is different from single question and answer. Its characteristic is to "serve a massive number of agents at the same time." At this time, the real ceiling is often not the computing power of a single card, but electricity. Using "parallel capacity per million watts" as the unit is equivalent to directly aligning the cost and power supply reality of the data center. For teams that plan to deploy agents on a large scale, this indicator is closer to the electricity bill you will pay in the end than "a few tokens per second".
【My point of view】
I think the emergence of this benchmark is more worthy of note than "which hardware is leading". When a capability has a recognized way of measuring it, marketing rhetoric is forced back to verifiable numbers. This is a good thing for people making decision-making evaluations - you can finally compare using "parallel capacity per million watts" instead of watching a cut demo.
But be cautious: this type of benchmark is currently released under the leadership of manufacturers. NVIDIA itself claims that it is ahead. How to set the SLO and which model to measure it with will affect the conclusion. I would regard AA-AgentPerf as the starting point for "the industry to start to have a common language", rather than the final judge. What is really valuable is whether other manufacturers are willing to come forward for testing and whether the numbers can be cross-verified.
For the Taiwanese team, a pragmatic reading is: if you want to provide large-scale agent services, procurement and cloud selection should start to put "power efficiency/parallel capacity" into the comparison table, rather than just looking at single card specifications. The signal of this round is that the cost competition for agents has moved from "how strong the model is" to "how much the same power can run."
【FAQ】
What is the difference between AA-AgentPerf and general LLM benchmarks?
Generally, LLM benchmarks mostly measure the quality or accuracy of a single answer. AA-AgentPerf measures the parallel efficiency and power efficiency under agent workload - using real coding agent execution traces to measure "how many agents can run simultaneously per million watts of power", which is closer to actual deployment and cost.
Is it true that GB300 NVL72 is 20 times faster than H200?
According to data released by NVIDIA, under the first SLO setting, GB300 NVL72 can run 61.4K parallel agents per million watts, while H200 can run 2.6K, a gap of about 20 times. However, this is the result under the DeepSeek-V4-Pro model and specific SLO conditions. The numbers will change if the model or settings are changed.
Do teams that don’t have their own computer rooms need to care about this?
Needed. The cost of cloud agent services will ultimately be linked to the power efficiency of the hardware. The higher the efficiency, the lower the cost per unit agent. Even if you don't purchase hardware, parallel capacity and power efficiency are still worthy comparison dimensions when choosing a cloud provider or solution.
Original source:NVIDIA Blackwell Leads on First Agentic AI Infrastructure Benchmark（NVIDIA）、NVIDIA Achieves Leading Agentic Coding Performance on First Agentic AI Benchmark（NVIDIA Developer）
What to take away
The article's value is in the evidence and trade-offs behind agent Infrastructure Has Entered the Benchmark Era, not in treating the conclusion as universal.