DriveNets and Bentaus announced a deployment on November 12 that combines AMD Instinct MI325X GPUs, DriveNets Ethernet fabric and Bentaus's Ziani power management software. DriveNets reported that Bentaus retained 97% of token throughput while using 30% less GPU power in its cluster test. The accompanying case study attributes that result to compute power management on the deployed cluster. Deployment announcement · DriveNets case study
The release also describes the result in terms of energy per token. GPU power, energy per token and facility demand each need their own measurement boundary. The case study gives operators a reason to test a similar configuration, with the deployment's figures treated as vendor-reported results for that cluster.
Measure from the GPU to the service entrance
Start by documenting what each meter covers. Use GPU telemetry to confirm that the cap took effect. Read rack or IT distribution meters to see how server and network demand changed. If the target is a reduction in utility draw, measure the applicable service entrance or agreed metering point as well.
Host CPUs, switches and cooling can remain active while GPU power falls. Cooling may respond on a different timescale from the accelerators. A short measurement at the GPU tells you when the control acted. A time series at the service entrance tells you whether demand fell by the required amount and stayed there for the required interval.
For energy per token, integrate energy over the measurement interval and divide it by the tokens successfully delivered in that interval. Report the meter boundary and counting rule alongside the result. Keep the denominator consistent across runs so retries and abandoned requests do not inflate output.
Put latency beside token throughput
Run the same model, serving configuration and arrival pattern with and without the power cap. Include a representative distribution of prompt lengths and output lengths. Keep the software versions and active accelerator count in the test record.
Track time to first token, time between output tokens, request completion, queue depth and failures alongside total throughput. A cluster may maintain aggregate output while some users wait longer. Look at latency percentiles by workload class to see which requests absorb the slowdown.
Test at a sustained busy load as well as under bursts. This helps distinguish a cap the cluster can tolerate during a quiet interval from one it can sustain during an actual demand event. Set the acceptable service limits before running the comparison.
Keep the network configuration fixed first
The deployment combines a tuned fabric and power orchestration. I would first hold the fabric configuration constant and vary the GPU cap. That isolates the change in behavior caused by the cap under the tested workload. Run a separate comparison if the goal is to quantify the fabric's contribution.
On multi-GPU requests, correlate serving traces with fabric telemetry. Look for longer collective operations, congestion or idle intervals as the power limit changes. A stable token rate is easier to interpret when you can see whether the workload had spare compute capacity or was spending much of its time waiting on another resource.
Test the return to normal power
A demand event has an end as well as a start. Include the recovery ramp in the test window. Observe what happens when the cap is released while requests are queued. Measure the peak draw at the same facility meter and the time required for service latency to settle.
For an operating agreement, record the size and duration of the metered reduction, the allowable recovery ramp and the workload limits observed during the test. Specify who can activate the cap and what conditions cause operators to release it. That gives the facility and serving teams a shared record of the demand reduction they can deliver while meeting their service requirements.
