Chris Wolf’s Post

View profile for Chris Wolf, graphic

Global Head of AI and Advanced Services at VMware by Broadcom

Check out our latest #PrivateAI performance benchmark, showcasing the value of virtualization. You can share capacity among teams or tenants, share GPUs among applications, have a lower TCO and predictable costs, all without sacrificing performance. Great work Uday K., Lan Vu, and Hari Sivaraman to show how you can even connect a VM to 8 physical H100s (our max is 16), and achieve excellent MLPerf results. The summary nails the overall value, and check out the full results in the link. "We used only 32 logical CPU cores and 128GB of memory for this inference benchmarking—that’s a key benefit of virtualization. This lets you use the remaining CPU and memory capacity on the same systems to run other workloads, save on the cost of ML/AI infrastructure, and leverage the virtualization benefits of vSphere for managing data centers." https://lnkd.in/gywggCPN

As AI-based workloads shift into the enterprise data center for inference, etc, we'll see the value of running these workloads on shared, software-defined (aka virtualized) infrastructure to both maximize performance but also to maximize utilization of all those expensive GPUs and memory, etc.

Like
Reply

To view or add a comment, sign in

Explore topics