Advanced Computing in the Age of AI | Sunday, May 19, 2024

30,000 Images/Second: Xilinx and AMD Claim AI Inferencing Record 

Xilinx Alveo Accelerator Card

On the heels of its dual announcement at the Open Compute Project Summit in Amsterdam this week (see related story), Xilinx yesterday disclosed that AMD and Xilinx have teamed to set an AI inference processing record of 30,000 images per second.

The joint work of the two companies, announced at the Xilinx Developer Forum in San Jose by Xilinx CEO Victor Peng and AMD CTO Mark Papermaster, connects AMD’s EPYC CPUs and the new Xilinx Alveo FPGA accelerator card, announced yesterday at the OCP Summit.

The record, running a batch size of 1 and Int8 precision, was accomplished on a system that leverages two AMD EPYC 7551 server CPUs with PCIe connectivity, along with eight Alveo U250 accelerator cards. In a blog post, Xilinx said the inference performance is powered by Xilinx ML Suite, which allows developers to optimize and deploy accelerated inference and supports various machine learning frameworks, such as TensorFlow. The benchmark was performed on the GoogLeNet convolutional neural network.

According to the blog, “AMD and Xilinx have shared a common vision around the evolution of computing to heterogeneous system architecture and have a long history of technical collaboration. Both companies have optimized drivers and tuned the performance for interoperability between AMD EPYC CPUs with Xilinx FPGAs. We are also collaborating with others in the industry on cache coherent interconnect for accelerators (the CCIX Consortium – pronounced “see-six”), focused on enabling cache coherency and shared memory across multiple processors.”

With its 32 cores, 64 threads, eight memory channels with up to two TB of memory per socket, and 128 PCIe lanes coupled with a hardware-embedded x86 server security solution, Xilinx said AMD EPYC is “the perfect” CPU platform for accelerating artificial intelligence and high performance computing workloads.

“EPYC is designed to deliver the memory capacity, bandwidth and processor cores to efficiently run memory-intensive workloads commonly seen with AI and HPC. With EPYC, customers can collect and analyze larger data sets much faster, helping them significantly accelerate complex problems,” Xilinx said.