Deci Collaborates with Intel to Achieve 11.8x Accelerated Inference Speed at MLPerf 

TEL AVIV, Israel, Dec. 1, 2020 -- Deci, a deep learning company building the next generation of AI, announced its inference results that were submitted to the open division of the MLPerf v0.7 inference benchmark (full results here). On several popular Intel CPUs, Deci’s AutoNAC (Automated Neural Architecture Construction) technology accelerated the inference speed of the well-known ResNet-50 neural network. It reduced the submitted models’ latency by a factor of up to 11.8x and increased throughput by up to 11x-- all while preserving the model’s accuracy within 1%.

“Billions of dollars have been spent on building dedicated AI chips, some of which are focused on computer vision inference,” says Yonatan Geifman, CEO and co-founder of Deci. “At MLPerf we demonstrated that Deci’s AutoNAC algorithmic acceleration, together with Intel’s OpenVINO toolkit, enables the use of standard CPUs for deep learning inference at scale.”

According to MLPerf rules, Deci’s goal was to reduce the latency, or increase throughput, while staying within 1% accuracy of ResNet-50 trained on the Imagenet dataset. Deci’s optimized model improved latency between 5.16x and 11.8x when compared to vanilla ResNet-50. When compared to competing submissions, Deci achieved throughput per core that was three times higher than models of other submitters.

“Intel’s collaboration with Deci takes a significant step towards enabling deep learning inference on CPU, a longstanding challenge for AI practitioners across the globe,” said Guy Boudoukh from Intel AI. “Accelerating the latency of inference by a factor of 11x enables new applications and deep learning inference tasks in a real-time environment on CPU edge devices and dramatically cuts cloud costs for large scale inference scenarios.”

MLPerf gathers expert deep learning leaders to build fair and useful benchmarks for measuring training and inference performance of ML hardware, software, and services. The models submitted were optimized using Deci’s AutoNAC technology and quantized with Intel’s OpenVINO to 8-bit precision.

Deci’s patent-pending AutoNAC technology uses machine learning to redesign any model and maximize its inference performance on any hardware - all while preserving its accuracy.

For more in-depth information, see here.

About Deci

Deci is ushering in a new AI paradigm by using AI to build and operate AI models. Deci’s deep learning platform enables data scientists to transform their AI models into production-grade solutions on any hardware, crafting the next generation of AI for enterprises across the board. Deci’s proprietary AutoNAC (Automated Neural Architecture Construction) technology autonomously redesigns an enterprise’s deep learning models to squeeze the maximum utilization out of its hardware. Founded in 2019 and based in Tel Aviv, Deci’s team of deep learning experts are dedicated to eliminating production-related bottlenecks across the AI lifecycle to allow developers and engineers the time to do what they do best - create innovative AI solutions for our world’s complex problems.

Source: Deci