Advanced Computing in the Age of AI | Thursday, February 29, 2024

AI Startup NeuReality Aims Its Inference Platform to ‘Replace Outdated AI System Architecture’ 

Only three months after emerging from stealth with $8 million in seed funding, Israeli AI startup NeuReality has revealed plans for its all-new NR1-P novel AI-centric inference platform, which the company claims will provide a 15X improvement in performance per dollar compared to the GPUs and ASICs from other deep learning chip vendors.

NeuReality says the NR1-P is a new type of System-on-Chip (SoC) that replaces what it calls the “outdated” AI system architecture from competitors.

The NR1-P platform, which is designed for use in cloud data centers and edge nodes, is being demonstrated to customers and partners and is expected to be followed by other implementations, according to the company. The new prototype platform is being used to validate the technology and is giving customers the chance to integrate it in orchestrated data centers and other facilities and see it in real-world use.

NeuReality “is highly focused on its follow-on NR1 SoC, which will leverage its unique fusion architecture to bring a 15X improvement in performance per dollar compared to the available GPUs and ASICs offered by deep learning accelerator vendors,” the company said in its press release.

The startup’s prototype platform is aimed at improving AI development in a wide range of applications, including public safety, e-commerce, social networks, medical use-cases, digital assistants, recommendation systems and Natural Language Processing (NLP), according to the company. The NR1-P platform is targeted at cloud and enterprise data centers, carriers, telecom operators and other near-edge compute requirements.

This is the NeuReality AI-centric platform integrated with Xilinx. Image courtesy NeuReality

The NR1-P is built in a 4U server chassis equipped with sixteen Xilinx Versal Adaptive Compute Acceleration Platform (ACAP) cards, which are integrated, software-programmable compute platforms that the company says can provide performance beyond that of traditional FPGAs. The NR1-P SoC leverages these capabilities to implement the AI-centric compute engine while fusing unique system functions with datapath functions and the neural network processing function. These features come together to optimize critical AI-inference flows while removing system bottlenecks, according to NeuReality.

“NeuReality’s scalable approach couples the required AI compute engines with all the needed system functions and turns them into a managed resource element in the disaggregated and orchestrated data center of today,” the company told EnterpriseAI in a statement. “Through its unique fusion technology, NeuReality managed to repartition the system in a way that moves expensive data path functions from software to hardware. The novel approach eliminates single points of failures, lowers the cost and power consumption of various system functions, and helps deliver a much more affordable infrastructure for NeuReality’s customers.”

Benchmark testing done by the startup showed that the NR1-P prototype platform delivered more than a 2X improvement in Total Cost of Ownership (TCO) compared to Nvidia’s CPU-centric platforms running disaggregated deep learning service, according to NeuReality. “The current achievement also paves the way towards additional dramatic improvements that will be integrated into NR1, NeuReality’s flagship product,” the startup said.

Founded in 2019, NeuReality has been working with Xilinx to bring the AI platform project to fruition. The company received its $8 million in seed funding in February and also announced the appointment of Dr. Naveen Rao, an AI industry expert, and former general manager of Intel’s AI products group, to its board of directors. Prior to his role at Intel, Dr. Rao had been the CEO of Nervana Systems, which was acquired by Intel in 2016.

NeuReality executives could not be reached by EnterpriseAI by press time to further discuss the system’s architecture and history, however CEO Moshe Tanach recently shared more about the product in detail on an Orchestrate all the Things: Connecting the Dots with George Anadiotis podcast on Spotify.

Alex Norton, analyst, Hyperion Research

Alex Norton, the principal technology analyst and data analysis manager for HPC and emerging technologies with Hyperion Research, said the NeuReality strategy is impressive for a startup.

“NeuReality is focusing on a very specific part of the AI food chain, the inference portion,” said Norton. “And they have picked a few initial applications to target for their platform, something many other companies have done, but is crucial in bringing a product that can differentiate from others. In my mind, competition is good, it will continue to push these companies to get it right, but it will also allow for users to find the solution that best fits the problems at hand.”

The company is also wise because though it is emerging quite quickly from stealth, it is giving users early access to their platform as a service, which “is a great way of getting a lot of end users to experiment and run applications on their hardware without having to mass produce their hardware,” said Norton. “They are developing a software stack to go with their hardware, a key part in getting their technology out there and in the hands of end users. Ease of use will be essential for any new hardware solution in this space, so the software, as well as their participation in the Open Neural Network Exchange (ONNX) [community project], will certainly enable more end users to run on their platform.”

There are many companies already in the AI chip market, and there is no clear winner yet, said Norton. “Yes, Nvidia still holds a large market lead in the accelerator technologies, but it is still very early for application-specific architectures. At the end of the day, it’s hard to accurately predict where a company like this could go without seeing any real-world use cases or benchmark numbers, but the technology is definitely different from others, and could be exciting to watch.”

And while everyone seems to talk about China and the US when discussing the AI chip market, Israel also plays an integral part in the development of these technologies, said Norton. “Israeli-based companies have been emerging with novel and impressive technologies in this space for a few years now.”

Karl Freund, analyst, Cambrian AI Research

Karl Freund, the founder and principal analyst for machine learning, HPC and AI with Cambrian AI Research, said he sees NeuReality’s appointment of former Intel alumnus Naveen Rao to its board as “perhaps even more significant than this first chip.”

Freund said “we will see how [the chip] fares; there are scores of companies developing similar platforms, from FPGA vendors like Flex Logix to many ASIC developers.”