Advanced Computing in the Age of AI | Thursday, March 28, 2024

AI Chip Makers Must Meet Customer Needs to Gain Design Wins for Success 

AI Hardware Summit – With more than 80 AI accelerator chip companies battling for elusive market share around the world, it’s no longer enough for upstart AI chip makers to simply develop and build powerful and innovative chips.

What nascent AI chip makers must do to gain standout design wins is to directly meet and exceed the specialized requirements of prospective and hungry AI chip customers, especially when most of the GPU market is already in the hands of dominant global market leader, Nvidia.

Despite the challenges, though, there are ways to be successful, according to Andrew Richards, the co-founder and CEO of Codeplay Software, which provides tools to enable AI and HPC software to be accelerated by graphics processors and specialized AI processors. Richards spoke recently at the AI Hardware Summit, which was held virtually due to the COVID-19 pandemic. EnterpriseAI and HPCwire are media partners for the event.

Andrew Richards, CEO and founder of Codeplay Software

In his presentation, “Getting Design Wins for AI Accelerators,” Richards argued that when large customers seek new AI accelerators for their products, they want chips that will meet their benchmarks, fit their future roadmaps and meet their total cost of ownership requirements. And when evaluating prospective new chips, customers want to be sure that whatever they select won’t be an unreasonable risk compared to their current AI hardware platforms.

For AI chip makers, especially the newcomers and smaller players, that means focusing on the business model, as well as the technology side, he said.

“There is a huge market for AI accelerators, but currently it's massively dominated by GPUs from one vendor,” said Richards. “[Competing AI chip makers] want to take a portion of that market. So, the first thing we need to do is understand the market and the customers, and then we need to define a product that customers are going to buy. And then we need to build that product.”

That means looking at the market from the customer’s perspective, instead of touting a chip maker’s technical and design promises, he said.

Look at The Market with a New Perspective – What Do Customers Need?

“From a customer's point of view, they are going to want to see the benefits of adopting your AI processors,” Richards told event attendees, primarily AI chip developers. "There's going to be some performance or cost improvement, but they're also going to want to see some costs.”

They'll also need to make investments in porting their software to a vendor’s chip platform, so they’ll also want to understand those investment costs as well, he said. In addition, there are other issues that must be considered for the long term, such as will the chips be used in vehicles where long-term safety and maintenance are considerations.

“A customer is going to want to understand that for their whole application, not just for one neural network,” said Richards.

As the prospective sales process continues, customers will likely want to create representative workloads to test the chip designs using their specific benchmarks. “Then they're going to work with you to try and port those workloads to your chip,” said Richards. “And they're looking for two things here. They're looking for the performance of their workload, but they're also looking for how much effort is it to get their workload onto your chip.”

With that information, prospective customers will likely then estimate the size of their project and take it into account when working out the cost benefit analysis of buying a vendor’s chip, said Richards.

Product Roadmaps Are Critical

To resolve those concerns, chip makers must provide a roadmap for the future of their chips because the customers want assurance when investing to port their software, he explained.

“If it takes them a year or two to port their software across, then actually they're not going to be buying your first generation chip other than as a development case," Richards said. “The second generation chip is the one they're going to be interested in. That's why it's so important for them to understand your roadmap, to see what the future is. That allows them to understand the total cost of adoption and creates a future sales relationship. If you get through that, you get a design win.”

What AI chip makers must do is gain a larger vision of the process, he said. “You're not just selling a chip. You're selling an organization that your customers can invest in. They're not investing in your equity, like a venture capitalist would. They're investing in your platform. They're investing in your software development effort. So you've got to come across as investible from that point of view.”

For many companies, that’s where the process can collapse, said Richards.

“You've got to be really clear about what's your secret sauce and what's out in the open – who owns what” in your product and software, he said. “And you need to do that really early on because fundamentally this is your product definition. Your product definition to your customers is not that it runs ResNet-50 really fast. You're defining a platform that your customer's going to build that software on top of. So, actually the software capability of that platform is absolutely crucial to the product's definition.”

Be Clear, Be Open and Be Honest

For potential customers, they must have clarity about these issues, he added. “But also your engineers need to have clarity on it. They need to know who owns what, because they're going to be working on both sides of this IP barrier. That's one of the big challenges that you have. You're going to have to work out how to expose that by your specifications in some way. And your specifications will be based on things like industry standards.”

While Richards admitted that this to-do list for AI chip makers is long and complex, he also said that if such steps are undertaken eagerly, that real progress and more design wins can be achieved.

“I hope I didn't make it look too tough,” he said. “I'm highlighting the challenges where people really need to focus. There genuinely is an opportunity there – I can see those opportunities. We see them from customers wanting to be able to buy processors that are optimized for different kinds of workloads. And there's a whole load of reasons why people might want to move away from just having one proprietary supplier.”

 

EnterpriseAI