What Enterprises Need to Know to Get AI Inference Right 

AI is rapidly shifting from a technology used by the world’s largest service providers – like Amazon, Google, Microsoft, Netflix and Spotify – to becoming a tool that midsize enterprises adopt to build new products, grow revenue, increase customer engagement and retention, and expand into new markets.

Yet despite all the excitement around AI, many CIOs have a tough time figuring out how to create a skill set within their organizations to handle not just AI development, but the training and deployment of anything developed. Indeed, just 26% of organizations this year have AI in production, while 43% percent are in the evaluation stage, according to research firm O’Reilly. This emphasizes the difficulty of moving AI from development and into operation – and as we’ll see, back to development again.

There’s potential growth in every challenge, though, and the global AI economy is expected to reach $15.7 trillion by 2030, according to a PwC study. Any organization looking to capitalize on this growth, whether in smart manufacturing, retail, healthcare, energy and many more industries, needs to know how inference is key to putting AI to work.

Where the AI Rubber Meets the Road

There’s plenty of handy information and expertise in the area of AI development. For many a CIO, it’s deployment that represents the biggest challenge. Inference – the process of running an AI model in production – is where the rubber can literally meet the road. In autonomous vehicles, for instance, critical AI decision-making happens in millisecond increments – with people’s lives on the line.

Unlike a traditional application, like sales management software, the AI models running inference need to be continually retrained and redeployed to stay current. This makes it more complex to manage the AI application lifecycle, even if the benefits are significant.

Boost Sales, Avoid Shutdowns, Serve Customers with Inference

Inference is key to solving a number of challenges that many industries face today. Deep learning can help automate functions, recommend products and even offer natural language processing.

Across retail and entertainment, and even professional social networks, recommender systems running inference can help to smooth out lumpy sales cycles and aid in customer retention. Even if a customer doesn’t immediately make an additional purchase, a strong recommender that hits the mark can plant seeds that drive future sales. It can also boost affinity for the brand, showing items that align with a consumers’ tastes and interests.

In manufacturing, inference helps companies catch production errors and even spot potential equipment breakdowns before they put people at risk. AI-powered industrial inspections recognize objects, obstacles and people, performing millisecond-level calculations that reduce downtime. These benefits make AI vision systems a top priority for any company working in a complex production environment.

Call centers use inference to automate customer service and quickly route customer issues to those most able to help. When someone needs assistance from an airline, bank, or internet service provider, they often want to talk to a human as quickly as possible. Amid the challenges of today’s broadening labor shortages, AI helps solve simple problems and ensure that customers are quickly connected to the right people who can address their more complex issues.

Work Smarter with Pretrained Models to Grow Your Team

Knowing how inference can help is only the beginning of the AI journey. The next steps are developing a strategy and executing the plan. The trouble is, companies are challenged to find top talent to fill roles of all kinds. For an enterprise just starting with AI, it can be even more daunting to build out a team of AI development experts.

Enterprises can overcome the scarcity of talent by working smarter and leveraging third-party and open source pretrained models and frameworks to gain a head start. These resources significantly lighten the load for teams working to deploy enterprise-ready AI because the developers can adapt and customize existing models to run inference, rather than trying to build from scratch.

Companies can also nurture their existing engineers and developers through AI training. Increasingly, partner companies provide free development labs for enterprises, with step-by-step guides on key AI use cases, including building chatbots for customer service or sales support, image classification systems for safety, price prediction models for better operations and many other foundational AI use cases.

IT at the Helm of Production AI

Once the foundation is in place to move forward with inference work, CIOs should adopt supported software for a production application – whether it’s running on bare-metal, a virtualized data center infrastructure, or in the cloud.

Enterprise-ready AI software that’s fully supported not only for inference but also for the complementary practices of data science and model development also should be considered because it simplifies AI deployments. Teams can rely on a comprehensive solution rather than having to develop unique workflows as AI expands from initial deployment into new areas of the business.

AI workloads are different from traditional enterprise applications, but it’s easier than ever to learn from experts to make sure they’re implemented correctly. Understanding the tools available for efficient, cost-effective enterprise AI inference with pre-trained models, professional development labs and enterprise-grade support can ensure CIOs are prepared with a plan to solve the challenges faced by every enterprise embarking on an AI journey.

John Fanelli is vice president of the Nvidia Enterprise Software product group, which includes responsibility for product management, product marketing and virtualization alliances. An industry veteran, John has led marketing teams at DataTorrent, Citrix, LeftHand Networks, Wind River and Sun Microsystems. He holds an MBA from the J.L. Kellogg Graduate School of Management at Northwestern University, a master’s degree in computer science from the Illinois Institute of Technology and a bachelor’s degree in computer science from the University of Michigan.