Advanced Computing in the Age of AI | Friday, June 9, 2023

ServiceNow and Nvidia Partner to Develop Custom Enterprise AI Models 

Nvidia has announced a partnership with ServiceNow to develop domain-specific generative AI models for the enterprise.

ServiceNow is a SaaS provider focused on IT service management tools. The company is developing custom large language models trained on data specifically for its ServiceNow Platform, a cloud-based workflow automation platform. ServiceNow will use Nvidia’s NeMo foundation models as a starting point and will train its fine-tuned models on Nvidia GPUs, starting with models for the IT domain.

Rama Akkiraju, Nvidia’s VP of AI/ML for IT, explained at a press briefing how generative AI is transforming business. Text generation and summarization, real-time language translation, coding assistance, prompt-based image generation, and drug discovery are promising use cases for generative AI in the enterprise. Akkiraju noted how business leaders are actively building proofs-of-concept to discover how the capabilities of these technologies apply to their own use cases. A recent Accenture report found that 98% of global executives agree that AI foundational models will play an important role in their organization's strategies in the next three to five years.

With their tendency to hallucinate incorrect information, foundation models like GPT-4 that have been trained with data from the public domain are not necessarily ready for enterprise use cases, specifically those requiring a high level of accuracy.

“We know that generative AI models are good at learning from public domain data sources to do multiple tasks like text summarization and translation, recording assistance and image generation, etc. But they don't specifically know about the data in an enterprise because they haven't seen it,” said Akkiraju.

Domain-specific enterprise data can fine-tune these foundational models to customize them for specific industries. These customized models can then be deployed and hosted and when used in production, businesses can continuously apply guardrails to ensure they are used safely and effectively.

“To bring generative AI to enterprises, we must customize these models, the foundational models, to teach them the language of the enterprise and enterprise-specific skills, so that they can provide more in-domain responses with proper guardrails,” Akkiraju said.

Akkiraju gives the example of a new hire looking to connect to a company VPN: “If I ask a generative AI model how to connect to VPN as a new employee, it shouldn't be answering it based on some public domain knowledge of some other company. It should in fact be giving me an answer based on the knowledge articles or the information that’s available within the intranet of the company and be very specific to the policies that are given for connecting to a VPN.”

This is one of the multiple use cases for generative AI capabilities in the IT domain including summarizing help desk and IT tickets and enterprise search capabilities to help IT professionals with finding in-domain knowledge. AI can also help detect, predict, and mitigate IT service outages. Auto resolution is another avenue, as chatbots and question-and-answer systems can help users resolve issues themselves without waiting in line.

NeMo offers prompt tuning, supervised fine-tuning, and knowledge retrieval tools. (Source: Nvidia)

As part of this partnership, ServiceNow is also streamlining Nvidia’s IT operations using Nvidia data to customize Nvidia NeMo foundation models running on hybrid cloud infrastructure—specifically, Nvidia DGX Cloud and its on-prem DGX SuperPOD AI supercomputers. ServiceNow is using Nvidia’s AI Foundations cloud services along with the Nvidia AI Enterprise software platform that includes the Nvidia NeMo framework. NeMo offers prompt tuning, supervised fine-tuning, and knowledge retrieval tools, as well as safety and security features as part of its NeMo Guardrails software.

One of the first use cases for this collaborative effort is IT ticket summarization. Current help desk summarizing takes about seven to eight minutes for each agent, estimates Akkiraju. Generative AI can save precious time for busy service agents by automating these summaries and adding to the company’s knowledge base.

The two companies also anticipate using this technology to improve the employee experience by identifying growth opportunities such as customized learning and development recommendations based on their natural language queries.

“IT is the nervous system of every modern enterprise in every industry,” said Jensen Huang, founder and CEO of Nvidia, in a release. “Our collaboration to build super-specialized generative AI for enterprises will boost the capability and productivity of IT professionals worldwide using the ServiceNow platform.”

“As adoption of generative AI continues to accelerate, organizations are turning to trusted vendors with battle-tested, secure AI capabilities to boost productivity, gain a competitive edge, and keep data and IP secure,” said CJ Desai, president and chief operating officer of ServiceNow. “Together, Nvidia and ServiceNow will help drive new levels of automation to fuel productivity and maximize business impact.”