Forrester Report Shows That You Need More Than an LLM to Build a GenAI Application
The rapid rise of generative AI (GenAI) has left businesses scrambling to find new and innovative methods to harness the power of this technology for business applications. Many businesses believe that Large Language Models (LLMs) have reinvented how AI-powered business applications get built, and all that is needed is to input data in one of the LLM models from the big players, and it will get the job done. However, things aren’t that easy.
Forrester, a leading research and advisory company, released a new report that highlights that GenAI business applications require more than a general-purpose LLM. Even the most carefully fine-tuned and prompted LLM might not be enough to build and safely operate a GenAI-powered application. The simplistic approach won’t allow organizations to use all their proprietary knowledge to work. It also has several other risks including issues with scaling, security, and cost.
The Forrester report was compiled by surveying the 15 largest service providers on how they are using GenAI and they help over 2000 companies around the globe to build Gen-AI-powered business applications. The findings of the report show that businesses need to assemble a “layers, gates, and pipes” architecture to safely and effectively operate GenAI-powered applications.
The layers, gates, and pipes architecture draws on resources from many layers of intelligence to bring together internal and external capabilities. It also needs Input and output control gates to protect people, the firm, and the models themselves. In addition, it requires application pipes to prompt, embed, and orchestrate layers of intelligence to translate requests into outputs. Finally, testing and learning loops are needed to test and monitor results and make adjustments accordingly.
Digging deeper into the elements of the layers, gates, and pipes architecture, the report states that the layers of intelligence include a wide range of capabilities including general-purpose, embedded, and specialized GenAI models.
Intelligence resources that organizations should create and manage themselves include software applications, AI/ML models, private GenAI models, structured and unstructured data, and people’s prompts and behaviors. The intelligence sources that organizations should source from suppliers should include domain-specific GenAI models, and public GenAI tools, bundled GenAI models, such as SaaS apps.
The use of input gates helps reject bad requests, bogus prompts, and dangerous searches. It can also turn vague requests into answerable prompts. The output gates help validate the outputs for issues according to the compliance requirements, security, and more.
Application pipes are used to tie it all together through an API-first workflow. They help stitch the resources from the layer of intelligence so they can flow smoothly end-to-end. The last element of the architecture is testing through the feedback loops used for testing. They help in establishing trust, confidence, and validity in the application.
The Forrester report also adds that businesses can assemble applications from parts today as they build a full architecture to support GenAI applications over the next few years. GenAI-powered applications are like vibrating organisms that need constant monitoring and optimization. Only through proper care can businesses fully benefit from the power of GenAI business applications.
Related Items
Will GenAI Take Traditional AI Along for the Ride?
