Has GPT-4 Ignited the Fuse of Artificial General Intelligence? 

Following the release of GPT-4, Microsoft AI researchers have published a paper suggesting that the OpenAI language model demonstrates “sparks” of artificial general intelligence (AGI).

GPT-4 is OpenAI’s latest large language model, and for six months the researchers had access to its unrestricted version before it was fine-tuned for public release. “Sparks of Artificial General Intelligence: Early experiments with GPT-4” is the result of observations made by Microsoft’s Machine Learning Foundations team while working with GPT-4 to integrate it with the Bing search engine. The researchers clarify that this is only an initial step towards increasingly intelligent systems and highlight that their study is based on this early version of GPT-4.

The concept of AGI, or software that can learn any task or subject, is a hotly debated topic in the computing world and the actual definition is sometimes murky. The AI of the present moment can be described as narrow, as it is made up of machine learning models that are designed to perform a specific function. The idea of generally intelligent software or computers has thus far been relegated to fantasy depictions in movies and television.

The Microsoft researchers argue that GPT-4 exhibits more general intelligence than previous models. They claim that GPT-4 is particularly adept at performing various tasks without specific training and has achieved near-human-level performance in areas like mathematics and coding. For this research, the team defines intelligence based on “the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience,” and they say AGI refers to systems that demonstrate these broad capabilities of intelligence.

The paper’s abstract reveals the authors’ impressions: “We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology, and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4’s performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4’s capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system.”

One interesting observation is that GPT-4 can use external tools like search engines and APIs with “very minimal instruction and no demonstrations.” The model can even use a combination of tools for more complex tasks such as managing both calendar invites and email. In an example, the researchers asked GPT-4 to coordinate a dinner with three people and schedule it on a night when everyone is free. “GPT-4 uses the available APIs to retrieve information about the user’s calendar, coordinate with other people over email, book the dinner, and message the user with the details,” the paper states.

The researchers also tested GPT-4’s image capabilities, comparing its ability to generate a Kandinsky painting in Javascript code with ChatGPT’s output. (Source: Microsoft)

Coding is another area where GPT-4 excels, the authors say. The researchers noted that its skill level enabled it to pass several mock technical interviews on LeetCode, an online programming education platform, and that it could theoretically be hired as an entry-level software developer.

The team acknowledges that GPT-4 still has flaws, such as hallucinations and mathematical struggles. However, the model has shown significant improvements compared to its predecessors, excelling in tasks like the legal Bar exam, LSAT, and Certified Sommelier theory test.

While the researchers admit GPT-4’s intelligence patterns are not human-like, they argue that its capabilities represent a dramatic change in the field of computer science: “Our primary goal in composing this paper is to share our exploration of GPT-4’s capabilities and limitations in support of our assessment that a technological leap has been achieved. We believe that GPT-4’s intelligence signals a true paradigm shift in the field of computer science and beyond.”

Though this paper is an interesting dive into GPT-4’s capabilities, it is important to consider the potential for bias on the part of Microsoft, as it has a $10 billion partnership with OpenAI. In the race for AI supremacy, declaring an early AGI victory, whether true or not, could be an image booster.

Sam Altman, CEO of OpenAI, gave his opinion of GPT-4’s AGI possibilities on the Lex Fridman Podcast that aired on March 25: “I think that GPT-4, although quite impressive, is definitely not an AGI. Still, isn’t it remarkable that we’re having this debate?”

Access the paper at this link, or listen to a related podcast featuring Sébastien Bubeck, the principal researcher for this project, here.