Thanks to improved memory and reasoning capabilities, generative AI systems might soon pass Ph.D. exams. Microsoft CTO Kevin Scott shared this exciting development at a Berggruen Salon event in Los Angeles earlier this week.
Enhancing AI memory
Scott discussed the current limitations of AI memory, explaining that interactions with AI agents are entirely episodic. “You have a transaction, and you do a thing. It is useful or not for whatever task you were doing, and then it forgets all about it,” he said. AI systems do not remember previous interactions, making it impossible to refer to past tasks.
However, Scott is optimistic about the future. He believes that technology is progressing towards creating AI systems with durable memories. This advancement will allow AI to respond more naturally and accurately over multiple conversations, not just within a single session.
OpenAI has taken a significant step forward by testing a new persistent memory system. Since February, a select group of free and Plus subscription users have utilised this feature. It allows AI to remember user preferences in tone, voice, and format between conversations and even suggest new project ideas based on past chats.
Improving AI reasoning
Scott also addressed the need to improve the “fragility” of AI reasoning. Current AI systems struggle with complex math problems, often requiring assistance from other systems. “Reasoning, I think, gets a lot better,” Scott said, comparing current models like the GPT-4 to high schoolers passing their AP exams. However, the next generation of AIs could pass advanced exams, such as Ph.D. qualifications.
Generative AI systems have already shown impressive capabilities, outperforming humans in various exams. For instance, in November, GPT-4 passed the Multistate Professional Responsibility Exam (MPRE) with 76% accuracy, six points higher than the national average for humans.
Despite these achievements, Scott cautioned against overestimating the significance of training AIs to pass Ph.D. exams. “The real test will be what we choose to do with it,” he emphasised.
Democratising AI
Scott expressed excitement about the decreasing barriers to entry in AI development. He recalled the challenges of entering the field two decades ago, which required graduate-level knowledge, extensive technical papers, and around six months of coding. Today, a high school student could achieve similar results in a single morning.
This simplification in AI development is set to accelerate the democratisation of AI. Scott underlined that addressing global social, environmental, and technological challenges should not be confined to “just the people at tech companies in Silicon Valley or just people who graduated with Ph.D.s from top-five universities.” He firmly believes that the 8 billion people in the world, each with unique ideas, should have access to powerful tools to contribute to finding solutions.
