Tuesday, 2 September 2025
28 C
Singapore
30.8 C
Thailand
20.7 C
Indonesia
27.6 C
Philippines

Elon Musk acknowledges we’ve run out of real-world AI training data

Elon Musk warns that AI training data is running out and suggests synthetic data as the solution while experts weigh its benefits and risks.

Elon Musk has agreed with leading AI experts that the world is running out of real-world data to train artificial intelligence models. During a live-streamed conversation with Stagwell chairman Mark Penn on X (formerly Twitter) on Wednesday night, Musk remarked, “We’ve now exhausted the cumulative sum of human knowledge … in AI training. That happened last year.”

The data drought and its implications

Musk, who owns the AI company xAI, is not the first to raise concerns about the shortage of AI training data. Ilya Sutskever, a former chief scientist at OpenAI, highlighted this issue during a December speech at the NeurIPS machine learning conference. He termed the situation “peak data” and predicted that this shortage would fundamentally change how AI models are developed.

As the pool of real-world data dwindles, researchers are turning to synthetic data—data created by AI models themselves—to fill the gap. Musk echoed this sentiment, explaining, “The only way to supplement [real-world data] is with synthetic data, where the AI creates [training data]. With synthetic data … [AI] will sort of grade itself and go through this process of self-learning.”

Tech giants embrace synthetic data

Several major tech companies, including Microsoft, Meta, OpenAI, and Anthropic, are already using synthetic data to train their flagship AI models. A report from Gartner estimates that by 2024, 60% of the data used in AI and analytics projects will be synthetically generated.

For instance, Microsoft’s recently open-sourced Phi-4 model combines synthetic data with real-world inputs. Similarly, Google’s Gemma models and Meta’s latest Llama series have also benefited from synthetic data in their development. Anthropic used synthetic data in creating its Claude 3.5 Sonnet system. At the same time, AI startup Writer developed its Palmyra X 004 model almost entirely with synthetic data at a fraction of the usual cost.

Synthetic data has clear cost advantages. The writer revealed that its Palmyra X 004 model cost just US$700,000 to develop, compared to the estimated US$4.6 million spent on an OpenAI model of similar size.

The challenges of synthetic data

Despite the advantages, synthetic data is not without its drawbacks. Research has shown that reliance on synthetic data can lead to “model collapse,” where AI systems become less creative and more biased over time. This happens because models generating synthetic data often amplify the biases and limitations present in the original training datasets, which can severely compromise the functionality of AI systems in the long run.

Musk’s comments reflect a growing consensus in the AI industry: synthetic data is the way forward, but it must be used cautiously. The challenge now lies in finding ways to create artificial data that mitigate biases and maintain the creativity and reliability of AI systems.

Hot this week

Keeper Security introduces KeeperAI for real-time cyber threat defence

Keeper Security has launched KeeperAI, an AI-powered tool that detects and stops cyber threats in real time by monitoring privileged sessions.

Yooka-Replaylee remaster launches on consoles and PC on 9 October

Yooka-Replaylee launches on 9 October with digital and physical editions, Switch 2 full cartridge, enhancements, and a 30% upgrade discount.

Chipolo unveils rechargeable Loop and Card Bluetooth trackers

Chipolo launches its first rechargeable Bluetooth trackers, the Loop and Card, offering six-month battery life and IP67 durability.

YouTube TV faces potential loss of Fox channels this week

YouTube TV may lose access to Fox channels this week due to stalled contract negotiations, potentially disrupting coverage of the NFL and college football.

ASEAN battery conference highlights regional leadership and collaboration

The 3rd ASEAN Battery Technology Conference in Phuket showcased new partnerships, safety standards, and innovation for clean energy.

FIFAe partners with Lenovo to power FIFAe Finals 2025 in Riyadh

FIFAe partners with Lenovo to power the FIFAe Finals 2025 in Riyadh, featuring top esports teams competing on Legion gaming devices.

Canon Singapore and NLB expand cartridge recycling programme with new green pledge

Canon Singapore and NLB expand cartridge recycling to more libraries with the Inkfinity Green Pledge, supporting Singapore’s Zero Waste goals.

FairPrice Group opens AI-powered Store of Tomorrow at Punggol Coast Mall

FairPrice Group has opened the world’s first AI-powered supermarket at Punggol Coast Mall, transforming shopping and staff experiences.

Banks urged to balance opportunities and risks of agentic AI

Moneythor warns that while agentic AI offers exponential potential for banks, the risks are just as significant without safeguards.

Related Articles

Popular Categories