Google is ushering in a new era of control for publishers over their content through Bard Vertex AI. This fresh initiative enables publishers to precisely calibrate the AI’s access to their content, fostering a more customised and advantageous engagement between publishers and the AI.
A leap towards better content management
Bard Vertex AI embodies a forward step towards enhanced content management and control. Publishers now have the reins to determine the level of access and interaction the AI has with their content. This augmented control heralds a positive shift, creating a more friendly relationship between publishers and AI technology.
In a rapidly changing landscape of generative artificial intelligence (AI), Google endeavours to cultivate a balanced ecosystem accommodating web publishers and AI advancements. The unveiling of Google-Extended, a control mechanism, empowers web publishers to decide if or how their sites contribute to the refinement of Google Bard, Vertex AI generative APIs, and prospective AI models. Rooted in the philosophy of responsible AI development, this initiative resonates with Google’s steadfast AI principles and commitment to consumer privacy.
Embracing a future of refined interactions
The advent of Bard Vertex AI reflects a glimpse into a future where polished interactions between publishers and AI technology nourish a more productive digital ecosystem. This manoeuvre underscores Google’s dedication to orchestrating a more equitable and advantageous online milieu for all stakeholders involved.
Google-Extended serves as a “standalone product token” that web publishers can employ to manage how their sites enhance Bard and Vertex AI generative APIs and the AI models driving them. Although Google-Extended lacks a separate HTTP request user agent string, the crawling is conducted with the existing Google user agent strings, utilising the robots.txt user-agent token in a control capacity.
For instance, within your robots.txt file, you could specify:
User-agent: Google-Extended
Disallow: /paywall-content/
Allow: /
Here, User-agent: Google-Extended indicates that the subsequent rules apply to Google-Extended. Disallow: /paywall-content/ instructs Google-Extended to abstain from accessing or utilising the content in the “paywall-content” directory to refine Bard and Vertex AI generative APIs. Allow: / authorises Google-Extended to access and employ content from all other site directories to enhance future AI products.
Managing AI access to website content
As AI applications burgeon across all sectors, the maze of managing diverse companies’ access to content for AI training data is a reality that web publishers must navigate. Google has resolved to liaise with web and AI communities, exploring more machine-readable channels to bestow choice and control upon web publishers.
Publishers seeking more information can fill out a form to join Google’s AI Web Publisher Controls Mailing List for upcoming updates. Furthermore, publishers wishing to restrict content usage in future OpenAI models should also contemplate the GPTbot for limiting or denying access.
