Confluent has introduced new querying capabilities to its Data Streaming Platform, allowing AI agents, applications and analytics systems to access continuously updated business information. The addition completes the platform’s five core functions and supports what Confluent calls a Streamhouse, a data architecture designed to provide a shared, governed view of business operations.
Central to the update is Lightning Tables, now generally available on AWS through Confluent Cloud. The feature stores Apache Kafka event data in continuously updated tables that applications can query through SQL or an API, a capability Confluent says reduces the need for separate databases, caches and data processing pipelines when applications require access to current operational information.
Giving AI agents a consistent view of business operations
Enterprise information is often distributed across systems that update at different intervals. Confluent argues that conventional data warehouses and lakehouses, which were primarily designed for reporting and batch processing, can leave AI agents working with outdated or conflicting information when making operational decisions. A Streamhouse addresses this by continuously capturing changes across connected systems, processing those events and making the resulting information available to applications and AI agents. It works alongside existing warehouses and lakehouses, providing an operational layer that keeps business context current without requiring organisations to consolidate their data first.
To illustrate the approach, Confluent points to a hypothetical airline handling a cancelled flight. Aircraft movements, departure schedules and seat availability may be managed by separate systems, each reflecting a different point in time. An automated rebooking agent could therefore offer a connection that appears valid based on available seats but is impossible because the aircraft cannot arrive in time.
Under the Streamhouse architecture, updates to aircraft locations, departure times and reservations feed into a continuous stream of events instead. Rebooking agents, customer-service teams and other applications can then access a shared operational picture when making decisions. The example illustrates how the architecture is intended to operate across interconnected workflows.
Confluent, an IBM company, forms part of IBM’s wider data strategy, which brings together real-time streaming, integration, governance and AI capabilities across hybrid environments. The Streamhouse architecture is designed around continuously updated information, production service levels and decentralised access to data across distributed systems.
Lightning Tables brings live data into applications
Query joins Stream, Connect, Process and Govern as Confluent’s fifth platform capability, providing different ways for applications, analytics platforms and AI agents to access information as business events occur. Three technologies support this function. Confluent Tableflow delivers streaming information to warehouses and lakehouses using open table formats for analytics, while the Real-Time Context Engine provides current business information to AI agents through the Model Context Protocol (MCP). Alongside them, Lightning Tables serves applications requiring low-latency access through SQL queries and a direct API.
Lightning Tables operates as a read-only serving layer within Confluent Cloud, maintaining tables derived from Apache Kafka data as new events arrive. According to Confluent’s internal testing, information becomes available for querying approximately 100 milliseconds after creation, with query latency ranging from 10 to 100 milliseconds and throughput exceeding 1,000 queries per second. The service applies governance controls from data capture through serving. Support for schema changes helps applications accommodate changes in incoming data, while integration with Apache Flink allows processed information to be stored for machine learning applications, including real-time personalisation.
The remaining platform functions handle the movement and preparation of data before it reaches the query layer. Stream uses Apache Kafka and Confluent’s Kora engine to capture and distribute events, with Confluent reporting sub-millisecond streaming latency in internal testing, although actual latency varies by configuration and workload. For integration across enterprise infrastructure, Connect provides more than 150 fully managed and custom connectors alongside hybrid connector management.
To combine and enrich information as events occur, Process uses Apache Flink, with additional batch execution capabilities for initialising streaming workloads from existing data. Platform governance is handled by Govern, which provides data discovery, classification, lineage, cataloguing, quality controls, encryption, schema management and access management across the platform.
Accenture and Extend describe operational deployments
Accenture is using a Streamhouse architecture powered by Confluent to connect information across its global operations. Thomas Mahaffey, SAI Chief Data Strategist at Accenture, described the changes to its data operations.
“Operating across more than 120 countries, our global enterprise was previously constrained by legacy batch and point-to-point architectures where duplicative data extractions drove up operational overhead and analytics data lagged. Moving to a Streamhouse architecture powered by Confluent decouples data producers from consumers, transforming high-volume event streams into governed, reusable real-time data products that flow seamlessly across operational and analytical systems. With Confluent handling infrastructure automation and providing seamless enterprise connectivity, our developers operate in a standardised environment without the friction of building custom integrations or micro-managing data movement. This continuous data foundation has dropped our time-to-insight from months to days and slashed reporting times, empowering our teams to focus on delivering high-value AI for our 800,000 employees.”
Extend, an AI-native commerce platform, has also implemented a fully serverless Streamhouse architecture built around Confluent Cloud. Its deployment followed difficulties with traditional message queues and static data lakes, including limitations involving event replay, data persistence and strict schema enforcement at scale.
According to Jonathan Kropp, Director of AI and Architecture at Extend, the architecture uses Kafka as an operational source of information, allowing downstream services to consume and query real-time data asynchronously while maintaining data contracts. Extend identifies merchant onboarding, real-time pricing adjustments and fraud detection as operations requiring rapid access to information, with its streaming architecture also supporting real-time AI agent workflows.
