With Datastream, Snowflake introduces a new service to bring real-time data directly to its own platform without external connections.
Snowflake is launching Datastream at its Summit in San Francisco. With Datastream, Snowflake aims to give users the tools to link external data to the Snowflake environment in real time. This establishes a foundation on which organizations can deploy AI agents, which can then work with that up-to-date data.
Kafka-compatible
Datastream is built natively on Snowflake and is compatible with Kafka. This allows organizations to link existing streaming applications and data flows without reworking their environment. Snowflake combines Kafka compatibility for streaming ingestion with the management and security of its own platform.
“Datastream is compatible with the Kafka Wire Protocol, but does not use Kafka under the hood,” Kleinerman clarifies. “The technology is pure Snowflake and is deeply integrated, but with great care and attention to compatibility.”
Teams can thus stream data directly to Snowflake tables. They no longer need to maintain a separate broker or streaming infrastructure alongside their data platform. Snowflake states that streaming data automatically receives the same governance, access controls, masking, and lineage as other data on the platform.
Basis for Coco
Snowflake is immediately linking the launch of Datastream to AI assistant CoCo. While Datastream is responsible for the continuous supply of up-to-date data, CoCo helps developers build applications on top of it.
With a new integration, developers can set up and operationalize real-time pipelines within Snowflake using simple prompts. Datastream also opens the door to AI applications that work directly on live streaming data. Snowflake is thus positioning the combination of Datastream and CoCo as a foundation for real-time AI with less management overhead.
Opportunity
With Datastream, Snowflake wants to make building real-time AI easier. The service aims to bring fragmented data together, make separate streaming infrastructure redundant, and keep governance and security centralized at the same time.
This should be particularly relevant for organizations that want to keep AI agents and other AI apps up-to-date and accurate. With CoCo, Snowflake also immediately provides the tools to get started with this in a concrete way.
According to Snowflake, the opportunity is enormous. The market for streaming analytics is expected to grow to 128 billion dollars by 2030. Snowflake Datastream will soon be available in a limited capacity as a private preview.