Databricks Will Soon Make Lakeflow Designer Available in Private Preview
At its Data + AI Summit in San Francisco, Databricks launches Lakeflow Designer. This visual tool allows non-technical users to build production-ready data pipelines without coding. Lakeflow Designer uses an AI assistant and integrates with the existing Lakeflow solution, which is now generally available.
No-Code Pipelines without Compromising Control and Scalability
Lakeflow Designer offers a drag-and-drop interface that allows business users to design ETL pipelines. They can automate tasks using natural language commands without the help of data engineers. The tool is integrated with Unity Catalog for data management and with Databricks Assistant for AI support. This makes it possible to create visual workflows that offer the same reliability and scalability as code-based alternatives.
The introduction of Lakeflow Designer aims to address a common problem in data teams: on one hand, there are tools that offer simplicity but are limited in terms of control and scalability; on the other hand, robust pipelines usually require the intervention of technical profiles. With Lakeflow, Databricks attempts to combine both worlds.
Extensive Innovations within the Lakeflow Platform
In addition to Lakeflow Designer, Databricks also announces the general availability of Lakeflow. This platform unifies data processing, transformation, and orchestration in one environment. Within Lakeflow, there are new pipelines that can be written entirely in SQL or Python without infrastructure management. There’s also a new development environment that offers AI-supported functionality, such as debugging and validation.
Furthermore, Databricks launches new point-and-click connectors for data sources such as Google Analytics, ServiceNow, SQL Server, and SharePoint. Through the Zerobus functionality, users can write high volumes of event data to their data lake in real-time without additional infrastructure.
Databricks is also starting a public preview of the new development environment for data engineering. The Zerobus feature and Lakeflow Designer itself will soon be available in private preview.