Data warehouses and lakes will merge

by | Oct 21, 2022 | Technology

Register now for your free virtual pass to the Low-Code/No-Code Summit this November 9. Hear from executives from Service Now, Credit Karma, Stitch Fix, Appian, and more. Learn more.

My first prediction relates to the foundation of modern data systems: the storage layer. For decades, data warehouses and lakes have enabled companies to store (and sometimes process) large volumes of operational and analytical data. While a warehouse stores data in a structured state, via schemas and tables, lakes primarily store unstructured data. 

However, as technologies mature and companies seek to “win” the data storage wars, companies like AWS, Snowflake, Google and Databricks are developing solutions that marry the best of both worlds, blurring the boundaries between data warehouse and data lake architectures. Additionally, more and more businesses are adopting both warehouses and lakes — either as one solution or a patchwork of several. 

Primarily to keep up with the competition, major warehouse and lake providers are developing new functionalities that bring either solution closer to parity with the other. While data warehouse software expands to cover data science and machine learning use cases, lake companies are building out tooling to help data teams make more sense out of raw data. 

But what does this mean for data quality? In our opinion, this convergence of technologies is ultimately good news. Kind of. 

Low-Code/No-Code Summit
Join today’s leading executives at the Low-Code/No-Code Summit virtually on November 9. Register for your free pass today.

Register Here

On the one hand, a way to better operationalize data with fewer tools means there are — in theory — fewer opportunities for data to break in production. The lakehouse demands greater standardization …

Article Attribution | Read More at Article Source

Share This