Advanced Computing in the Age of AI | Wednesday, December 6, 2023

Snowflake Builds Data Warehouse in the Cloud 

At a time when many developers decry the need for a data warehouse, startup Snowflake Computing this week revealed general availability for its Elastic Data Warehouse, a solution designed for the cloud.

Headed by CEO Bob Muglia, former president of Microsoft's server and tools business in partnership with ex-Oracle executives, among others, Snowflake's data warehouse is designed to combine the cloud's advantages – flexibility, speed, and elasticity – with new capabilities in big data and data warehouses, the developer said.

Organizations tried to succeed without data warehouses but the surge in information – especially the unstructured, machine-generated data already wending its way from the Internet of Things (IoT) – creates havoc for many enterprises that lack, among other things, adequate human resources, Muglia said in an interview.

BobMuglia"The majority of things we've seen customers do with Hadoop is struggle with this machine-generated data. Because Hadoop requires data scientists and specialized people to run it, the data is not available to business users to run. What they do is, after doing all this work in Hadoop, is put it in a relational data warehouse," he said. "It's an incredibly complicated process that doesn't allow people to get what they want. They lose data in the process."

Rather than forcing new technologies to tackle old approaches, Snowflake wanted to reinvent data warehousing using today's technologies, Jon Bock, vice president of sales and marketing, told Enterprise Technology.

"Where would you start [to build] a data warehouse today? It became very obvious to us that you'd start with the cloud," he said. "If you're a CIO or CTO without a cloud strategy, basically you're a CIO or CTO with an expiration date."

Snowflake's Elastic Data Warehouse is a SaaS- and SQL-based solution that taps Amazon's Web Services cloud to lower price and eliminates Snowflake's need to provide customers with the same services. The company also takes advantage of Amazon S3, further lowering customers' costs and increasing simplicity, and expanding the availability of data warehouses to companies that might otherwise not be able to afford them.

Organizations need only import their data, not always a simple task, and query this information, said Maglia.

"Snowflake itself doesn't require system integration but getting the data into Snowflake can require help, depending on the environment," he said.

In addition to general availability, new capabilities include: Native support for traditional, structured data and machine-generated semi-structured data in one database engine; multi-dimensional elasticity to quickly scale workloads and users up or down; self-tuning service to manage and tune data distribution, data storage, metadata, and query execution, and high data and service availability via a fully distributed design. The solution's Time Travel feature includes secure authentication, granular access control, and industrial-grade encryption for data in-flight and on-disk, according to Snowflake.

The data warehouse sees millions of queries per week, Muglia. It encounters few problems and those are fixed fast due to the cloud-based, SaaS structure, he said.

"Because we're a service, we resolve those issues in an incredibly short period of time," said Muglia. "There isn't a single line of code base that is older than three years old. For that we are incredibly mature and incredibly stable."




About the author: Alison Diana

Managing editor of Enterprise Technology. I've been covering tech and business for many years, for publications such as InformationWeek, Baseline Magazine, and Florida Today. A native Brit and longtime Yankees fan, I live with my husband, daughter, and two cats on the Space Coast in Florida.