Why Snowflake Is So Great for Data-Heavy Applications

More than 90% of businesses rely on the cloud to a greater or lesser extent in order to function. That reliance is increasing steadily with every passing year.

According to Osterman Research, the world creates 2.5 quintillion bytes of data every day. That’s 2.5 raised to the power of 18, or 2.5 billion gigabytes, if that is any easier to conceptualise.

The point is most of this data needs to go somewhere. Whether it is required for business reasons, such as transaction details, or for analytics purposes, the majority of data is retained, at least for a certain period of time.

The role that the cloud plays in our day-to-day management and use of data has become increasingly significant, but also more varied. Four out of five businesses use multiple public or private cloud apps to fulfil different purposes. When businesses first started using the cloud it was as a repository for data lakes - somewhere to store unsorted data in its native format.

Data on the cloud

It was a little like those dusty and disorganised off-site document archives that businesses used to store boxes upon boxes of hard copy data. From job files to previous years’ accounts to sundry management reports, everything was in there somewhere if you knew where to search and didn’t mind getting a bit dusty in the process.

Data warehousing

Data warehousing takes the concept of this data lake to the next level and underpins Snowflake’s SaaS offering. But before we get into the nuts and bolts of it, we first need to be clear about what data warehousing is.

A data warehouse takes all that information that was swilling around in the lake and preserves it in an integrated and organised way.

This allows it to be easily accessed and analysed.
The data might come from disparate sources such as CRM systems, OLTP databases and so on, but it is processed and stored in such a way that users can access it to run reports and gather information. This will typically be through SQL queries.

Operating an effective data warehouse brings a range of challenges, largely depending on the type of data it receives. Highly structured data from Excel spreadsheets and so on is simple enough to handle, but warehouses also have to deal with unstructured data ranging from PDFs to photographs to audio files.

This is the universe in which Snowflake operates. Next, let’s look at why Snowflake is such a popular choice and how it eclipses the competition.

Why Snowflake is different?

Data warehousing is nothing new and choosing the right technology stack for web development is always a case of playing off different options against one another. However, there are several reasons why Snowflake is attracting so much attention.

Unlike other data warehouses, Snowflake’s SaaS solution is natively designed to run on the public cloud. What’s more, it is what is known as cloud-agnostic, meaning it can operate across any cloud infrastructure.
The developers originally built it on Amazon Web Services (AWS), but Snowflake can also run on Microsoft Azure or Google Cloud.

Snowflake’s unique architecture

The key to why Snowflake is better than other data warehouse software lies in its unique 3-layer architecture. This is comprised as follows:

• Snowflake architecture
• Centralised Database Storage

Sometimes referred to as the storage layer, Snowflake stores data in Hybrid Columnar Storage. Unlike the conventional sort of data storage in rows and columns, Snowflake compresses data and stores it in blocks. This allows for much faster data retrieval during query processing.

Query Processing

This layer is where all the hard work is done. It consists of virtual warehouses that perform processing tasks on different queries. The ability to perform parallel processing on large amounts of data is what makes the difference between Snowflake and alternative solutions. Even when you have petabytes of data, when it is separated into multi-clusters, the data can be staged to speed up query processing.

This process also means users can scale up and down according to their needs - providing Snowflake’s unique “pay as you use” facility, which we will talk about more in just a moment.

Cloud Services
Finally, the Cloud Services layer ties everything together and is sometimes referred to as the “brain” of the system. This is where authentication and access are controlled and everyday administrative tasks are automated.

Why Snowflake is so popular?
Clever caching
Intertwined with Snowflake’s architecture is a three-level system of caching data that is retrieved through queries. The three levels are as follows:

Result Cache - here, Snowflake stores the results of all queries executed over the past 24 hour period. These results can be retrieved by any user of the same virtual warehouse. So if multiple people ask the same question, the response will be ready - assuming the underlying data has not changed since the query was initially made.
Local Disk Cache - the local disk is used as a cache for SQL queries. Data that is required for a specific query is retrieved from the Remote Disk, and cached in SSD and / or memory as appropriate.
Remote Disk - this is effectively the long term storage repository. This level is focused on ensuring data integrity and resilience, even if an entire data centre was to go down for some reason.
User friendly
Snowflake is easy to use. The interface is intuitive and everything is SQL based, so you don’t have to worry about learning new syntax. Even if you have little or no experience in SQL, the learning curve is not a steep one and even users with no coding experience can be up and running with Snowflake in next to no time.

Fully automated
The serverless nature of Snowflake means there is practically nothing you need to do in terms of systems maintenance or management. No installation, no updates, no scaling, it all happens behind the scenes in the Cloud. So all you need to do is jump in and get on with using the software to interrogate your data and start deriving insights.

Highly compatible
Whatever your favourite BI tools might be, you can almost certainly use them with Snowflake. The software is compatible with Einstein Analytics, Tableau, Looker and more. It can also support most popular programming languages, such as Python, C, Java and so on.

Awesome performance
Thanks to the unique architecture mentioned earlier, Snowflake can handle a practically limitless number of workloads concurrently with no impact on performance. This means it can also deal with multiple queries coming in from different directions, or indeed from the same point of origin - single users can throw multiple queries at Snowflake with no dip in performance.

Auto scaling
Snowflake offers auto-scaling and auto-suspend functionality that you will not find with alternative software. This has the effect of starting and stopping warehouses so they are only active when needed. Snowflake automatically undertakes vertical as well as horizontal scaling. The latter means adding more cluster nodes, while vertical scaling involves upgrading CPUs to add more processing power to existing warehouses. All of this happens automatically behind the scenes, so users do not even have to think about it.

Read also:
AWS Databases User Guides: How to Choose the Right One.
Future-proof flexible
The combination of Snowflake’s analytical functionality, query services and data lake gives businesses complete flexibility to use these different functions concurrently. The secure presence of the data lake gives you the comfort of knowing that even as new software might emerge on the scene, you have all the data right there at your fingertips to leverage it ahead of the competition. Nothing can be guaranteed completely future-proof, but this comes very close!

Competitive pricing

Businesses large and small need to pay close attention to pricing, especially in these challenging times. Most data warehouses charge a set monthly fee on a rolling scale. Not so with Snowflake, where you only ever pay for what you use.

The system uses on-demand pricing, meaning monthly bills are calculated according to how much data you store and how much analysis you perform. Note that you can set the warehouse to auto-stop after a certain period of time so you are not paying for it standing idle while the user has gone for lunch.

Snowflake in action

Now we have had a glimpse of why Snowflake is so popular, it is worth looking at some real-world examples of customers who use Snowflake to help them manage, understand and leverage vast amounts of customer data.

Jet Blue

As a business that prides itself as being one of the most technologically innovative in its industry, Jet Blue was excited about bringing in Snowflake to help it in two core areas: finding cost efficiencies and improving the customer experience.

Snowflake allowed Jet Blue’s analysts to access more relevant data, and most importantly of all, to see it in real time. This has allowed the airline to construct more accurate fuel analyses when combining data with external sources such as weather forecasts.

Yet even that is just the beginning. Jet Blue uses Snowflake to organise and make available data reports in a matter of minutes that would have taken weeks using legacy systems. For this user, the sky really is the limit.

Sainsbury's

The supermarket space has never been more competitive in the UK. In these difficult economic times, the big name brands like Sainsbury’s and Tesco are under more pressure than ever from budget alternatives like Aldi and Lidl.

Sainsbury’s works with hundreds of product lines, from groceries to home electronics to clothing, and each of these faces competition from diverse angles.

Snowflake’s data cloud allowed the supermarket to launch a product-matching price comparison service for customers that lines Sainsbury’s products up against those of its competitors.

It is a brave step from a brand that has traditionally focused more on quality and brand loyalty than cost. It is also one that has proved popular with customers, and helped to strengthen that loyalty.

Square

This B2B Fintech business has expanded from simply providing the means for businesses to take credit card payments to a range of marketing, operational, banking and risk management solutions.

We know first hand through fintech app development services that the sheer volume of transaction data can be immense. Square is no exception to this rule.

The company uses Snowflake to store and analyse more than one petabyte of data. This analysis takes a variety of forms, from helping identify new product opportunities to fraud prevention. Square works with data that spans multiple public clouds as if it was all in one place.

Snowflake is also core to Square’s vision for the future. Square’s Data Foundations Manager, Joir-Dan Gumbs says:

The business is striving towards a self-service platform where clients can perform their own analyses. Adding more user privileges within Snowflake is a core step in achieving that vision.

Kraft Heinz

Kraft Heinz is one of the world’s most traditional brands. However, it has gone through a dramatic digital transformation over the past year or two to increase the speed of innovation and to react faster to an ever-changing marketplace.

Mani Gopalakrishnan, the company’s VP of Digital Transformation, explained that data has gone from being a closely guarded secret to something that is part of the service offering. He said:

“Having information is powerful, for sure, but sharing information can be even more powerful.”
The company decided to replace its in-house data platform and upgrade to Snowflake Data Cloud. It chose Snowflake because it needed a platform that could scale as necessary and provide superior speed and performance.

Snowflake has helped guide Kraft Heinz through a transformation that has been cultural as well as technological.

FAQs

Why is snowflake so popular?

Snowflake delivers a platform that is fast, flexible, and user-friendly. It provides the means for not only data storage, but also processing and analysis. It is considered cloud-agnostic, as it operates across Amazon Web Services (AWS), Microsoft Azure or Google Cloud.

What is the purpose of snowflake?

Snowflake provides a means for the storage, processing, and analysis of large amounts of data. By using cloud technology, it allows users to perform multiple tasks faster than traditional big data platforms built on conventional databases.

Why is Snowflake so valuable?

Snowflake’s market cap often raises eyebrows at first glance. The company’s value is due to a combination of factors including its incredibly rapid growth rate, its 160 percent retention rate and a Net Promoter Score in the 70s - that’s even higher than Apple!

Why is Snowflake better for data warehouse?

Snowflake’s unique architecture allows independent storage and compute to scale, so customers can use and pay for storage and computation separately and as needed. Furthermore, this sharing functionality facilitates secure real-time data sharing.

Why is Snowflake better than other databases?

Again, it all comes down to architecture. Snowflake’s unique and patented architecture means it is better equipped to handle every aspects of data and analytics. In short, it is faster, simpler more flexible and more affordable than other database solutions.