Snowflake Architecture
Snowflake's architecture is the engine that makes everything possible — the fast queries, the ability to run thousands of users at once, the automatic scaling, and the data sharing without copying. Understanding this architecture gives you a mental model that helps you make better decisions about performance, cost, and design throughout your career with Snowflake.
Most database systems bundle storage and compute together inside one machine or one cluster. Snowflake designed its system from scratch to keep those two concerns completely separate. That single design decision is what makes Snowflake different from almost everything that came before it.
The Three-Layer Architecture: A City Analogy
Think of Snowflake as a modern city. The city has three distinct systems that work independently but talk to each other when needed.
- Layer 1 — Cloud Services (City Hall): City Hall manages rules, records, and coordination. It knows who lives where, what permits exist, and what rules apply. In Snowflake, Cloud Services manages metadata, security, query planning, and authentication.
- Layer 2 — Query Processing / Virtual Warehouses (The Factories): Factories do the actual work of producing goods. Workers operate machines and process raw materials. In Snowflake, Virtual Warehouses are the compute engines that run your SQL queries. You can have many factories (warehouses) running at the same time, each doing different jobs.
- Layer 3 — Database Storage (The Warehouse District): The warehouse district stores all physical goods. Factories pull materials from here when needed and return finished products here. In Snowflake, the Storage layer holds all your data in compressed, columnar files on cloud object storage (Amazon S3, Azure Blob Storage, or Google Cloud Storage).
SNOWFLAKE ARCHITECTURE DIAGRAM
================================
+--------------------------------------------------+
| LAYER 1: CLOUD SERVICES |
| Authentication | Query Optimizer | Metadata |
| Access Control | Transaction Mgmt | Encryption |
+--------------------------------------------------+
| | |
v v v
+----------+ +----------+ +----------+
| LAYER 2 | | LAYER 2 | | LAYER 2 |
| Virtual | | Virtual | | Virtual |
| Warehouse| | Warehouse| | Warehouse|
| (Team A) | | (Team B) | | (Team C) |
+----------+ +----------+ +----------+
\ | /
\ | /
v v v
+--------------------------------------------------+
| LAYER 3: CENTRALIZED STORAGE |
| Micro-Partitions (compressed columnar files) |
| Stored on S3 / Azure Blob / GCS |
+--------------------------------------------------+
The most important thing to notice in this diagram is that all virtual warehouses share the same single storage layer. They do not each hold their own copy of the data. This is why three teams can run completely different queries on the same dataset at the same time without slowing each other down — they each have their own compute, but they all read from the same storage.
Layer 1: Cloud Services — The Brain of Snowflake
Every request you make to Snowflake — logging in, submitting a query, creating a table — passes through the Cloud Services layer first. This layer never touches your actual data. It only manages the rules and the roadmap for how data is handled.
What Cloud Services Manages
- Authentication: Verifies your username, password, and multi-factor authentication tokens before granting access
- Access Control: Checks whether your role has permission to read from a specific table or schema
- Query Parsing and Optimisation: Takes your SQL statement, checks it for syntax errors, figures out the most efficient execution plan, and instructs the virtual warehouse on how to run it
- Metadata Management: Tracks exactly where every piece of data lives across all micro-partitions — which files contain which rows, what the min/max values are in each column, and how many rows exist in each partition
- Transaction Management: Ensures that two users editing the same data at the same time do not corrupt each other's work
- Infrastructure Management: Starts, stops, and resizes virtual warehouses automatically based on your settings
Cloud Services runs 24 hours a day at no extra charge as part of your Snowflake subscription. Snowflake does not charge credits for Cloud Services activity unless it exceeds 10% of your daily compute usage — which is rare in normal operation.
Layer 2: Virtual Warehouses — The Compute Engines
A Virtual Warehouse (often called just a "warehouse") is a cluster of cloud servers that executes your SQL queries. When you run a SELECT statement, it is the warehouse doing the actual reading and calculating. When you run an INSERT or UPDATE, the warehouse writes results back to storage.
How a Virtual Warehouse Processes a Query
YOU SUBMIT: SELECT SUM(sales) FROM orders WHERE year = 2024;
STEP 1: Cloud Services receives the query
- Authenticates your session
- Checks your permissions on the "orders" table
- Parses the SQL for correctness
STEP 2: Cloud Services plans the query
- Looks up metadata: which micro-partitions contain year=2024 rows?
- Prunes partitions that definitely contain no matching rows
- Sends the execution plan to your virtual warehouse
STEP 3: Virtual Warehouse executes
- Pulls only the needed micro-partition files from storage
- Decompresses and scans only the "sales" and "year" columns
- Computes the SUM across all matching rows
- Returns the result to Cloud Services
STEP 4: Cloud Services returns result to you
- Result displays in Snowsight or your connected tool
- Query metadata saved to query history
Virtual Warehouses come in eight sizes from X-Small to 6X-Large. Each doubling in size roughly doubles the speed for large queries, while also doubling the credit cost per hour. Choosing the right size is a performance-versus-cost decision you will learn to make based on your workload.
Multi-Cluster Warehouses
A standard warehouse has one cluster. A multi-cluster warehouse can automatically add more clusters when many users query simultaneously. Imagine a checkout line at a supermarket: when more customers arrive, the store opens more checkout lanes. Multi-cluster warehouses open extra compute clusters when your user demand spikes and close them when traffic drops — all automatically.
SINGLE-CLUSTER WAREHOUSE (Standard Edition) -------------------------------------------- 100 concurrent users --> all share one cluster --> some queries queue and wait MULTI-CLUSTER WAREHOUSE (Enterprise Edition) --------------------------------------------- 100 concurrent users --> Snowflake adds clusters automatically Cluster 1 handles users 1-30 Cluster 2 handles users 31-60 Cluster 3 handles users 61-100 When traffic drops, Clusters 2 and 3 shut down automatically
Layer 3: Database Storage — Where Your Data Lives
Snowflake stores all your data on the cloud provider's object storage — Amazon S3 if you chose AWS, Azure Blob Storage for Microsoft Azure, or Google Cloud Storage for GCP. You never interact with these storage services directly. Snowflake manages everything through its internal file format.
Micro-Partitions: The Storage Secret
When you load data into a Snowflake table, Snowflake automatically splits it into small compressed files called micro-partitions. Each micro-partition is typically 50 to 500 megabytes of uncompressed data. Snowflake stores these files in a columnar format — meaning all values from one column are stored together, rather than storing each row together.
ROW-BASED STORAGE (Traditional DB) COLUMNAR STORAGE (Snowflake)
---------------------------------- ---------------------------
File 1: Column File: CUSTOMER_ID
Row 1: [C001, Alice, 500, 2024] [C001, C002, C003, ...]
Row 2: [C002, Bob, 320, 2024]
Row 3: [C003, Carol, 780, 2023] Column File: AMOUNT
[500, 320, 780, ...]
To find SUM(AMOUNT): To find SUM(AMOUNT):
Read entire File 1 (all columns) Read ONLY the AMOUNT file
Ignore CUSTOMER_ID, NAME, YEAR Skip all other columns entirely
--> SLOW, reads unnecessary data --> FAST, reads only what's needed
Columnar storage is one of the main reasons Snowflake queries on large datasets are dramatically faster than row-based databases. A query that reads only two columns of a 100-column table reads 98% less data from disk.
Automatic Compression
Snowflake automatically compresses every micro-partition using algorithms suited to the data type in each column. Numeric columns compress differently than text columns. The result is that data in Snowflake typically takes 2x to 7x less storage space than the original uncompressed files. You never configure compression manually — Snowflake chooses the optimal algorithm automatically.
Metadata and Partition Pruning
Snowflake records the minimum value, maximum value, count, and null count of every column inside every micro-partition. The Cloud Services layer uses this metadata to skip entire partitions that cannot possibly contain rows matching your WHERE clause. This process is called partition pruning.
TABLE: SALES with 1,000 micro-partitions
YOUR QUERY: SELECT * FROM sales WHERE order_date = '2024-06-15'
METADATA CHECK:
Partition 1: order_date MIN=2022-01-01, MAX=2022-12-31 --> SKIP
Partition 2: order_date MIN=2023-01-01, MAX=2023-12-31 --> SKIP
...
Partition 847: order_date MIN=2024-06-10, MAX=2024-06-20 --> SCAN
Partition 848: order_date MIN=2024-06-14, MAX=2024-06-16 --> SCAN
...
Partitions 849-1000: order_date MIN=2024-07-01, MAX=... --> SKIP
RESULT: Only 2 partitions read out of 1,000
Query runs 500x faster than scanning everything
Why Separation of Layers Matters for You
The three-layer separation solves real business problems that organisations face every day.
Multiple teams, no conflict: Your sales team runs a heavy aggregation report on a Large warehouse. Your data engineering team loads millions of new rows using a Medium warehouse. Your executives run lightweight dashboard queries on an X-Small warehouse. All three happen simultaneously, reading the same data, without any team blocking or slowing another.
Pay only for what you use: Your storage bill is fixed each month based on how much data you store. Your compute bill only grows when warehouses are actively running. A warehouse suspended for 20 hours of a 24-hour day uses zero credits for those 20 hours.
Instant elasticity: When your business runs a quarter-end report that normally takes four hours on your current warehouse, you resize the warehouse to a larger size, run the report in 30 minutes, then resize back down. The cost is lower because you paid for a big warehouse for only 30 minutes instead of a small warehouse for four hours.
How the Three Layers Communicate
The three layers communicate over an internal high-speed network managed by Snowflake. When a virtual warehouse needs to read data from storage, it does so over this network. The latency is low because Snowflake keeps the warehouse and storage in the same cloud region. Moving data across regions or across cloud providers adds latency and network cost, which is why choosing the right region at account setup matters.
Key Points
- Snowflake's architecture has three distinct layers: Cloud Services (brain), Virtual Warehouses (compute), and Storage (data)
- Separating storage from compute lets multiple teams run independent queries on the same data at the same time without conflict
- Storage uses micro-partitions: small, compressed, columnar files that enable fast scans and automatic partition pruning
- Cloud Services handles authentication, query planning, metadata, and transaction management at no extra compute cost
- Virtual Warehouses scale up (bigger size) for faster single queries and scale out (multi-cluster) for more concurrent users
- You pay for compute only while a warehouse is running — suspending it immediately stops the credit consumption