DE Types of Data

Not all data looks the same. A spreadsheet of sales numbers, a photo from a customer's review, and a system log file are all data — but they require entirely different storage systems, processing tools, and engineering approaches. Knowing the different types of data is foundational to every decision a data engineer makes.

The Three Main Categories

Data engineering divides data into three broad types: structured, semi-structured, and unstructured. Each type describes how organized and predictable the data is.

Structured Data

Structured data fits neatly into rows and columns — like a spreadsheet or a relational database table. Every record has the same fields, and every field has a defined data type. A customer table might have columns for customer ID, name, email, and signup date. Every row follows that exact pattern.

Structured data is the easiest to query and analyze. SQL works naturally with it. It powers most traditional business reporting and is the backbone of databases used in banks, hospitals, and retail systems.

Example: Customer Table (Structured)
+----+------------+----------------------+------------+
| ID | Name       | Email                | Signup Date|
+----+------------+----------------------+------------+
| 1  | Sara Patel | sara@example.com     | 2024-01-15 |
| 2  | Tom Osei   | tom@example.com      | 2024-02-03 |
| 3  | Lin Huang  | lin@example.com      | 2024-03-21 |
+----+------------+----------------------+------------+

Semi-Structured Data

Semi-structured data has some organizational structure but does not fit into a rigid table format. JSON and XML are the most common examples. Each record may have different fields, nested objects, or arrays. It carries its own structure within the data itself using tags, keys, and values.

Web APIs almost always return semi-structured JSON data. Log files often use a semi-structured format. Semi-structured data is more flexible than structured data, but harder to query with traditional SQL without additional processing.

Example: Product Data (Semi-Structured JSON)
{
  "product_id": "P101",
  "name": "Wireless Headphones",
  "specs": {
    "battery": "30 hours",
    "color": ["black", "white"],
    "bluetooth": "5.2"
  },
  "reviews": [
    {"user": "Tom", "rating": 5},
    {"user": "Lin", "rating": 4}
  ]
}

Notice that the "specs" object and "reviews" array would not fit into a single flat row. This flexibility is why JSON is popular for APIs — different products can have different attributes without forcing a rigid schema.

Unstructured Data

Unstructured data has no predefined format or organization. Images, videos, audio files, PDF documents, emails, and social media posts are all unstructured. A data engineering system cannot simply query a photo with SQL. Accessing and processing unstructured data requires specialized tools.

Unstructured data makes up the majority of data created in the world. Processing it often involves machine learning — for example, using a computer vision model to extract text from images or a natural language processing model to analyze customer reviews.

A Comparison Diagram

Type              Example              Storage           Query Tool
------------------|--------------------|------------------|------------------
Structured        Database table       Relational DB      SQL
Semi-Structured   JSON file, XML log   Data Lake, NoSQL   SQL + parsing tools
Unstructured      Image, video, email  Object storage     ML models, search

Two More Ways to Classify Data

Beyond structure, data engineers also think about how data moves through time.

Batch Data

Batch data accumulates over a period and gets processed together at a scheduled time. A company that processes all payroll data once per month uses batch processing. Batch systems are simpler to build but introduce a delay between when data is created and when it is available for analysis.

Streaming Data

Streaming data flows continuously in real time. A fraud detection system that checks every credit card transaction the moment it happens uses streaming data. Streaming systems are more complex to build but deliver results instantly.

Batch Data:
Events ---> Collect for 24 hours ---> Process all at once ---> Results

Streaming Data:
Event 1 ---> Process immediately ---> Result 1
Event 2 ---> Process immediately ---> Result 2
Event 3 ---> Process immediately ---> Result 3

First-Party vs Third-Party Data

Data engineers also encounter data divided by its origin.

First-Party Data

First-party data comes directly from a company's own systems and interactions. A website's analytics, customer purchase histories, and internal employee records are all first-party data. Organizations own and control this data fully.

Third-Party Data

Third-party data comes from external sources. A company might purchase demographic data from a market research firm or receive location data from a mapping provider. Third-party data enriches internal data but comes with licensing restrictions and quality concerns the data engineer must account for.

Why Data Types Matter for Engineering Decisions

Every storage and processing decision depends on data type. A relational database is the right choice for structured transactional data. A data lake is the right choice for raw, mixed-format data including semi-structured and unstructured files. A streaming platform like Apache Kafka is the right choice for real-time event data.

Choosing the wrong storage or processing tool for a data type leads to slow queries, storage waste, or outright failures. A data engineer who understands data types from the start avoids costly redesigns later.

Summary

Data comes in three main structural types: structured (tables), semi-structured (JSON, XML), and unstructured (images, video). It also varies by movement pattern: batch or streaming. Understanding these distinctions helps data engineers choose the right tools and design systems that handle real-world data correctly from the beginning.