DS Data Import

Every analysis starts with getting data into Python. Data arrives in many shapes: spreadsheets, text files, web pages, databases, and online services. Pandas reads most of them with one short command, and the result lands in a table called a DataFrame.

Think of a kitchen that receives groceries from many shops. Boxes, bags, and crates all end up on the same counter. Importing does the same job for data.

Common Sources and Matching Commands

Source	Pandas Command	Typical Use
CSV file	read_csv	Exports from apps and tools
Excel file	read_excel	Business reports
JSON file	read_json	Web services and logs
SQL database	read_sql	Company data stores
Web table	read_html	Tables on public pages
Parquet file	read_parquet	Large, fast storage

The Import Flow

 [ CSV ]    [ Excel ]    [ JSON ]    [ Database ]    [ Web API ]
    \          |           |            |              /
     +---------+-----------+------------+-------------+
                            |
                     Pandas read_* command
                            |
                      DataFrame (table)
                            |
                   Check shape, types, head

Read a CSV File

import pandas as pd

df = pd.read_csv("sales.csv")
print(df.shape)
print(df.head())
print(df.dtypes)

A CSV file stores rows as lines of text with values separated by commas. The shape shows row and column counts. The dtypes line reveals whether Pandas read numbers as numbers and dates as text.

Useful options

df = pd.read_csv(
    "sales.csv",
    sep=";",
    usecols=["date", "product", "amount"],
    parse_dates=["date"],
    na_values=["NA", "missing", "-"],
    nrows=1000
)

• sep: sets the separator when the file uses semicolons or tabs.
• usecols: loads only the columns you need and saves memory.
• parse_dates: converts text into real dates.
• na_values: marks custom words as missing values.
• nrows: reads only the first rows for a quick peek at large files.

Read an Excel File

df = pd.read_excel("report.xlsx", sheet_name="January")
all_sheets = pd.read_excel("report.xlsx", sheet_name=None)

Setting sheet_name to None returns a dictionary that holds every sheet. Install the openpyxl library first, because Pandas relies on it to open modern Excel files.

Read a JSON File

df = pd.read_json("orders.json")
nested = pd.json_normalize(data, record_path="items", meta=["order_id"])

JSON often nests lists inside records. The json_normalize function flattens those layers into plain columns.

Read from a Database

import sqlite3

conn = sqlite3.connect("shop.db")
df = pd.read_sql("SELECT * FROM orders WHERE amount > 100", conn)
conn.close()

Let the database filter rows before they reach Python. A query that returns 10000 rows beats a query that returns 10 million rows and filters later.

Read Data from a Web API

import requests

url = "https://api.example.com/products"
response = requests.get(url, timeout=10)
response.raise_for_status()
df = pd.DataFrame(response.json())

An API is an online doorway that returns data on request. Check the provider's terms of use, respect rate limits, and store keys outside your code.

Handle Large Files

chunks = pd.read_csv("big.csv", chunksize=100000)
total = 0
for chunk in chunks:
    total += chunk["amount"].sum()
print(total)

Chunking reads a slice at a time, so a file bigger than your memory still works. Reduce memory further by converting text columns with few unique values to the category type.

Save Data Back to Disk

df.to_csv("clean_sales.csv", index=False)
df.to_parquet("clean_sales.parquet")
df.to_excel("clean_sales.xlsx", index=False)

First-Look Checklist

• Confirm the row and column counts match the source.
• Check that numbers, dates, and text carry the right types.
• Scan the first and last rows for stray headers or footers.
• Count blanks in every column.
• Note the file name, date, and source in your notebook.

Fix Common Import Problems

Symptom	Likely Cause	Fix
All values land in one column	Wrong separator	Set sep to a semicolon or tab
Strange symbols in names	Wrong text encoding	Set encoding to utf-8 or latin-1
Dates appear as text	No date parsing	Use parse_dates or to_datetime
Extra header rows inside the data	Report titles above the table	Use skiprows or header options

Open the raw file in a plain text editor when a result looks wrong. A thirty-second look often reveals the cause.

Key Points

• Pandas offers a read command for each common source.
• Options such as usecols, parse_dates, and chunksize control speed and accuracy.
• A quick first-look check catches import problems early.