Introduction to Pandas

Welcome! This tutorial will teach you the essential Pandas skills you need to start working with data in Python.

We'll use one small car sales dataset throughout the tutorial. This makes it easier to follow because you'll keep working with the same data as you learn new things.

You don't need to know Pandas already. You only need basic Python knowledge.

By the end, you'll be able to:

• Understand DataFrames and Series
• Load and save data
• Inspect and understand your data
• Select, filter, and sort rows and columns
• Clean messy data and handle missing values
• Create new features
• Summarize data with groupby and crosstab
• Create basic visualizations
• Apply custom transformations

We'll create the car sales dataset ourselves. It will also contain some common problems that you may find in real-world data, such as messy prices, inconsistent text, and missing values.

0. What is Pandas?

Pandas is a Python library that helps us work with table-like data.

Think about a table in Excel:

Pandas lets us create, read, change, filter, and analyze tables like this using Python.

Pandas mainly gives us two important things:

• DataFrame – the entire table
• Series – one column from the table

A simple way to remember this:

DataFrame = whole table
Series = one column

1. Setup

First, install Pandas if you haven't already.

Run this in your terminal:

pip install pandas 

Then import Pandas in your Python script or notebook:

import pandas as pd 

Here, pd is just a shorter name for Pandas. Without as pd, we would have to write pandas every time. With as pd, we can simply write:

pd.DataFrame() 

instead of:

pandas.DataFrame() 

2. A Quick First DataFrame

Before working with our car data, let's create a very small DataFrame.

# Create a simple DataFrame data = { "Name": ["Alice", "Bob", "Charlie", "Diana"], "Age": [25, 30, 35, 28], "City": ["New York", "London", "Paris", "Tokyo"] } df = pd.DataFrame(data) print(df) 

Output:

 Name Age City 0 Alice 25 New York 1 Bob 30 London 2 Charlie 35 Paris 3 Diana 28 Tokyo 

Let's understand what we're seeing.

There are three columns:

• Name
• Age
• City

There are also numbers on the left:

0 1 2 3 

These numbers are called the index. Pandas automatically creates an index for each row. The index helps Pandas identify each row. It is not one of our actual data columns.

3. Loading the Car Sales Dataset

Now let's create the dataset we'll use for the rest of the tutorial.

Our dataset contains these columns:

• Make – the car manufacturer
• Model – the car model
• Year – the manufacturing year
• Price – the price of the car
• Odometer (KM) – how many kilometers the car has been driven
• Doors – number of doors
• Color – the car's color

Some of the data is intentionally messy. For example, the price contains $ and commas, and Toyota is written in different ways.

data = { "Make": ["Toyota", "Honda", "toyota", "Ford", "BMW", "Honda", "TOYOTA", "Ford", "BMW", "Nissan"], "Model": ["Corolla", "Civic", "Camry", "Focus", "3 Series", "Accord", "RAV4", "Fiesta", "5 Series", "Altima"], "Year": [2015, 2018, 2017, 2016, 2019, 2020, 2014, 2018, 2017, 2016], "Price": ["$15,000", "$18,500", "$16,200", "$10,000", "$35,000", "$22,000", "$14,500", "$12,000", "$32,500", "$17,000"], "Odometer (KM)": [120000, 60000, 85000, 130000, 45000, 20000, 150000, 90000, 55000, 100000], "Doors": [4, 4, 4, 5, 4, 4, 4, 3, 4, 4], "Color": ["White", "Black", "Red", "Blue", "Silver", "White", "Red", "Green", "Black", "Blue"] } car_sales = pd.DataFrame(data) print(car_sales) 

Output:

 Make Model Year Price Odometer (KM) Doors Color 0 Toyota Corolla 2015 $15,000 120000 4 White 1 Honda Civic 2018 $18,500 60000 4 Black 2 toyota Camry 2017 $16,200 85000 4 Red 3 Ford Focus 2016 $10,000 130000 5 Blue 4 BMW 3 Series 2019 $35,000 45000 4 Silver 5 Honda Accord 2020 $22,000 20000 4 White 6 TOYOTA RAV4 2014 $14,500 150000 4 Red 7 Ford Fiesta 2018 $12,000 90000 3 Green 8 BMW 5 Series 2017 $32,500 55000 4 Black 9 Nissan Altima 2016 $17,000 100000 4 Blue 

We'll keep using this DataFrame. Later, we'll save it as a CSV file and load it again.

4. Data Structures: Series and DataFrame

Pandas has two main data structures:

• Series – one column
• DataFrame – the complete table

Our car_sales variable is a DataFrame. Each column inside car_sales is a Series.

For example:

# Get the 'Make' column as a Series make_series = car_sales["Make"] print(type(make_series)) print(make_series) 

Output:

<class 'pandas.core.series.Series'> 0 Toyota 1 Honda 2 toyota 3 Ford 4 BMW 5 Honda 6 TOYOTA 7 Ford 8 BMW 9 Nissan Name: Make, dtype: object 

So when we do:

car_sales["Make"] 

we get one Series containing the Make column.

A Series has:

• an index
• values

The index is on the left, and the actual values are on the right.

5. Import & Export

In real projects, data usually doesn't come directly from a Python dictionary. It might come from a CSV file. Pandas makes it easy to save data to a file and read it back.

Save to CSV

We can save our DataFrame as a CSV file:

car_sales.to_csv("car_sales.csv", index=False) 

The index=False means:

Don't save Pandas's index as an extra column in the CSV file.

Read from CSV

Now we can read the CSV file:

car_sales_loaded = pd.read_csv("car_sales.csv") print(car_sales_loaded.head()) 

We can also replace our original DataFrame with the loaded data:

car_sales = pd.read_csv("car_sales.csv") 

Now car_sales contains the data that came from the CSV file.

6. Understanding Your Data

Before working with data, it's a good idea to inspect it.

You want to know things like:

• What are the column names?
• What type of data is in each column?
• How many rows and columns are there?
• Are there missing values?
• What do the numbers look like?

Column names

To see the column names:

print(car_sales.columns) 

Output:

Index(['Make', 'Model', 'Year', 'Price', 'Odometer (KM)', 'Doors', 'Color'], dtype='object') 

This tells us which columns exist in our DataFrame.

Data types

We can check the type of each column:

print(car_sales.dtypes) 

Output:

Make object Model object Year int64 Price object Odometer (KM) int64 Doors int64 Color object dtype: object 

Notice something interesting:

Price object 

You might expect Price to be a number. But our prices look like:

"$15,000" 

Because they contain $ and ,, Pandas treats them as text. We'll fix this later.

Shape

shape tells us how many rows and columns our DataFrame has.

print(car_sales.shape) 

Output:

(10, 7) 

This means:

10 rows 7 columns 

The first number is the number of rows. The second number is the number of columns.

Index

We can look at the DataFrame's index:

print(car_sales.index) 

Output:

RangeIndex(start=0, stop=10, step=1) 

Our index starts at 0 and goes up to 9.

Summary statistics

We can ask Pandas for some basic statistics about our numeric columns:

print(car_sales.describe()) 

Output:

 Year Odometer (KM) Doors count 10.000000 10.000000 10.00000 mean 2017.000000 85500.000000 4.00000 std 1.763834 40963.059426 0.47140 min 2014.000000 20000.000000 3.00000 25% 2016.000000 55000.000000 4.00000 50% 2017.000000 87500.000000 4.00000 75% 2018.000000 115000.000000 4.00000 max 2020.000000 150000.000000 5.00000 

This gives us a quick summary of the numeric columns.

For example:

• min = smallest value
• max = largest value
• mean = average
• count = number of values

DataFrame info

Another useful method is info():

car_sales.info() 

Output:

<class 'pandas.core.frame.DataFrame'> RangeIndex: 10 entries, 0 to 9 Data columns (total 7 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Make 10 non-null object 1 Model 10 non-null object 2 Year 10 non-null int64 3 Price 10 non-null object 4 Odometer (KM) 10 non-null int64 5 Doors 10 non-null int64 6 Color 10 non-null object dtypes: int64(3), object(4) memory usage: 688.0+ bytes 

This gives us a quick overview of the DataFrame.

For example, we can see that every column currently has 10 non-null values. That means there are no missing values yet.

Basic calculations

Pandas can also perform calculations for us. To calculate the average of numeric columns:

print(car_sales.mean(numeric_only=True)) 

Output:

Year 2017.0 Odometer (KM) 85500.0 Doors 4.0 dtype: float64 

We can calculate the total odometer value:

print(car_sales["Odometer (KM)"].sum()) 

Output:

855000 

We can also count how many rows we have:

print(len(car_sales)) 

Output:

10 

7. Viewing Data

Sometimes you don't want to print the entire DataFrame. You may only want to see the first few or last few rows.

head()

car_sales.head() 

By default, head() shows the first 5 rows. You can also choose how many rows you want:

car_sales.head(10) 

tail()

tail() shows rows from the bottom:

car_sales.tail(3) 

This shows the last 3 rows. These methods are useful when you simply want to take a quick look at your data.

8. Selecting Columns

We can select one column from our DataFrame.

# Single column (Series) car_sales["Make"] 

This gives us a Series. If we want multiple columns, we put their names inside another list:

# Multiple columns (DataFrame) car_sales[["Make", "Price"]] 

The important difference is:

car_sales["Make"] 

selects one column.

While:

car_sales[["Make", "Price"]] 

selects multiple columns.

9. Selecting Rows with .loc and .iloc

Pandas gives us two useful ways to select rows:

• .iloc – select using the row's integer position
• .loc – select using the row's index label

A simple way to remember:

iloc = integer location
loc = label/location

With our current default index, they can look very similar.

Using .iloc

The first row is at position 0:

car_sales.iloc[0] 

The third row is at position 2:

car_sales.iloc[2] 

We can also select the first three rows:

car_sales.iloc[:3] 

This means:

start at 0 stop before 3 

So we get rows:

0 1 2 

Using .loc

We can select the row whose index label is 3:

car_sales.loc[3] 

We can also select a range:

car_sales.loc[0:2] 

With .loc, the ending value is included. So:

car_sales.loc[0:2] 

gives us rows:

0 1 2 

Selecting one specific cell

We can select a single value by giving the row and column positions:

car_sales.iloc[1, 0] 

This means:

row 1 column 0 

The result is:

Honda 

We can reach the same cell using the label-based approach:

car_sales.loc[1, "Make"] 

Output:

Honda 

.loc[1, "Make"] addresses the row by its index label and the column by its name, which is often easier to read than iloc's positional [1, 0].

10. Filtering Rows (Boolean Indexing)

Filtering means:

Show me only the rows that match a condition.

For example, we can ask:

Which cars are made by Toyota?

toyota_cars = car_sales[car_sales["Make"] == "Toyota"] print(toyota_cars) 

Only rows where Make is exactly "Toyota" will be returned. Notice that "toyota" and "TOYOTA" are not returned. That's because Python treats these as different strings. We'll fix this later when we clean our data.

Multiple conditions

We can combine conditions.

For example:

Find cars with more than 100,000 KM and exactly 4 doors.

filtered = car_sales[(car_sales["Odometer (KM)"] > 100000) & (car_sales["Doors"] == 4)] print(filtered) 

Here:

& 

means AND.

Both conditions must be true.

isin() for multiple values

Suppose we want cars made by either Toyota or Honda.

We can use isin():

selected = car_sales[car_sales["Make"].isin(["Toyota", "Honda"])] print(selected) 

This checks whether each value exists in the list:

["Toyota", "Honda"] 

Remember these operators:

• & = AND
• | = OR
• isin() = value is one of several choices

When using multiple conditions, put each condition inside parentheses.

11. Sorting Data

We can sort our DataFrame by a column.

For example, let's sort cars by year:

# Sort by Year (ascending) df_sorted = car_sales.sort_values("Year") print(df_sorted) 

By default, sorting is ascending.

That means:

2014 2015 2016 2017 ... 

If we want the newest cars first, we can sort in descending order:

df_sorted = car_sales.sort_values("Year", ascending=False) 

We can also sort by more than one column:

car_sales.sort_values(["Make", "Year"]) 

This first sorts by Make, and then sorts by Year within each make.

12. Crosstab – Frequency Tables

pd.crosstab() helps us count how often values appear together.

For example:

pd.crosstab(car_sales["Make"], car_sales["Doors"]) 

Output:

Doors 3 4 5 Make BMW 0 2 0 Ford 1 1 1 Honda 0 2 0 Nissan 0 1 0 TOYOTA 0 1 0 Toyota 0 1 0 toyota 0 1 0 

We can read this as:

How many cars of each make have 3, 4, or 5 doors?

For example:

BMW 0 2 0 

means BMW has:

• 0 cars with 3 doors
• 2 cars with 4 doors
• 0 cars with 5 doors

Notice that Toyota appears three times:

TOYOTA Toyota toyota 

Pandas treats them as different values because their capitalization is different. We'll clean this later.

13. GroupBy – Aggregating by Group

groupby() is useful when we want to divide our data into groups and calculate something for each group.

For example, we can group cars by their manufacturer:

car_sales.groupby("Make").mean(numeric_only=True) 

This asks:

For each car manufacturer, what are the average numeric values?

Output:

 Year Odometer (KM) Doors Make BMW 2018.0 50000.0 4.000000 Ford 2017.0 110000.0 4.000000 Honda 2019.0 40000.0 4.000000 Nissan 2016.0 100000.0 4.000000 TOYOTA 2014.0 150000.0 4.000000 Toyota 2015.0 120000.0 4.000000 toyota 2017.0 85000.0 4.000000 

The important idea is:

groupby("Make") 

puts rows with the same Make together.

Then:

.mean() 

calculates the average for each group.

Again, because Toyota has different capitalization, Pandas creates three separate groups. We'll fix this in the cleaning section.

14. Basic Data Visualization

Pandas can also create simple charts. Pandas uses Matplotlib for its built-in plotting features.

Line plot

We can create a basic line plot of the odometer values:

car_sales["Odometer (KM)"].plot() 

This gives us a visual representation of the values.

Histogram

We can also create a histogram:

car_sales["Odometer (KM)"].hist() 

A histogram helps us see how the odometer values are distributed. For example, it can help us see whether most cars have lower or higher mileage.

In a plain Python script (as opposed to a Jupyter notebook), Pandas charts won't appear unless you import Matplotlib and call show():

import matplotlib.pyplot as plt car_sales["Odometer (KM)"].plot() plt.show() 

In notebooks, the chart usually displays automatically after the cell runs.

15. Cleaning Data

Now let's start fixing the messy parts of our dataset. Real-world data is often not perfectly clean. Our dataset has a few problems:

• Prices contain $ and ,
• The same manufacturer is written with different capitalization
• We'll introduce some missing values

Let's fix these one at a time.

15.1 Clean the Price Column

Our prices currently look like this:

$15,000 $18,500 $16,200 

These are strings, not numbers. We want them to become:

15000 18500 16200 

First, remove the $:

.str.replace("$", "", regex=False) 

Then remove the commas:

.str.replace(",", "", regex=False) 

Finally, convert the result to a float:

.astype(float) 

We can do all of that together:

car_sales["Price"] = ( car_sales["Price"] .astype(str) # Convert to string type first to ensure .str accessor works  .str.replace("$", "", regex=False) # remove dollar sign  .str.replace(",", "", regex=False) # remove commas  .astype(float) # convert to float ) print(car_sales["Price"].head()) 

Output:

0 15000.0 1 18500.0 2 16200.0 3 10000.0 4 35000.0 Name: Price, dtype: float64 

Now Price is a numeric column. That means we can perform calculations on it.

15.2 Standardize the Make Column

Earlier, we saw these values:

Toyota toyota TOYOTA 

They all mean the same thing, but Pandas treats them as different values. We can convert all of them to lowercase:

Source：https://dev.to/iamismile/introduction-to-pandas-1hlh
My favorites

Related posts

The Building Is a System: What Software Engineers Can Learn from Construction Site Failures

Software engineers spend a lot of time thinking about systems. Google's Site Reliability Engineering book discusses how

0dev.to

Part 4: When the Screen Changes Silently: Live Regions, Headings and Focus in Compose

Build for Everyone · Part 4 of 5 Previously: Parts 1 to 3 made static screens accessible and kept them that way with tes

0dev.to

FlatCake: Why Tiered KV Cache Storage Is Holding LLM Serving Back

FlatCake: Why Tiered KV Cache Storage Is Holding LLM Serving Back When you send a long prompt to an LLM API, the service

0dev.to

Why Scrum is Not a Silver Bullet: 5 Structural Drawbacks and Fixes

The Uncomfortable Truth About Scrum Let's get one thing straight right out of the gate: Scrum is a fantastic framework f

0dev.to

Agentic Frameworks Compared: LangGraph, CrewAI & More

Originally published at GeekCoding101 . Introduction to the Agentic Framework Series Lately it feels like the world of A

0dev.to

The Real-World Pain

If you're running mission-critical automation pipelines on self-hosted n8n instances behind Cloudflare tunnels or revers

0dev.to