Data Manipulation with Pandas

Pandas is a powerful tool for manipulating and analyzing data. It provides a variety of functions and methods to clean, transform, and explore datasets.

Core Manipulation Techniques

• Selection and Indexing:
  • loc: Label-based selection.
  • iloc: Integer-based selection.
  • []: For column selection.
  • Boolean indexing for filtering rows based on conditions.
• Data Cleaning:
  • Handling missing values: fillna, dropna.
  • Removing duplicates: duplicated, drop_duplicates.
  • Handling outliers: statistical methods or domain knowledge.
• Data Transformation:
  • Applying functions to columns or rows: apply, map.
  • Creating new columns: using expressions or functions.
  • Renaming columns: rename method.
  • Pivoting data: pivot and melt functions.
• Data Aggregation:
  • Grouping data: groupby.
  • Applying aggregate functions: sum, mean, count, min, max.
• Concatenation and Merging:
  • Combining DataFrames: concat, merge, join.

Example:

Python

import pandas as pd

# Sample DataFrame
data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

# Selection 
print(df['Age'])  # Select a column
print(df.loc[0])  # Select a row by label
print(df.iloc[1:3])  # Select rows by position

# Data cleaning
df.fillna(0, inplace=True)  # Fill missing values with 0
df.drop_duplicates(inplace=True)  # Remove duplicates

# Data transformation
df['Age_Category'] = pd.cut(df['Age'], bins=[20, 30, 40], labels=['Young', 'Adult', 'Senior'])

# Data aggregation
grouped = df.groupby('City')
print(grouped.mean())

Additional Features

• Hierarchical indexing: Creating multi-level indexes.
• Time series analysis: Handling date and time data.
• Categorical data: Working with categorical variables.

By mastering these techniques, you can effectively manipulate and extract insights from your data using Pandas.

Data Manipulation with Pandas

Read More..