Pandas - Missing Values

Lecture 15:- Pandas - Missing Values

In data analysis, dealing with missing values is a common challenge. Pandas provides several methods for handling missing data in DataFrames. Missing values are typically represented by either NaN (Not a Number) or None in Pandas. Here are some common methods to handle missing values in Pandas:

Detecting Missing Values: To identify missing values in a DataFrame, you can use the isnull() or isna() functions, which return a DataFrame of the same shape as the input, where each element is a boolean value representing whether it's a missing value or not.

Output:

Dropping Missing Values: To remove rows containing missing values, you can use the dropna() function. By default, it drops any row that contains at least one missing value.

Filling Missing Values: To replace missing values with a specific value, you can use the fillna() function. It allows you to fill missing values with a constant or use various methods such as forward fill (ffill) or backward fill (bfill).

Imputation with Mean, Median, or Mode: You can use statistical measures like mean, median, or mode to fill missing values in numerical columns.

These methods offer different ways to handle missing values in a DataFrame. The choice of method depends on the specific dataset and the analysis you are conducting. Handling missing data effectively is crucial to obtain accurate insights from your data.

pythonCopy code

import pandas as pd # Assuming you have a DataFrame named 'df' df['Age'].fillna(df['Age'].mean(), inplace=True)  # Fills missing values in 'Age' column with mean # Output print(df)

pythonCopy code

import pandas as pd # Assuming you have a DataFrame named 'df' df.fillna(value=0, inplace=True)  # Fills missing values with 0 in place # Output print(df)

pythonCopy code

import pandas as pd # Assuming you have a DataFrame named 'df' df.dropna(inplace=True)  # Drops rows containing missing values in place # Output print(df)

mathematicaCopy code

      A      B 0  False  False 1  False  False 2   True  False 3  False   True 4  False  False

pythonCopy code

import pandas as pd # Create a DataFrame with missing values data = {'A': [1, 2, None, 4, 5],        'B': [10, 20, 30, None, 50]} df = pd.DataFrame(data) # Detect missing values missing_values = df.isnull()  # or df.isna() # Output print(missing_values)