📊 Visual Summary: Exploring, Cleaning, and Manipulating Data (Ch. 10)

Before you can build great models or visualize data, you need to clean it. Real-world data is messy—full of missing values, weird outliers, and inconsistent formats. In this chapter, you'll learn how to explore, fix, and reshape data so it’s ready for analysis.

🔍 Exploring Data

Summarizing Columns: Use functions like min, max, sum, or average to understand ranges and trends.
Identifying Outliers: Look for values that don’t make sense—like a GPA of 9.8 or a negative number of Canvas logins.

📘 QC Example: A student loads class data and notices that one student appears to have logged into Canvas 1,800 times. That’s likely a glitch—or a bot.

🧹 Cleaning Data

Missing Data: Fill in blanks with averages, constants, or drop the row entirely.
Standardizing Formats: Make sure data like dates or letter grades are consistent (e.g., "A", "a", and "Excellent" might all mean the same thing).
Removing Duplicates: Ensure each row in your dataset represents a unique case.

📘 QC Example: Some students have missing midterm grades. The instructor fills them in with the class average for analysis.

🔧 Manipulating Data

Filtering: Keep only rows that meet specific criteria (e.g., only students with 90%+ attendance).
Grouping: Aggregate data by category (e.g., average GPA by major).
Transforming: Apply functions to columns (e.g., create a “Participation Score” based on discussion posts).

📘 QC Example: A student creates a new variable called engagement_score by combining attendance, Canvas logins, and assignment completion rate.

📏 Working with Vectors and Lists

Python lists and dictionaries are used heavily to represent and manipulate datasets.
Use loops and functions to iterate over data, apply changes, or filter based on conditions.

📘 QC Example: A student creates a function that checks if each student’s engagement score is above the class median, and flags those who may need support.

🌟 Final Thought

Clean data is the foundation of great data science. Even the most powerful machine learning model won’t work on dirty or inconsistent data. The time you spend cleaning and exploring is never wasted—it’s what turns raw numbers into insight.