Five Common Ways Dirty Data Ruins Your Analysis

When a number on the dashboard contradicts their intuition, most teachers doubt their own judgment first. But in practice the real cause is far more often not the analysis method but contamination in the data that went in. "Garbage in, garbage out" applies to learning data exactly as it does anywhere else. However sophisticated the analysis, on dirty data it only produces a plausible lie. If you want to trust an analysis, start by checking the quality of the data.

Five kinds of contamination you will meet

You will find at least one of these five in almost any school's data.

1. Duplicate records: a student submits the same activity twice, or the system stores the same log twice. Participation rates quietly inflate. If a total is larger than your headcount, it is almost always duplication.
2. Ghost accounts: transferred-out students or test accounts left on the roster enlarge the denominator and make every rate look lower than it is. A 70% completion rate may really be 85% once five ghosts come off.
3. Time errors: a misconfigured server time zone records a submission made just before midnight as the next day, distorting your entire late-submission rate. If it is a Korean school and the server sits in another time zone, you are off by nine hours.
4. Confusing blanks with zeros: treating a missed test as a zero destroys the average. Not sitting a test and scoring zero mean completely different things, and they get merged into the same number.
5. Mismatched units: if one system logs study time in minutes and another in seconds, joining them produces a 60-fold error.

If an analysis result is surprising, weigh the possibility that you have found an error rather than an insight.

A five-minute check before you analyze

Before you get into the real analysis, the habit of checking these four things quickly filters out most contamination.

• Scan the extremes: values that could not happen are almost always errors — 999 minutes of study in a day, one student submitting an assignment 50 times. Sort the column and look at both ends first.
• Reconcile the totals: check that your class headcount matches the number of data rows. A mismatch means duplication or omission.
• Check the blank rate: if blanks in a given column exceed 30%, do not draw a conclusion from that metric alone. A conclusion on thin evidence is worse than none.
• Spot-check against reality: pick three students and eye-check their data against the actual records. Even automated numbers earn trust only after a human has verified them once.

Data quality is not something you check once and finish with. Every time a term turns over or a tool updates, the same contamination comes back in. So it is better to treat the quality check as a short ritual you always repeat right before analyzing. Once you are used to it, a 30-student class's data takes three minutes to scan. Only the first couple of rounds feel like a chore, and against the risk of giving a wrong explanation to a parent on the basis of a wrong number, those three minutes are the cheapest insurance there is.

Key takeaways

Good analysis comes only from good data. Duplicates, ghost accounts, time errors, blank handling, and mismatched units are the five most common culprits behind an illusion that looks like insight. When a result contradicts your intuition, doubt the data before you doubt your judgment. Five minutes of quality checking prevents a term's worth of bad decisions.