---
title: "Five Common Ways Dirty Data Ruins Your Analysis"
description: "When the numbers look wrong, five learning-data quality problems to suspect before you blame the algorithm."
lang: "en"
date: "2026-01-07T15:00:00.000Z"
updated: "2026-06-28T02:37:00.000Z"
tags: ["data quality","data cleaning","error checking","learning analytics"]
canonical: "https://flipsson.com/en/blog/mLaG4FeSieHo"
---

# Five Common Ways Dirty Data Ruins Your Analysis

When a number on the dashboard contradicts their intuition, most teachers doubt their own judgment first. But in practice the real cause is far more often ==not the analysis method but contamination in the data that went in==. "Garbage in, garbage out" applies to learning data exactly as it does anywhere else. However sophisticated the analysis, on dirty data it only produces a plausible lie. If you want to trust an analysis, start by checking the quality of the data.

## Five kinds of contamination you will meet

You will find at least one of these five in almost any school's data.

1. **Duplicate records**: a student submits the same activity twice, or the system stores the same log twice. Participation rates quietly inflate. If a total is larger than your headcount, it is almost always duplication.
2. **Ghost accounts**: transferred-out students or test accounts left on the roster ==enlarge the denominator and make every rate look lower than it is==. A 70% completion rate may really be 85% once five ghosts come off.
3. **Time errors**: a misconfigured server time zone records a submission made just before midnight as the next day, distorting your entire late-submission rate. If it is a Korean school and the server sits in another time zone, you are off by nine hours.
4. **Confusing blanks with zeros**: treating a missed test as a zero destroys the average. ==Not sitting a test and scoring zero mean completely different things==, and they get merged into the same number.
5. **Mismatched units**: if one system logs study time in minutes and another in seconds, joining them produces a 60-fold error.

> If an analysis result is surprising, weigh the possibility that you have found an error rather than an insight.

## A five-minute check before you analyze

Before you get into the real analysis, the habit of checking these four things quickly filters out most contamination.

- **Scan the extremes**: ==values that could not happen are almost always errors== — 999 minutes of study in a day, one student submitting an assignment 50 times. Sort the column and look at both ends first.
- **Reconcile the totals**: check that your class headcount matches the number of data rows. A mismatch means duplication or omission.
- **Check the blank rate**: if blanks in a given column exceed 30%, do not draw a conclusion from that metric alone. A conclusion on thin evidence is worse than none.
- **Spot-check against reality**: pick three students and eye-check their data against the actual records. Even automated numbers earn trust only after a human has verified them once.

Data quality is not something you check once and finish with. Every time a term turns over or a tool updates, the same contamination comes back in. So it is better to treat the quality check as a short ritual you always repeat right before analyzing. Once you are used to it, a 30-student class's data takes three minutes to scan. Only the first couple of rounds feel like a chore, and against the risk of giving a wrong explanation to a parent on the basis of a wrong number, those three minutes are the cheapest insurance there is.

## Key takeaways

Good analysis comes only from good data. Duplicates, ghost accounts, time errors, blank handling, and mismatched units are the five most common culprits behind ==an illusion that looks like insight==. When a result contradicts your intuition, doubt the data before you doubt your judgment. Five minutes of quality checking prevents a term's worth of bad decisions.