Data Cleaning Glossary
Every customer file needs the same handful of fixes before it is safe to store: trimming, standardizing, validating, and deduplicating. This glossary explains each one in plain language, with a JavaScript example you can run, the edge cases that break naive versions, and how Dromo handles it, with built-in field types and validators or a few lines of hook code.
Subscribe and stay up to date with the latest tips and news from Dromo.
Concatenating Fields
Combining two or more fields into one value.
Dropping Invalid Rows
Removing rows that fail validation, while keeping a record of why.
Dynamic Data Validation
Validating data against rules or reference data that are loaded at runtime.
Field Name Matching
Mapping the column headers in a file to the fields your system expects.
Finding Text Within a String
Locating a substring inside a value, and optionally replacing it.
Flagging Invalid Rows
Marking rows that fail validation so they can be fixed, not deleted.
Formatting Numbers
Changing how a number is displayed without changing the number.
Imputing Missing Values
Filling in missing values with a substitute, or deciding not to.
Normalizing Cases
Converting text to a consistent case so equal values compare as equal.
Removing Duplicates
Finding rows that represent the same thing and keeping one.
Requiring Values
Making sure required fields contain a real value.
Splitting Fields
Separating the contents of one field into several fields.
Standardizing Date Formats
Converting dates written in different formats into one consistent format.
Trimming Fields
Removing unwanted whitespace from the start and end of a value.
URL Validation
Checking that a value is a well-formed web address before it is saved.