What does "Data Massage" mean?

Question

I am doing some reading, and came across avoiding an internalStore if my application does not need to massage the data before being sent to SQL. What is a data massage?

Answer 1

Manipulate, process, alter, recalculate. In short, if you are just moving the data in raw then no need to use internalStore, but if you're doing anything to it prior to storage, then you might want an internalStore.

Answer 2

Sometimes the whole process of moving data is referred to as "ETL" meaning "Extract, Transform, Load". Massaging the data is the "transform" step, but it implies ad-hoc fixes that you have to do to smooth out problems that you have encountered (like a massage does to your muscles) rather than transformations between well-known formats.

Thinks that you might do to "massage" data include:

• Change formats from what the source system emits to what the target system expects, e.g. change date format from d/m/y to m/d/y.
• replace missing values with defaults, e.g. Supply "0" when a quantity is not given.
• Filter out records that not needed in the target system.
• Check validity of records, and ignore or report on rows that would cause an error if you tried to insert them.
• Normalise data to remove variations that should be the same, e.g. replace upper case with lower case, replace "01" with "1".

Answer 3

Clean up, normalization, filtering, ... Just changing the data somehow from the original input into a form that is better suited to your use.