What is data integration and data transfer?
What data integration means, how data moves out of an ERP into a warehouse, and the hidden cost of exporting to spreadsheets by hand every single month.
Data integration is the work of bringing data from different systems into one consistent place. Data transfer is the moving part of that work: read from the source, transform, write to the target.
Three common approaches
- Manual export — someone pulls the ERP into a spreadsheet, edits it, shares it
- Direct connection — the reporting tool queries the production database live
- Transfer layer — a scheduled process moves data into a warehouse
The invisible cost of manual export
Manual export looks free: no licence, no project. The cost sits elsewhere — five versions of the same report circulate, nobody knows which is current, the report stops when the person who prepares it goes on leave, and most importantly: where the number came from is no longer traceable.
A direct connection is not free either. The reporting tool queries the production database; a heavy query can slow down ERP users, and the report logic gets buried inside that tool — change the tool and you rewrite everything.
What a transfer layer changes
- The source system is queried read-only; nothing is written back
- Extraction is scheduled, so the production database never carries live report load
- Data is written to the warehouse with versions — history can be reprocessed
- Report logic lives in the definition layer above the warehouse, not in the visualisation tool
How do you know the transfer is correct?
There is one test: does the figure produced from the moved data agree with the source system's own report? If it does not, the deviation should be shown, not hidden. An integration project that does not establish reconciliation from day one becomes indefensible the moment the numbers are questioned.