Floemdata
Data & ETL

Data Retention and Archiving Decisions

How long should you keep data? Steps to frame retention, archiving and deletion; ask your legal adviser about statutory periods. Floemdata's current status.

Floemdata · · 4 min read
A card slides into a lidded archive box; a dashed empty card outline waits beyond it

The warehouse grows every day. Keep the sales lines from five years ago, move some to an archive, or delete them? In most companies this decision is never made; data piles up because it is "just sitting there". This post helps you frame the decision and is not legal advice.

Three separate questions

  • Retention: which data is kept, for what purpose, and for how long?
  • Archiving: can rarely used but irreplaceable data move to a cheaper, separate place? It leaves the reports but can still be found when needed.
  • Deletion: when the period ends, is the data really destroyed, backups included?

Who sets the period

Record-keeping periods can come from tax, commercial, employment and data protection law, and they differ by type of data. Do not take periods from this post or from a web search: ask your legal adviser or accountant and keep the answer in writing.

Take an inventory

  • Which tables are in the warehouse and which source does each come from?
  • Which hold personal data (customer names, contact details, employee data)?
  • Which reports need which history? A report that compares three years cannot work if you delete data older than three years.
  • Where are the other copies of this data: backups, exported files, e-mail attachments?

A practical structure

  • Hot: the current period that reports query, in the warehouse.
  • Cold: older periods, compressed and in a separate file; access is logged.
  • Deleted: what has expired. Record the deletion and its date; a year from now you should be able to explain what was deleted and why.

Try a small deletion first and confirm that no report breaks. For personal data, also decide who answers deletion or anonymisation requests, and within what time.

Where Floemdata stands

Floemdata also pulls fields such as the customer card name from SAP Business One (CardName is among the expected columns of the OCRD table). The Bronze layer has no retention definition, and we did not find code that deletes rows from Bronze or shrinks the file; data stays until you remove it. We also found no feature that deletes or anonymises one particular customer's data from the warehouse. This is the state of today's code, not a commitment.

To start, list the tables in your warehouse and write next to each one: "retention question: who to ask". Questions are welcome at info@floemdata.com or via Floemdata.

← All guides