How to detect anomalies in business data
Why hand-set thresholds go stale, why median and MAD beat mean and standard deviation, and why a rule should stay silent without enough history.
"Tell me if sales drop by 20%." It sounds reasonable, and it stops working the month after you set it. A hand-set threshold starts going stale immediately: the business grows, seasons change, the product mix changes. Three months later either everything alerts or nothing does — and both end the same way, with nobody looking.
Three problems with hand-set thresholds
- Thresholds go stale. Today's normal is not tomorrow's anomaly. When revenue doubles, "a 20% drop" means a completely different amount.
- Every breakdown needs its own threshold. Product group, customer, warehouse, sales rep — the count passes a hundred quickly and nobody maintains it.
- Seasonality cannot be caught by hand. December is always different; a fixed threshold alerts for nothing in December and misses a real drop in February.
The alternative is not a better guess but deriving the threshold from the data's own history — which means first choosing the right statistic.
Why mean and standard deviation fall short
The classic approach measures how many standard deviations a value sits from the mean. The problem: the mean and the standard deviation are themselves distorted by the very outlier you are trying to catch.
An example: eleven months of roughly steady revenue, then one month with a large one-off project invoice. That month pulls the mean up and inflates the deviation. When a genuine drop arrives the following month, the inflated deviation makes it look like it is within normal range. One anomaly hides the next.
The robust alternative: median and MAD
Use measures that outliers do not move. The median replaces the mean: one large value does not shift it. MAD (median absolute deviation) replaces the standard deviation: the median of each observation's distance from the median.
The robust z-score is built like this:
z = 0.6745 × (x − median) / MAD
The constant is not arbitrary: under a normal distribution it scales MAD onto the same footing as the standard deviation. "z = 3" therefore keeps the meaning it has in the classic z-score, and familiar thresholds still apply (say 2.5 for a warning, 3.5 for critical).
If MAD comes out zero — half the history is exactly the same value — fall back to mean and deviation; if that is zero too, the series never varied and no z is computed. This matters: producing a z-score for a series with no variance is just hiding a division by zero.
Without enough history, stay silent
On a fresh installation with two months of data, computing "deviation versus history" is meaningless: everything looks anomalous, the dashboard turns red, and the user loses trust on day one.
The correct behaviour is to not compute below a minimum number of observations. Ours is six periods; below that a rule stays in a "watching" state and produces no number at all. Inventing a threshold instead — "not much data, let's say 30% for now" — sells the user a false certainty.
An alert without a direction is meaningless
If net sales quadrupled, that is an anomaly but not a warning. Every rule needs a direction: on this measure, do I care about a decrease, an increase, or both?
A measured example: when we quadrupled net sales artificially, the robust z-score came out at +23.3 — statistically extreme. The rule still stayed silent, because its direction is decrease. A system that alerts on good news makes the bad news unhearable too.
Snapshot tables: you have to accumulate the history
Some tables carry a time dimension (monthly sales movements) and hold their own history. Others are snapshots: receivables ageing, stock position, the open order list. They tell you "right now" and nothing about yesterday.
To compare against history on those, you have to store the measurement yourself — and there is an easy mistake here: storing it on every run. If the pipeline runs hourly, "six periods minimum" fills up in six hours and the comparison loses its meaning, because you compared this hour's stock with stock from six hours ago. Keep one observation per day; later runs on the same day update that day's value.
Causes are computed, not guessed
"Sales dropped" is not actionable on its own; you need the "why". But guessing the cause — asking a model, or writing the most plausible story — risks producing an invented explanation.
There is a computable answer: run the same measurement once more with the breakdown and find each member's contribution share — the member's change against its own baseline, divided by the total change. The resulting list is arithmetic, not a guess.
A measured example: dropping net sales by 65% fired the rule at critical level (z = −5.59) and ranked the drivers as semi-finished goods 30.9%, trade goods 21.9%, raw materials 17.1%. On a measure with no breakdown, that field is left empty rather than filled in.
Identity checks work differently
Not every rule is statistical. Reconciliation differences, batch balances that disagree with stock, totals that do not add up — the correct value for these is exactly zero. Looking at history here would be wrong: a system that says "12 items disagreed last month, 13 this month, normal" has normalised a defect.
On identity checks a single deviation is critical and the threshold is fixed. These rules need no history; they work from day one.
How to test it: in both directions
There is only one way to validate an anomaly engine, and it asks two questions at once:
- Does it stay quiet on healthy data? A false alert is more damaging than no alert — by week two nobody looks. Our measured result: on healthy data all six time-series rules stayed silent, with z between −1.21 and 0.44; the three snapshot rules reported "not enough history".
- Does it fire on a real deviation? You have to break the data on purpose. An artificial 65% drop produced critical; breaking a single item produced critical plus the item name from the identity rule.
Miss either half and the engine is useless: a silent engine is safe but pointless, and an engine that talks constantly is noise.
Where the model comes in
One last point: do not hand any of this arithmetic to a language model. The model does not compute, does not set thresholds and does not invent causes. Everything is calculated deterministically; if the model has a job at all, it is only to put the given numbers into a sentence. Otherwise you end up with a system that answers the same question two different ways.
Floemdata builds its anomaly layer on exactly these rules: the measurement goes through the same compiler as the report, all arithmetic is deterministic, and when a rule stays silent the dashboard says why.