Bias in Reports: Data Bias Traps
Reports can mislead even with correct numbers: Simpson's paradox, truncated axes, averaged ratios. We cover common traps with fictional examples.

This article is not legal or financial advice. A report can show every figure correctly and still lead the reader to the wrong conclusion. The problem is usually not the data but which slice is shown and how the numbers are combined. The examples below are fictional.
Simpson's paradox
According to the Stanford Encyclopedia of Philosophy, Simpson's paradox is when an association between two variables in a population emerges, disappears or reverses when the population is divided into subpopulations. The same source describes the UC Berkeley graduate admissions case (Bickel et al. 1975) as a classic example: men had a higher overall acceptance rate, but when examined by department, no gender bias appeared in any single department.
Fictional example: there are two sales teams. Team A makes 16 sales from 20 conversations with easy customers (80 percent) and 32 sales from 80 conversations with hard customers (40 percent); in total 48 sales from 100 conversations, or 48 percent. Team B makes 60 sales from 80 conversations with easy customers (75 percent) and 6 sales from 20 conversations with hard customers (30 percent); in total 66 sales, or 66 percent. In the overall picture Team B looks more successful, yet A is better with both customer types. The difference comes from the customer mix.
Other common traps
- Averaging a ratio: the average of monthly percentages is not the overall ratio; sum the numerator and denominator separately and divide.
- Truncated axis: if the axis does not start at zero, a small difference looks big.
- Survivorship bias: if only still-active customers are reported, those who left are missing from the table and satisfaction looks higher than it is.
- Period choice: choosing the start and end dates to suit the result changes the trend; comparing months without accounting for seasonality produces a fake decline or rise.
- Mistaking correlation for cause: two indicators moving together does not mean one causes the other.
Countermeasures
- Check an important overall number with at least one breakdown (customer type, region, product group).
- Show the numerator and denominator of ratios side by side.
- State the axis and period choice in the report.
- Say which records are in and out of the report (cancellations, customers who left).
Where Floemdata stands
In the measure catalog of Floemdata Analytics, ratios (for example margin percentage) are calculated by dividing after summing, not by averaging; the dimension catalog also offers breakdowns such as customer group, region and product group. This helps prevent some of the traps above but does not guarantee that a report is read correctly. Automatic bias or paradox detection is not a verified product feature (not verified).
To get started, recalculate one overall number in your most important report with one breakdown and see whether the direction of the result changes. Source: Stanford Encyclopedia of Philosophy, Simpson's Paradox.