Is an outlier, defined by being further than 1.5x the range of the Interquartile Range outside of the IQR, an actual outlier or merely a potential outlier? Different sources seem to use different semantics.
Asked
Active
Viewed 2,528 times
2
-
1These are just heuristics. I think they're from Tukey (1973). The problem of distinguishing outliers from data is a central problem of statistics. – Brian Tung Mar 07 '17 at 22:45
1 Answers
0
Early boxplots used the rule that observations outside 'inner fences' at $Q_1 - 1.5(\text{IQR})$ and $Q_3 + 1.5(\text{IQR})$ were called possible outliers and those outside 'outer fences' at $Q_1 - 3.0(\text{IQR})$ and $Q_3 + 3.0(\text{IQR})$ were called probable outliers.
Recent statistical software packages seem to have adopted the $1.5(\text{IQR})$-rule and ignored the $3.0(\text{IQR})$-rule. Possibly, the reason is simplicity in making boxplots for presentations.
Neither rule is entirely satisfactory for all applications. In R statistical
software the parameter range=1.5 is the default, but other reasonable
values can be specified.
When using boxplots to 'detect' outliers, it is good practice to specify the rule you are using.
Hotes: The term outlier has different meanings in various contexts.
For a normal sample of intermediate size the $1.5(\text{IQR})$-rule will often show a few 'outliers'. While 'almost all' of normal observations lie in the interval $\mu \pm 3\sigma$, normal tails actually extend to $\pm \infty,$ so having occasional extreme observations is a property of the distribution.
Also, using $1.5(\text{IQR})$-rule, almost all samples from the right-skewed, heavy-tailed exponential distribution will show outliers; so something may be wrong if you don't see any.
The distribution of the magnitudes of California earthquakes is strongly right-skewed; the only quakes noticed by (or of interest to) the general public are the outliers.
In emerging technilogies, outliers may be the only useful observations; perhaps, the ones with a strong enough 'signal' to overcome the 'noise' of a process that may not yet be well-understood.
An outlier may simply be a 'mistake': due to equipment failure, operator error, or a data entry error. Sometimes the source of the error can be identified and corrected, sometimes the incorrect observation needs to be disregarded.
So an 'outlier' is an observation that is worth a second look; it may or may not result from a mistake or need to be 'cleaned' from the dataset.
The figure below shows outliers of both types for 20 normal samples of size $n=75$ (first row); similarly for 20 exponential samples of size $n = 75.$
BruceET
- 51,500