Robust statistics are statistics with good performance for data drawn from a wide range of probability distributions, especially for distributions that are not normally distributed. Robust statistical methods have been developed for many common problems, such as estimating location, scale and regression parameters.
Robust statistics are insensitive to deviations from their underlying assumptions and outliers. Such methods are useful it is not possible to detect and remove outliers or to appropriately test the assumptions required by a given statistic. A robust statistic is meant to achieve three goals:
efficiency - it should have an optimal or nearly optimal efficiency as the assumed model stability - small deviations from the assumptions should have only a small influence on performance breakdown - larger deviations from the assumptions should not lead to a complete failure Examples of robust statistics are median regression as estimation technique, or Huber-White standard errors for statistical inference. Note that "robust" is not equivalent to "better". Robustness is always based on compromise as it sacrifices efficiency to ensure against larger deviations from the assumptions from the model (Anscombe, 1960).
For further reading see
Huber, P.J. and Ronchetti, E.M. (2009) "Robust Statistics", 2nd Edition, Wiley Series in Probability and Statistics, John Wiley & Sons, Inc., New Jersey Anscombe, F.J. (1960) "Rejection of Outliers", Technometrics, Vol. 2, pp. 123-147
