2

As I read an introductory statistics book, a question struck me which am curious to find a good answer for.

Background

The author teaches that statistical classification (understand this to mean organizing raw data into say classes -e.g by grouping the data into equal-sized categories, and assigning them frequencies, like what happens in preparation of a histogram) is a form of data compression. He further teaches that much as this compression approach aids in extraction and learning of the important information embedded in the data, he warns that certain approaches (e.g the use of the mode as a measure of central tendency) risk leaking away important information.

Of significance, he states that the mean (a measure of central tendency) and variance (a measure of variation / dispersion) are most important.

Problem

Now, I wonder, assuming one was to follow this line of thinking, and given a data sample, proceeded to statistically-compress it (in the sense indicated above).

  1. What statistical compression mechanism is sufficient to be able to recover without any loss whatsoever, the original data?
  2. And assuming such a lossless recovery is not possible, how can one reconstruct the most statistically-close (a probably approximate) approximation to the original data?

Side-Note

What compression information is sufficient to best reconstruct (or approximate) the original dataset:

  1. Only the mean and variance?
  2. Knowledge of the probability distribution of the mean and that of the variance?
  3. Or a smaller, but more representative sample of the original population?
  4. Or just the probability distribution of the data?
  5. Any other better mechanism?

Illustration / Application / Clarification:

Asume am given the following datasets (each line is a different dataset):

66, 66, 66, 67, 67, 67, 68, 69

52, 53, 61, 67, 71, 72, 78, 82

43, 44, 50, 54, 67, 90, 91, 97

Note that the mean for all three of the above samples is 67. Now, assume there was some statistical operation(s),$f$, such that applying it to any of the above samples would yield a (smaller) set of information,$s$, from which the original dataset could be reconstructed (preferably without any loss -- but a probably approximate reconstruction is equally good for me). The problem in this case would be to find such a function $f$ that can yield the small set of information $s$, from which (via some reverse-transformation, $f'$), one can obtain the original data:

$$f(Data) = s$$ such that $$f'(s) = Data$$

And $f$ is an application of statistical methods.

JWL
  • 217
  • "What statistical compression mechanism is sufficient to be able to recover without any loss whatsoever, the original data?" There are lots of ways you can transform data without losing anything, but most of them aren't useful. What is the intended application? – Qiaochu Yuan Jan 04 '13 at 10:46
  • @QiaochuYuan, would it clarify my problem if i add: using only basic statistical tools e.g mean, variance and probability distributions? – JWL Jan 04 '13 at 11:22
  • No. I don't know what you mean by compressing data using basic statistical tools. In general I am suspicious of such questions unless you have an intended application in mind, which has the tendency of dramatically focusing questions. – Qiaochu Yuan Jan 04 '13 at 11:50
  • @QiaochuYuan, is the added application / clarification helpful? – JWL Jan 04 '13 at 12:10
  • You're aware that there is no possible function that can losslessly compress every input to a smaller set of data? –  Jan 04 '13 at 12:23
  • @RahulNarain, in my problem, lossless is just a bonus for the transformation, I think, probably approximate would be good too. Also, note that I explore this problem for pure learning / theoretical purposes. I currently don't have any serious applications of it in mind - though they are obvious if at all this is possible. – JWL Jan 04 '13 at 12:48
  • 1
    You can look into principal component analysis. Here's a page that briefly explains how it can be used for compression: http://www.willamette.edu/~gorr/classes/cs449/Unsupervised/pca.html –  Jan 04 '13 at 18:09

0 Answers0