0

I have N measurements, each reports a Gaussian distribution of the scalar value (i.e. each reports mean and variance of it: $(\mu_i, \sigma_i^2)$).

What is the probability that these measurements are "correct"? Or, to phrase it differently, what is the probability that there exists a true value that all the N measurements detected?

This is a practical problem: I have ensemble of N neural networks, they get the same input and output $(\mu_i, \sigma_i^2)$ each. I want to measure how much they "agree" in a principled way. I.e. if they output exactly the same distribution the agreement should be high, if they output low sigma and means that are far away from each other - the agreement should be low.

Grigory Yakushev
  • 1
  • How were the N measurements generated? This sounds like the framework for confidence interval estimation with bootstrapping. – Godfather Jun 04 '21 at 14:41
  • Are you saying you have $N$ measurements $(\mu_1, \sigma_1^2), \dots, (\mu_N, \sigma_N^2)$ and you want to know some sort of probability they all were obtained from a common distribution? You'd have to specify something about the sampling process. Or maybe you mean something different, such as the confidence you have in each measurement. In that case @AdityaDua has the right idea. – A rural reader Jun 04 '21 at 14:45
  • This is a practical problem: I have ensemble of N neural networks, they get the same input and output $(\mu_i, \sigma_i^2)$ each.

    I want to measure how much they "agree" in a principled way. I.e. if they output exactly the same distribution the agreement should be high, if they output low sigma and means that are far away from each other - the agreement should be low.

     – Grigory Yakushev Jun 04 '21 at 14:53
  • Also clarified by editing the question. – Grigory Yakushev Jun 04 '21 at 15:14

0 Answers0