1

We know that for a gaussian distribution,

$f(x) = 1/\sqrt{2\pi\sigma^2} \exp[ -(x-\mu)^2/2\sigma^2 ]$

I was wondering that given an i.i.d. random variable with a given distribution, whether there exists some function $f(x)$.

For e.g: Let us say, $X$ is a random variable whose values are $X = {-3,-2,-1,0,1,2,3}$ with Probabilities $P{X} = \{0.1, 0.1, 0.2, 0.2, 0.2, 0.1 ,0.1\} $respectively. Its mean is $0$, and variance is $3$. Can we now find an equation for $f(x)$ like we found one for the gaussian distribution?

guest196883
  • 6,049
Rohit
  • 13

1 Answers1

0

The function you describe is called a probability density function for a continuous random variable. The analogue for a discrete distribution is the probability mass function. This, however, is basically exactly what you have described by declaring the possible values of the random variable and their associated probabilities.

We can, however, form a sort of pdf for the discrete random variable using the Dirac delta function. This is a "function" $\delta(x)$ which satisfies $\delta(x) = 0$ for $x\neq 0$ and $$\int_a^b\delta(x)dx = 1$$ if $0\in (a,b)$. Thus if we have $n$ outcomes for a discrete random variable, say $x_1,x_2,\dots,x_n$ occurring with probabilities $p_1, p_2, \dots, p_n$ respectively, then

$$f(x) = \sum_{i=1}^np_i\delta(x-x_i)$$ Acts like the pdf that we want. That is, $\int_{-\infty}^\infty f(x)dx = 1$ [do you see why?], and $F(x) = \int_{-\infty}^x f(t)dt = P(X\le x). $

There are a few fuzzy details here, however, which is why you probably shouldn't think of a discrete random variable in this way. One is that $f(x_i)$ is not defined for one of the outcomes of the random variable. You lose the ability to work with individual points and must work with small intervals around possible outcomes. This is one of the many reasons why the current theory of probability doesn't use the Riemann integral, which is the traditional integral you learn about in introductory calculus. It allows us to use roughly the same tools to think about continuous and discrete random variables. In this framework, a probability distribution is a way of assigning probability to what are called "measurable" sets in the set of possible outcomes (usually $\Bbb{R}$ or a subset of it). If the random variable is discrete, the measurable sets are just those subsets of all possible outcomes, while for continuous random variables, the situation is more complicated but is well known to behave just like the Riemann integral in most "nice" cases. For the particular example of $x_i$ and $p_i$ described above, which is a generalization of the problem you posed, this probability distribution assigns to the set that contains $x_j$ for $j$ in some index set $J\subset\{1,2,\dots,n\}$, the probability $\sum_{j\in J}p_j$.

guest196883
  • 6,049
  • This is perfectly right. The sum of the probability mass function multiplied by dirac impulses at each point is the total probability. – Rohit Dec 08 '13 at 11:47
  • Yes, but it is (at least to me) somewhat "harmful" to think of it that way. Notably because $P(X = 3)$ in the discrete case is identically $0.1$. In the continuous case, it is $\int_0^0 0.1\delta(x)dx$. This is a monstrosity. Perhaps you could consider $\int_{-\epsilon}^\epsilon 0.1\delta(x)dx$ as $\epsilon \to 0$, but is that really any better? – guest196883 Dec 08 '13 at 11:51
  • However, I am now trying to calculate P{Y|X=-1}/P{Y|X=1}. In the gaussian distribution case, this neatly reduces to 2*y/$\sigma^2$. Is there a similar neat reduction for the i.i.d.? – Rohit Dec 08 '13 at 11:53
  • I'm not sure I entirely understand your question. What is $Y$? And $X$? – guest196883 Dec 08 '13 at 11:57
  • Suppose random variable X is an input to a BPSK channel which can take values $\in {-1,1}$, and gaussian noise is added to it resulting in a random variable Y (y=y+noise). Then at the receiver the log likelihood ratio of the bit is $\log (P{X=-1|Y}/P{X=1|Y})$ which is equal to $\log(P{X=-1} \cdot P{Y|X=-1} \cdot \frac{1}{p(y)} / P{X=1} \cdot P{Y|X=1} \cdot \frac{1}{p(y)})$ which reduces to $\log(P{X=-1}/P{X=1}) + \log (P{Y|X=-1}/P{Y|X=1})$

    Now

     – Rohit Dec 08 '13 at 12:07
  • Now the second term $\log(\frac{P\left{Y|X=-1\right}}{P\left{Y|X=1\right}})$ is equal to $\log((\frac{1}{\sqrt{2 \cdot \pi \cdot \sigma^2}) \cdot e^{(y-(-1)^2/2\sigma^2)})$ reduces to $\frac{2y}{\sigma^2}$. Can we do something similar for i.i.d. I'm thinking not, but wanted to make sure. – Rohit Dec 08 '13 at 12:10
  • Since this is a new question, which deserves a strong response on its own, I suggest you post a new question. I'll be happy to attempt to answer. – guest196883 Dec 08 '13 at 12:15