PDF of joint multivariate normal distribution

Question

I'm trying to understand what this means:

$\mathcal{P}(\boldsymbol{\mathrm{s}}|\boldsymbol{\mathrm{r}}, A) = \int_{-\infty}^{\infty} \cdots \int_{-\infty}^{\infty} \mathcal{P}(\boldsymbol{\mathrm{s}},\boldsymbol{\mathrm{p}},\boldsymbol{\mathrm{t}}|\boldsymbol{\mathrm{r}}, A) d\boldsymbol{\mathrm{p}} d\boldsymbol{\mathrm{t}}$ which appears in section 2, on page 3 of this paper:

https://www.microsoft.com/en-us/research/publication/trueskilltm-a-bayesian-skill-rating-system/

I take it from the text that $\boldsymbol{\mathrm{s}}$, $\boldsymbol{\mathrm{p}}$ and $\boldsymbol{\mathrm{t}}$ are vectors of normally distributed random variables thusly (for a game involving two teams of two):

$\boldsymbol{\mathrm{s}} = \left[ \begin{array}{c} S_1 \\ S_2 \\ S_3 \\ S_4 \end{array} \right]$ $\boldsymbol{\mathrm{p}} = \left[ \begin{array}{c} P_1 \\ P_2 \\ P_3 \\ P_4 \end{array} \right]$ $\boldsymbol{\mathrm{t}} = \left[ \begin{array}{c} T_1 \\ T_2 \end{array} \right]$

And further that these vectors contain random variables that follow Normal distributions and are tightly inter-related as follows:

$S_i \sim \mathcal{N}(s_i | \mu^2, \sigma^2) \implies \mathcal{P}(s_i) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(s_i-\mu)^2}{2\sigma^2}}$ $\begin{align*} P_i \sim \mathcal{N}(p_i|s_i,\beta^2) &\implies \mathcal{P}(p_i) = \frac{1}{\sqrt{2\pi\beta^2}} e^{-\frac{(p_i-s_i)^2}{2\beta^2}}\\ &\implies \mathcal{P}(p_i) = \frac{1}{\sqrt{2\pi(\sigma^2+\beta^2)}} e^{-\frac{(p_i-\mu)^2}{2 (\sigma^2 +\beta^2)}} \end{align*}$ $\begin{align*} T_j = \sum_{i=1}^{n_j} \omega_{ij} P_{ij} &\implies T_j \sim \mathcal{N}(t_j|\sum_{i=1}^{n_j}\omega_{ij}\mu_{ij}, \sum_{i=1}^{n_j}\omega_{ij}^2(\sigma_{ij}^2+\beta^2))\\ &\implies \mathcal{P}(t_i) = \frac {1} {\sqrt{2\pi\sum_{i=1}^{n_j}\omega_{ij}^2(\sigma_{ij}^2+\beta^2)}} e^{-\frac{(t_i-\sum_{i=1}^{n_j}\omega_{ij}\mu_{ij})^2}{2 \sum_{i=1}^{n_j}\omega_{ij}^2(\sigma_{ij}^2+\beta^2)}} \end{align*}$

that the matrix $A$ is simply some assignment of the players (so $S$ and $P$) to teams (so $T$) and $r$ remains a mystery for now in this context.

Taking things one step at a time and having read up at length on various sources about multivariate distributions and on joint distributions, and having looked for material on joint multivariate distributions I find myself stuck interpreting this:

$\mathcal{P}(\boldsymbol{\mathrm{s}},\boldsymbol{\mathrm{p}},\boldsymbol{\mathrm{t}}|\boldsymbol{\mathrm{r}}, A)$

which is clearly a joint multivariate distribution. What I'm struggling to understand is what that looks like. Set aside $r$ and $A$ for now and consider just the joint multivariate distribution $\mathcal{P}(\boldsymbol{\mathrm{s}},\boldsymbol{\mathrm{p}},\boldsymbol{\mathrm{t}})$, is that a vector of probabilities, a matrix of probabilities, a scalar function of those three vectors or what? And what does it look like in the exemplar form, for the aforementioned game between 2 teams of 2 players.

It is an understanding of the nomenclature and what it means that I am struggling with I admit as none of the literature I've at hand or found on-line which deals wonderfully with multivariate distributions and well with joint distributions, has provided me with a concrete example of a joint multivariate distribution.

Of course, there is a chance I am misinterpreting things thus far too. But the paper is strong on assumed understanding of the nomenclature and I am using what I think are fairly normal interpretations thus far. Just a tad stumped on what this: $\mathcal{P}(\boldsymbol{\mathrm{s}},\boldsymbol{\mathrm{p}},\boldsymbol{\mathrm{t}})$ looks like and is in the exemplar form rather than the abstract form.

In any case to make sense of the integrals I would love to understand the integrand first though I don't rightly understand what the \cdots between the two integrals means and I wonder if the implication is that one integrates with respect $p_1$ then $p_2$ etc through to $t_2$, but I can make little sense of any of that until I can digest what this integrand looks like.

Answer 1

the symbol $\mathcal P$ in your equation stands for a probability density function. In this case, the arguments are vectors. If you want, you can see it as a function that takes in three arguments ($\mathbf s$, $\mathbf p$, and $\mathbf t$) and spits out the density of that triple.

As for the shape of the joint density, it is probably not possible to write it (e.g., I'm missing some details about $\omega$). However, you can use the law of conditional probabilities to say that $\mathcal P(\mathbf s,\mathbf p ) = \mathcal P(\mathbf p|\mathbf s )\mathcal P (\mathbf s)$ (go on from there to complete the full joint distirbution).

As for the integral, that just means that the authors marginalize $\mathbf p$ and $\mathbf t$ to find the density of $\mathbf s$ alone (you could in fact do as you wrote; i.e. integrate $s_1$, then $s_2$, and so on). In some way, you can interpret it as saying that you are looking for the distribution of the skill $\mathbf s$ of the players given the observed ranking $\mathbf r$ and assignment $A$ of players into teams. To find this density, you would need the performance of the players $\mathbf p$ and the performance of the teams $\mathbf t$; because these last two are not observed (they are latent variables) you integrate them out to average their influence over all the possible values.

In general, in Bayesian modeling, models are constructed as hierarchies of random variables that depend on other random variables until they define a big joint multivariate model; then the the model is conditioned on the observed variables and the latent variables are integrated out (marginalized, as in your example are $\mathbf p$ and $\mathbf t$). In general, it is not possible to solve the integrals of the models and these are approximated. In the paper you pointed to, the authors use some variant of expectation propagation.