Let's assume they are around a round table. For each participant X, let $A$ be the guy to his left, $B$ the guy in front, $C$ to his right.
Let's write the numbers on the hats in basis $3$:
$$n_X=x_2*9+x_1*3+x_0.$$
So $X$ says the number $a_0+b_1+c_2 \pmod 3$ (red=0, yellow=1, nothing=2).
He hears from $A$ the number $b_0+c_1+x_2 \pmod 3$. Since he already knows $b_0,c_1$, that tells him $x_2$.
He hears from $B$ the number $c_0+x_1+a_2 \pmod 3$. Since he already knows $c_0,a_2$, that tells him $x_1$.
- He hears from $C$ the number $x_0+a_1+b_2 \pmod 3$. Since he already knows $a_1,b_2$, that tells him $x_0$.