Say you have a Gamma distribution with parameters $\alpha$ known and $\theta$ unknown. Find a transformation of $\theta$, $\eta=g(\theta)$ such that ${\cal I} (\eta)$, the Fisher Information, is constant in terms of $\eta$.
My problem is that the only way I can do this is by trial and error. At the moment I haven't been able to come up with any transformations that make this work. Is there a method that this can be done?
What this is asking you for is the normalizing transform, since the fisher information is the second derivative of the log likelihood, the transformation must result in a quadratic log-likelihood in $\eta$.
Take a look at Section 8.4 of this link, especially p.270. Has what you are looking for. I will use that formula in the below:
$\mathcal{I}(\eta)=\frac{\mathcal{I}(\theta)}{(\partial\eta/\partial\theta)^2}$ [From Pawitan 2001, p.270]
Re-state goal mathematically:
$\frac{\partial\mathcal{I}(\eta)}{\partial\eta}=0=\frac{\partial}{\partial\eta}[\frac{\mathcal{I}(\theta)}{(\partial\eta/\partial\theta)^2}]=\frac{\partial}{\partial\eta}[\mathcal{I}(\theta){(\partial\theta/\partial\eta)^2}]=2\mathcal{I}(\theta)(\partial\theta/\partial\eta)\frac{\partial^2\theta}{\partial\eta^2}+(\partial\theta/\partial\eta)^2\frac{\partial\mathcal{I}(\theta)}{\partial\theta}\frac{\partial\theta}{\partial\eta}$
Divide out the common $\partial\theta/\partial\eta$ to get the nonlinear differential equation: $2\mathcal{I}(\theta)\frac{\partial^2\theta}{\partial\eta^2}+(\frac{\partial\theta}{\partial\eta})^2\frac{\partial\mathcal{I}(\theta)}{\partial\theta}=0$
Take reciprocals of each derivative to get a non-linear PDE in terms of $\eta(\theta)$:
$-2\mathcal{I}(\theta)(\frac{\partial^2\eta}{\partial\theta^2})(\frac{\partial \eta}{\partial \theta})^{-3}+(\frac{\partial\eta}{\partial\theta})^{-2}\frac{\partial\mathcal{I}(\theta)}{\partial\theta}=0$
Simplyfing we get:
$\frac{\partial\mathcal{I}(\theta)}{\partial\theta}(2\mathcal{I}(\theta))^{-1}=(\frac{\partial^2\eta}{\partial\theta^2})(\frac{\partial \eta}{\partial \theta})^{-1}$
You'll need to solve this PDE for your specific problem to get the answer. Not a trivial task.
In fact, it is a trivial task ! Denoting by prime the derivative with respect to $\theta$, the differential equation writes \begin{equation} \frac{\mathcal{I}^\prime(\theta)}{2 \mathcal{I}(\theta)} = \frac{\eta^{\prime\prime}(\theta)}{\eta^{\prime}(\theta)}. \end{equation} Integration is straightforward \begin{equation} \log(\eta^{\prime}(\theta))= \frac{1}{2}\log(\mathcal{I}(\theta)) + \text{cst} \end{equation} which leads to \begin{equation} \eta(\theta) \propto \int_0^\theta \sqrt{\mathcal{I}(t)} \mathrm{d}t + \text{cst}. \end{equation}
