I am familiar with solving matrix equations by multiplying by the inverse of a matrix (if it exists), but how can I solve a matrix equation for $A$, if it contains $A^TA$?
As a specific example:
Solve for A: $$A^TA = VDV^T$$
(Note: $VDV^T$ is an eigendecomposition, so $D$ is diagonal. I'm not sure if that's useful.)
Without further information about $A$, it cannot be done. The solution is not unique.
However, if $D\in\mathbb{R}^{n\times n}$ is positive semidefinite (it should be, under the circumstances), and $V\in\mathbb{R}^{p\times n}$, we can say that $A=UD^{1/2}V^T$, where $D^{1/2}$ is a square root of $D$ and $U\in\mathbb{R}^{m\times n}$ is any matrix satisfying $m\geq n$ and $U^TU=I$.
It looks to me that this equation came from a discussion of the singular value decomposition. Because you say that $VDV^T$ is an eigendecomposition, the diagonal elements of $D$ are actually the squares of the singular values of $A$, and the columns of $V$ are right singular vectors of $A$. The corresponding equation $AA^T=UDU^T$ would give you the left singular vectors and the same squared singular values.
A full singular value decomposition of $A\in\mathbb{R}^{m\times n}$ looks like $A=U\Sigma V^T$, where $U^TU=I$, $V^TV=I$, and $\Sigma$ is a diagonal matrix with positive entries (though not necessarily square!). In this case, then $D_{ii}=\Sigma_{ii}^2$, $i=1,2\dots,\min\{m,n\}$.
In general this will not be unique because even under the friendly condition that $A$ is symmetric, then we could get non-unique answers for $A$. For example,
$$ \begin{bmatrix} -1 & 0 \\ 0 &-1 \end{bmatrix} \begin{bmatrix} -1 & 0 \\ 0 &-1 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 &1 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 &1 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 &-1 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 &-1 \end{bmatrix} = \begin{bmatrix} -1 & 0 \\ 0 &1 \end{bmatrix} \begin{bmatrix} -1 & 0 \\ 0 &1 \end{bmatrix} $$
So they all solve $$A^TA=VDV^T$$
In the specific case you mention, let $D_0$ be some diagonal matrix such that $D_0^2 = D$. Then A = $D_0 V^T$ is a solution. I suppose that this suggests some sort of general approach: get $A^T A$ on one side of the equation, and then try to break the other side into a matrix and its transpose. I can imagine this being hard in general, though in your particular question it is very easy.
Another approach would be to use the fact that you can decompose a symmetric matrix as $U^T D U$ where U is unitary and D is diagonal and note that $A^TA$ is symmetric. This would again, not likely resolve all questions of this nature.
