The answer is essentially "yes". More precisely, $\text{Hol}(D)$ is only well-defined up to conjugation in $G$ and the statement is that if $D$ restricts to some $H$ subbundle $Q\subset P$ then $\text{Hol}(D)$ is the conjugacy class of a subgroup of $H$.
This is an immediate consequence of the definitions of the terms involved (the Theorem of Ambrose-Singer is not necessary). Here is a quick sketch of the argument (mostly uncovering definitions).
A connection $D$ on a prinicipal $G$-bundle $\pi:P\to M$ is given by a distribution $\mathcal H\subset TP$, $G$-invariant and transversal to the fibers of $P\to M$.
The holonomy of $D$ is defined by picking a base point $p\in P$, then for each closed curve $\gamma:[0,1]\to M$ such that $\gamma(0)=\gamma(1)=\pi(p)$, you first lift it horizontally to the unique curve $\tilde\gamma:[0,1]\to P$ such that $\gamma=\pi\circ\tilde\gamma$, $\tilde\gamma(0)=p$ and ${d\over dt}\tilde\gamma\in\mathcal H$, then its holonomy is defined to be the element $h\in G$ such that $\tilde\gamma(1)=\tilde\gamma(0)h.$ Then $\text{Hol}(D,p)\subset G$ is the set of all holonomies of all such $\gamma$.
If you switch to another base point instead of $p$ then it follows immediately from the $G$-invariance of $\mathcal H$ that $\text{Hol}(D,p)$ gets conjugated, hence $\text{Hol}(D)$ is only well defined up to conjugation.
If $Q\subset P$ is an $H$-subbundle, where $H\subset G$ is a subgroup, then "$D$ restricts to $Q$" means that $\mathcal H$ is tangent to $Q$, i.e. $\mathcal H|_Q\subset TQ$. In particular, if you pick a base point $q\in Q$, all lifted $\tilde\gamma$ remain in $Q$ hence $\text{Hol}(D,q)\subset H$.