The vector addition is a nice solution, but if you care about algorithmic complexity the square roots (in the norm) and divisions are not ideal.
Using your nomenclature of known angles $\alpha$, $\beta$ and unknown angle $c$, and assuming all three angles are from the same reference (eg. the x-axis), it feels like it should be as simple and taking the average of $\alpha$ and $\beta$. But this fails in two ways:
- If the angles are not numerically within $360^\circ$ of each other, the average no longer works as an angle average.
- Sometimes the average points C in the opposite direction. In other words, sometimes the angle is in the middle of the reflex angle, not "between" A and B in the usual sense.
Counter-intuitively, the best way I found to solve robustly this is to introduce the difference operation as inspired by this answer. To wit:
- Find the difference $\delta = \beta - \alpha$.
- While $\delta > 180°$, subtract $360^\circ$.
- While $\delta < -180°$, add $360^\circ$.
- $c = \alpha + \delta / 2$
In steps 2 and 3 (of which only zero or one of which will take effect), both failures 1 and 2 are taken care of by performing a modulo operation, but also by making sure we only consider the "inner" angle. The last step miraculously takes care of direction of rotation.
Some examples:
- $\alpha = 20°, \beta = -10° \implies \delta = -30° \implies c = 20° - 15° = 5°$
- $\alpha = -10°, \beta = 20° \implies \delta = 30° \implies c = -10° + 15° = 5°$
- $\alpha = 20^\circ, \beta = 350^\circ \implies \delta = 330^\circ, -30^\circ \implies c = 20^\circ - 15^\circ = 5^\circ$
- $\alpha = 190^\circ, \beta = -190^\circ \implies \delta = -380^\circ, -20^\circ \implies c = 190^\circ - 10^\circ = 180^\circ$
- $\alpha = -80^\circ, \beta = 360^\circ+260^\circ \implies \delta = 700^\circ, 340^\circ, -20^\circ \implies c = -80^\circ - 10^\circ = -90^\circ$
Addendum: I actually devised this algorithm to find the angle of the "apex" of a vertex. That is, given a vector heading into a vertex and a vector heading out, what is the angle of the vector halfway around the vertex? So for me, direction was important, not inner/outer. But they turn out to be equivalent!