I think some comparison with spherical geometry can help. What is the distance between e.g. two opposite points on the sphere? Well, on the one hand you can say that the arc length for that distance is equal to $\pi r$ so it depends on the size of the sphere. On the other hand you can say that the two rays that go from center to one the poles go in opposite directions, so they span an angle of $\pi$ or $180°$. In this sense, the angle is a scale-invariant form of expressing that distance: it doesn't depend on the radius. Multiply by the radius to get the actual distance along the surface.
And if you pick the unit sphere $r=1$ (either by scaling your sphere or by using its radius as your unit of distance measurement) then angle and distance become the same number, and you can use them interchangeably.
Note that this depends on the convention of radians, namely that a full turn is $2\pi$. Someone might reasonably have decided to measure angles in fractions of a full turn. In that case, a radius of $r=\frac{1}{2\pi}$ would make angles and distances match. So it's all down to convention, since by their nature angle and distance are distinct concepts. Blue's comment goes into depth about why our normal convention is probably better than this hypothetical one here, but while some conventions might be more convenient than others, that doesn't make the less convenient ones wrong.
How does this relate to curvature? The Gaussian curvature of a sphere is $r^{-2}$. So the relationship between scale-invariant angle and scale-dependent length is the inverse square root of the curvature.
The same general idea holds for hyperbolic geometry. You can have a scale-invariant representation for some things, e.g. angles. You can have scale-dependent measurements for other things, e.g. distances. And the relationship between the two would again be proportional to the inverse square root of the Gaussian curvature. Since the Gaussian curvature of the hyperbolic plane is negative, $-1$ is in many ways the most natural choice to aim for. In a certain way that corresponds to a sphere of radius $r=i$, the imaginary unit.
For the sphere I explained how the rays at the center of the sphere form a scale-invariant measurement. It's hard to imagine an intuitive counterpart to this in the hyperbolic plane. But there is another connection between angles and distances that works for both sphere and hyperbolic plane: you can measure the area of a polygon by looking at the angle sum. In spherical geometry, the area of a polygon is proportional to the angle surplus: the amount of angle in excess of what you'd have in the Euclidean case. In hyperbolic geometry, the area is proportional to the angle deficit: the amount less than in the Euclidean case. And if you make the Gaussian curvature have absolute value one, then the factor of proportionality becomes one in both cases. The area (as e.g. obtained by some double integral over area element) matches the angle excess or deficit exactly.
So for this reason, choosing your unit of distance measurement such that the Gaussian curvature of the hyperbolic plane becomes $-1$ is most natural in many setups and simplifies a lot of calculations. But it is a convenient convention, not some inherent truth.
Side note: I recall writing an answer where curvature $-4$ would have been a reasonable outcome from one of the formulas. So the $-1$ is my no means universal.