No, it doesn't mean 5 miles as the crow flies.
It means you will be 5 miles before your starting point, measured along the circular track. You have travelled 4 miles along the circular track, so there are 5 more miles left to go until you return to your starting point.
You'll actually be about 3 miles from your starting point, measured as the crow flies.
Suppose we stipulate that travel around the track in the original direction is "positive" and travel in the reverse direction direction is negative. Then, to get back to the starting point, you can either go 5 miles (forward) or -4 miles (backwards). So, in some sense $5$ is equal to $-4$. And, if you are using what mathematicians call "module 9" arithmetic, then, in this system, it is true that $5=-4$, and $4=-5$, and $9=0$, and so on. So, in this strange world of arithmetic module 9, it's true that $2+2=-5$, but it's still not true that $2+2=5$.
Totally irrelevant anecdote: in my youth, I often heard the phrase "as the crow flies", and I thought people were saying "as the crofe lies". I thought there was some long straight object called a crofe that could be layed across the land to indicate a direct route. I looked up "crofe" and "croaf", but, of course, I never found anything.