The lay concept of symmetry has a lot to do with the idea of a "mirror image". The capital letter 'A' has symmetry across its vertical axis because its left side is a mirror image of its right side. The location of the mirror is that vertical axis (or some vertical line).
This means that, if you print your letter 'A' backwards, nobody will know. A backwards 'A' is the same as a normal 'A'.
The capital letter 'B' has a different symmetry, because if it's backwards, you can tell, but if it's upside-down, you can't. Upside-down 'B' is the same as normal 'B'.
Capital letters such as 'N' and 'S' and 'Z' have a different kind of symmetry, because if you rotate them by 180 degrees, you can't tell that anything changed.
This symmetry does not correspond to a "mirror image" defined across some line or plane, and yet, we call such transformations "reflection through the origin". Reflection through a point has very little to do with mirrors.
Other symmetries that don't correspond to mirror images are translation symmetries. If you take the graph $y=\sin(x)$, and shift it to the right $2\pi$ units, you can't tell the difference. Of course, that same graph is also symmetric under a reflection through the origin, like the letter 'Z', but that's a different issue.
Translation symmetry also applies to an infinitely extended line of any symbol: ".....RRRRRRRR.....". If you shift that line of 'R's one character to the right, nobody will know, so that's a "symmetry".
Consider Noether's Theorem: The fact that the laws of physics stay the same, as time unwinds, corresponds to a translation symmetry in the time direction. Performing a physics experiment today will (presumably) produce the same results as if we were to perform it tomorrow.
All of this is to say that, YES, the lay idea of symmetry is a subset of the mathematical idea. We call objects "symmetric" when they (approximately) have a reflection symmetry across some axis or plane.
The mathematical idea is that a symmetry is any change you can make – perhaps a reflection, but often something else – that results in your original object appearing to be unchanged. For example, the expression $x+y$ is symmetric under swapping the variables $x$ and $y$, while the expression $x+2y$ is not.