A string is a sequence of symbols. Depending on the context, this might mean exclusively finite sequences or infinite sequences might be permitted.
An $X$ over a set is an $X$ whose "elements" are taken from that set ... what exactly the elements are depends on the context.
Let $\mathbb{N}$ be the numbers $1, 2, 3 \cdots$.
If $\Sigma$ is an alphabet and $\Sigma^*$ is the set of strings over $\Sigma$, then each element of $\Sigma^*$ is a total function from $\{x : x \in \mathbb{N} \;\text{and}\; 1 \le x \le n \}$ to $\Sigma$, where $n \in \mathbb{N} \cup \{\infty\}$ and the string in question has length $n$.
Your next question raises a subtle point.
A language $L$ is typically defined as a subset of $\Sigma^*$. The reason why $L$ is a subset in general is that it lets you rule out ill-formed strings. For example $((((x+$ is ill-formed in the language of arithmetic, but $(x+1)$ is not.
$L$ sometimes has a fairly natural ordering called the lexicographic ordering. If $\Sigma$ itself is ordered, then we can order any two strings relative to each other by comparing their elements one by one and arbitrarily decreeing that the absence of an element is less than any given element, so $ab$ would come before $aba$ in lexicographic order. However, the lexicographic ordering does not give you a sequence in general, for example these strings are in lexicographic order, but are not order-isomorphic to $\mathbb{N}$.
$$ a, aa, aaa, a^4, a^5, \cdots, b $$
Because of issues like this, it's convenient not to insist up front that all the strings of a given language be possible to arrange into a sequence.