Meaning of linear independence with row vectors

Question

So far I have understood that a set of vectors $S = {v_1, v_2, . . . , v_k }$ in a vector space V is linearly independent when the vector equation $c_1v_1 + c_2v_2 + . . . + c_kv_k = 0$ has only the trivial solution$c_1 = 0, c_2 = 0, . . . , c_k = 0.$

An example in matrix form is:

$\begin{bmatrix}1 & 1 & 2 & 4 \\ 0 & -1 & -5 & 2 \\ 0 & 0 & -4 & 1 \\ 0 & 0 & 0 & 6 \\ \end{bmatrix} \begin{bmatrix} c_1\\ c_2 \\ c_3 \\ c_4 \\ \end{bmatrix} = \begin{bmatrix} 0\\ 0 \\ 0 \\ 0 \\ \end{bmatrix} $

But a matrix of this form

$\begin{bmatrix}1 & 1 & 2 & 4 \\ 0 & -1 & -5 & 2 \\ 0 & 0 & -4 & 1 \\ 0 & 0 & 0 & 0 \\ \end{bmatrix} \begin{bmatrix} c_1\\ c_2 \\ c_3 \\ c_4 \\ \end{bmatrix} = \begin{bmatrix} 0\\ 0 \\ 0 \\ 0 \\ \end{bmatrix} $

is linearly dependent because it has more than a trivial solution

However, I am confused about row vectors, specifically the idea that to get a basis for a subspace using row vectors we must put the matrix in reduced row echelon form to find the linearly independent vectors. For example here the accepted answer gives an example of finding a basis with row vectors using this

$\begin{bmatrix}1 & 1 & 2 & 4 \\ 2 & -1 & -5 & 2 \\ 1 & -1 & -4 & 0 \\ 2 & 1 & 1 & 6 \\ \end{bmatrix} \Rightarrow \begin{bmatrix}1 & 1 & 2 & 4 \\ 0 & -3 & -9 & -6 \\ 0 & -2 & -6 & -4 \\ 0 & -1 & -3 & -2 \\ \end{bmatrix} \Rightarrow \begin{bmatrix}1 & 1 & 2 & 4 \\ 0 & -3 & -9 & -6 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ \end{bmatrix}$

and then goes on to say that "Only two of the four original vectors were linearly independent." In what respect are these two vectors linearly independent? This looks exactly like the second example that I gave in which the vectors were dependent because they had more than a trivial solution? Does linear independence with regard to row vectors mean something else? Or does this also only have a trivial solution, and if so, how?

Answer 1

Linear independence is linear independence is linear independence. It's defined entirely independently of matrices. It is, instead, defined in terms of a vector equation, which in finite dimensions, can be turned into a system of linear equations. As such, matrices are an excellent tool to determine linear independence.

The definition of linear independence is precisely what you wrote. We say $v_1, \ldots, v_n$ are linearly independent if the only solution of $$a_1 v_1 + \ldots + a_n v_n = 0 \tag{$\star$}$$ for scalars $a_1, \ldots, a_n$, is the trivial solution $a_1 = a_2 = \ldots = a_n = 0$. That is, no other possible choices of scalars will make the above linear combination into the $0$ vector.

This doesn't matter if they are row vectors, column vectors, or more abstract vectors (such as matrices, functions, graphs on a fixed vertex set, algebraic numbers, etc). All you need to define linear independence is an (abstract) vector space.

For example, the real functions $\sin^2(x)$, $\cos^2(x)$, and the constant function $1$ are not linearly independent because $$2 \cdot \sin^2(x) + 2 \cdot \cos^2(x) - 2 \cdot 1 \equiv 0,$$ i.e. the linear combination is exactly the $0$ function, even though the scalars aren't all $0$.

On the other hand, the functions $\sin^2$ and $\cos^2$ are linearly independent, because, if we assume $$a_1 \sin^2(x) + a_2 \cos^2(x) \equiv 0,$$ that is, is equal to $0$ for all $x$, then trying $x = 0$ yields $$0 = a_1 \sin^2(0) + a_2 \cos^2(0) = a_2$$ and trying $x = \pi/2$ yields $$0 = a_1 \sin^2(\pi/2) + a_2 \cos^2(\pi/2) = a_1.$$ Thus, we logically come to the conclusion that $a_1 = a_2 = 0$, i.e. the functions are linearly independent.

---

So, where do matrices come in? If our vectors belong to $\Bbb{R}^m$ (or $\Bbb{C}^m$, or indeed $\Bbb{F}^m$ where $\Bbb{F}$ is a field), then equation $(\star)$ turns into a system of homogeneous linear equations. When you turn this system of linear equations into a matrix of coefficients, the columns will turn out to be precisely the vectors $v_1, \ldots, v_n$, expressed as column vectors. It doesn't matter whether $v_1, \ldots, v_n$ are expressed originally as column vectors or row vectors! Once you turn them into equations, then a matrix, they will become columns. (You should try this for yourself to convince yourself of this fact.)

So, if you take the rows of a given matrix, and try to figure out (by definition) whether they are linearly independent or not, you'll inevitably end up with these vectors being columns, i.e. you'll get the same matrix, just transposed.

Further, we also get a nice technique for proving linear (in)dependence of vectors in $\Bbb{R}^3$, and pruning them down to a linearly independent set: stick them as columns in a matrix $A$, row reduce to a row-echelon form $B$, and if the $i$th column of $B$ does not have a pivot in it, then the $i$th column of $A$ depends linearly on the previous columns of $A$, and hence can be removed without damaging the span.

If you instead stick the vectors in as rows in a matrix and reduce as above, then this will not tell you which vectors depend on each other, in the same way that the column approach does. However, row operations preserve the span of the row vectors, hence the non-zero rows of a row-echelon form of a matrix will be a basis for the span of your vectors. This basis may have no vectors in common with your original set of vectors, however!

Answer 2

Yes, it is "exactly like the second example" you gave. In that example, you were showing that the original four vectors were dependent. But there will always be some subset of those vectors that is independent. (That subset might consist of a single vector. A set containing just one vector is always "independent".) In this last problem you are looking for that independent subset.

Your "four vectors" are the rows of the matrix (1, 1, 2, 4), (2, -1, -5, 2), (1, -1, -4, 0), and (2, 1, 1, 6). To determine whether they are "independent" or "dependent", you look for numbers, a, b, c, d, such that a(1, 1, 2, 4)+ b(2, -1, -5, 2)+ c(1, -1, -4, 0)+ d(2, 1, 1, 6)= (a+ 2b+ c+ 2d, a- b- c+ d, 2a- 5b- 4c+ d, 4a+ 2b+ 6d)= (0, 0, 0, 0). So we have the four equations, a+ 2b+ c+ 2d= 0, a- b- c+ d= 0, 2a- 5b- 4c+ d= 0, and 4a+ 2b+ 6d= 0. Adding the first two equations eliminates c: 2a+ b+ 3d= 0. But notice that the fourth equation, 4a+ 2b+ 6d= 0, is just two times that so is not really a new equation. If we add the third equation to 4 times the first equation we also eliminate c: 6a+ 3b+ 9d= 0. We can divide by 3 to get 2a+ b+ 3d= 0.

We have two equations, 3a+ b+ 3d= 0 and 2a+ b+ 3d= 0. Subtracting the second of those from the first gives a= 0. But then we have b+ 3d= 0 so b= -3d but d can be anything. Putting a= 0, b= -3d into a+ 2b+ c+ 2d= 0 we have -6d+ c+ 2d= c- 4d= 0 so c= 4d. That means that a(1, 1, 2, 4)+ b(2, -1, -5, 2)+ c(1, -1, -4, 0)+ d(2, 1, 1, 6)= (0, 0, 0, 0) can be written -3d(2, -1, -5, 2)+ 4d(1, -1, -4, 0)+ d(2, 1, 1, 1, 0)= (0, 0, 0, 0). Then d(2, 1, 1, 1 0)= 3d(2, -1, -5, 2)- 4d(1, -1, -4, 0) or, dividing by d, (2, 1, 1, 0)= 3(2, -1, -5, 2)- 4(1, -1, -4, 0). So those 4 vectors can actually be written in terms of these last two. These two vectors form an independent subset of the original four vectors. The space they span is actually two dimensional, not four dimensional. Using "row reduction" with a matrix is just a more convenient way of doing that.