How do you extend a basis?

Question

I recently posted a question about this; now that I've acquired some new info, I have some follow up questions about extending a basis (I'm not too sure if this is actually the name for it, so my apologies if it isn't.)

Let's say I have a set $\{(1,1,1)\}$ and wanted to add vectors such that the set is a basis of $\mathbb R^3$. Tt's rather obvious that I could easily find a couple of vectors that would make the set linearly dependent, but I'm looking for a method that I could depend on.

I've seen a couple of ways of doing this, neither of which I understand fully.

The first is using dot products. Let's call the vectors I want to create $a,b \in \mathbb R$. I want an $a$ and a $b$ such that $a \cdot (1,1,1) = 0$ and that $b \cdot (1,1,1) = 0$. If I've understood things correctly, this is because if the dot product of two vectors is $0$, then they are orthogonal. If $a$ and $b$ are both orthogonal to $(1,1,1)$, then $\{(1,1,1), a, b)\}$ is linearly independent, and therefore a basis on $\mathbb R$$^3$. Is that all true?

If so, then I suppose I understand this solution conceptually, but I'm not sure how to go about it otherwise.

The second method I've found is by creating a matrix $A$ out of my vectors, create a basis of the nullspace $N(A)$, and then go from there. Honestly, I don't understand much of anything here. I know how to create a matrix from my 3 vectors (2 of which are unknown). I also understand that the nullspace of A is the set of vectors such that, if multiplied with $A$, the product is the zero vector (if this is also wrong, please correct me.) I also understand that if my matrix is built of linearly independent vectors, the nullspace is simply the zero vector. However, I don't understand what it means to create a basis of the nullspace, and thus don't know how to carry on.

The last thing that bothers me is, I don't see how these two methods are any more powerful/helpful than proving that having any linear combination of the vectors equaling zero implies that the scalars multiplying the vectors are also zero.

Any help is greatly appreciated, thank you.

Answer 1

A simple approach is to take any basis $\{b_1, b_2, b_3\}$ for $\mathbb{R}^3$ and consider the set $\{(1,1,1), b_1, b_2, b_3\}$.

If $b_1$ is proportional to $(1,1,1)$, discard it. If not, keep it.

If $b_2$ is a linear combination of $\{(1,1,1), b_1\}$ then discard it. Otherwise keep it.

If $b_3$ is a linear combination of $\{(1,1,1), b_1, b_2\}$ then discard it. Otherwise keep it.

You will arrive at a three-element set containing $(1,1,1)$ which is a basis for $\mathbb{R}^3$. This is because the resulting set is linearly independent by construction, and discarding linearly dependent elements in each step does not reduce the span.

To illustrate, take the canonical basis and consider $\{(1,1,1), (1,0,0), (0,1,0), (0,0,1)\}$.

We see that the only element we need to discard is $(0,0,1)$ so the resulting basis is $\{(1,1,1), (1,0,0), (0,1,0)\}$.

Answer 2

"Find a couple vectors" is very dependable. Nearly any choice of vectors you make will work. You almost have to actively try to fail with this method; the only real obstacle is if you restrict your choices to very special forms, and some of those forms happen to be in the span of the previous vectors.

However, there is a very simple way forward that is guaranteed. A basis for $\mathbb{R}^n$ is the same thing as an $n \times n$ matrix whose rows are linearly independent; you can take the rows for a basis.

(Aside: while it's usually more natural to treat vectors of $\mathbb{R}^n$ as columns, switching to rows here doesn't real;y change anything. I only do so because doing row operations as a matrix is a more familiar procedure than doing column operations)

There is a family of matrices that are extremely easy to identify as having linearly independent rows: those in row echelon form.

$$\left( \begin{matrix} 1 & 1 & 1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} \right) \\ {}_{\text{These rows are clearly independent}}$$

If you are given several vectors as a starting point that cannot be immediately filled into a row echelon matrix, that is easy to repair: do row operations to put your given vectors in row echelon form, and then fill in the rest of the matrix.

Answer 3

In $\mathbb{R}^3$ it is easy. The second independent vector may be obtained e.g. by replacing non-zero coordinate by zero. Next you can take the vector product, which is even orthogonal to the first two vectors.

Answer 4

First, if $\mathbf a$ and $\mathbf b$ are both orthogonal to some given vector $\mathbf c$, it does not follow that $\{\mathbf a,\mathbf b,\mathbf c\}$ is linearly independent. The pair $\{\mathbf a,\mathbf b\}$ must itself be linearly independent, too.

Anyway, the second method that you describe is exactly how one might go about finding a linearly independent set of vectors orthogonal to the ones you’re given: the null space of a matrix is the orthogonal complement of its row space, so if you assemble the given vectors into a matrix and find a basis for its null space, you end up with a linearly independent set of vectors that, together with the original set, span the entire parent space. The most common way to perform this computation is by reducing the matrix to echelon form. Row operations leave the row space of a matrix unchanged, to the two will have the same row space, and hence also the same null space. As Hurkyl describes in his answer, once you have the matrix in echelon form, it’s much easier to pick additional basis vectors. A systematic way to do so is described here.

To see the connection, expand the equation $\mathbf v\cdot\mathbf x = 0$ in terms of coordinates: $$v_1x_1+v_2x_2+\cdots+v_nx_n=0.$$ Since $\mathbf v$ is a given fixed vector all of the $v_i$ are constant, so that this dot product equation is just a homogeneous linear equation in the coordinates of $\mathbf x$. If you have a fixed set $\{\mathbf v_1,\dots,\mathbf v_m\}$, you have the system of homogeneous linear equations $$\begin{align} \mathbf v_1\cdot\mathbf x &=0 \\ \mathbf v_2\cdot\mathbf x &=0 \\ \vdots \\ \mathbf v_m\cdot\mathbf x &=0 \end{align}$$ which can be written in matrix form as $$\begin{bmatrix}\mathbf v_1^T\\\mathbf v_2^T\\\vdots\\\mathbf v_m^T\end{bmatrix}\mathbf x = 0.$$ The solution set is, as you already know, precisely the null space of the coefficient matrix on the left.

As a final note, although it can be convenient to do so, you don’t need to use vectors in the orthogonal complement of a subspace to extend its basis. Any set of linearly independent vectors that don’t lie in the subspace will do. Quite often in contrived exercises, by examining where the given vectors have zero coordinates you can guess at some standard basis vectors or simple combinations of them that might work.