What does calculating the inverse of a matrix mean?

Question

Assume I have 3 equations, $x+2y+z=2, 3x+8y+z=12, 4y+z=2$ which could be represented in matrix form ($Ax = b$) like this: $\begin{pmatrix} 1 & 2 & 1\\ 3 & 8 & 1\\ 0 & 4 & 1 \end{pmatrix}\bigl .\begin{pmatrix} x\\ y\\ z \end{pmatrix} = \begin{pmatrix} 2\\ 12\\ 2 \end{pmatrix}$ Then, the inverse of $A$, $A^{-1}$, would be: $\begin{pmatrix} 2/5 & 1/5 & -3/5\\ -3/10 & 1/10 & 1/5\\ 6/5 & -2/5 & 1/5 \end{pmatrix} $. So, my question is, what does this even mean? We know that $A$ is a coefficients matrix that represents the 3 equations above, so what does $A^{-1}$ mean with respect to these 3 equations? What have I done to the 3 equations is exactly my question.

Please note that I understand very well how to find the inverse of a matrix, I just don't understand the intuition of what's happening and sort of the meaning of the manipulations I am applying to the equations when they are in matrix form.

Answer 1

Matrix multiplication corresponds to substituting new variables for the given ones in the system of linear equations. In more detail, for a system of $n$ equations in $n$ unknowns $X_1,\dots ,X_n $, suppose that $A$ represents the system of equations. Suppose now that you introduce new variables $Y_1,\dots ,Y_n$ and you express each $X_i$ as a linear combination of the new variables. If you write $B$ for the matrix of coefficients of the $X_i$ represented as combinations of the $Y_i$, then the matrix $AB$ corresponds to the matrix of coefficients of the original system of equations after substituting the new variables in. If you work this out for the case $n=2$ it's easy to see what is going on. This in fact is one way to motivate the definition of matrix multiplication (in general, not just for square matrices).

Now, what all this tells you is that if you have $A$ and you found that $B=A^{-1}$ is its inverse, then if you introduce new variables $Y_1,\dots , Y_n$ and express the $X_i$ in terms of those by reading the coefficient in the inverse matrix $B$, then substituting these variables into the original system will result in a very very simple system. Namely, the coefficient after substituting will be the coefficients in $AB=I$. This is the simplest system in the world. So, find the inverse of a matrix is equivalent to finding a change of coordinates, from the $X_i$'s to the $Y_i$'s, which make the system of equations particularly nice.

Again, this holds true for all systems, not just $n\times n$.

Answer 2

Let's look at your original three equations.

$$x+2y+z=2$$ $$3x+8y+z=12$$ $$4y+z=2$$

---

Now let's multiply by $\frac{2}{5}$, $\frac{1}{5}$, and $\frac{-3}{5}$ respectively. We get

$$\frac{2x}{5}+\frac{4y}{5}+\frac{2z}{5}=\frac{4}{5}$$ $$\frac{3x}{5}+\frac{8y}{5}+\frac{z}{5}=\frac{12}{5}$$ $$\frac{-12y}{5}+\frac{-3z}{5}=\frac{-6}{5}$$

Now add the three equations together. We get

$$x + 0y + 0z = 2$$

or

$$x=2$$

---

Now multiply the same three equations by $\frac{-3}{10}$, $\frac{1}{10}$, and $\frac{1}{5}$ respectively. We get

$$\frac{-3x}{10}+\frac{-6y}{10}+\frac{-3z}{10}=\frac{-6}{10}$$ $$\frac{3x}{10}+\frac{8y}{10}+\frac{z}{10}=\frac{12}{10}$$ $$\frac{4y}{5}+\frac{z}{5}=\frac{2}{5}$$

Summing

$$0x + y + 0z = \frac{10}{10}$$

or

$$y = 1$$

---

Now multiply the same three equations by $\frac{6}{5}$, $\frac{-2}{5}$, and $\frac{1}{5}$ respectively. We get

$$\frac{6x}{5}+\frac{12y}{5}+\frac{6z}{5}=\frac{12}{5}$$ $$\frac{-6x}{5}+\frac{-16y}{5}+\frac{-2z}{5}=\frac{-24}{5}$$ $$\frac{4y}{5}+\frac{z}{5}=\frac{2}{5}$$

Summing

$$0x + 0y + z = \frac{-10}{5}$$

or

$$z = -2$$

---

And if you look at the numbers by which we multiplied, they are from

$$\begin{pmatrix} 2/5 & 1/5 & -3/5\\ -3/10 & 1/10 & 1/5\\ 6/5 & -2/5 & 1/5 \end{pmatrix} = A^{-1}$$

We essentially did the matrix multiplication of $A \cdot A^{-1}$ to get $I$ manually when we could have just done

$$\begin{pmatrix} x\\ y\\ z \end{pmatrix} = \begin{pmatrix} 2/5 & 1/5 & -3/5\\ -3/10 & 1/10 & 1/5\\ 6/5 & -2/5 & 1/5 \end{pmatrix}\begin{pmatrix} 2\\ 12\\ 2 \end{pmatrix}=\begin{pmatrix} 2\\ 1\\ -2 \end{pmatrix}$$

and gotten the same answer. $A^{-1}$ is essentially the numbers by which we multiply the equations so we can add them together and get the solutions. Solving for the inverse is determining those numbers. Of course, if you're working with the equations, it would be easier to substitute in than to come up with all nine numbers. The convenient thing here is that we don't have to multiply $A\cdot A^{-1}$, as we already know the result. We can just do the right side multiplication.

Answer 3

Let us organize our equations in this way:

$x\begin {bmatrix} 1 \\ 3 \\ \vdots \\ 0 \end{bmatrix}$ + $y\begin {bmatrix} 2 \\ 8 \\ \vdots \\ 4 \end{bmatrix}$ + $z\begin {bmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{bmatrix}$ = $\begin {bmatrix} 2 \\ 12 \\ \vdots \\ 2 \end{bmatrix}$

When you have done inverse we got:

$2\begin {bmatrix} 2/5 \\ -3/10 \\ \vdots \\ 6/5 \end{bmatrix}$ + $12\begin {bmatrix} 1/5 \\ 1/10 \\ \vdots \\ 2/5 \end{bmatrix}$ + $2\begin {bmatrix} -3/5 \\ 1/5 \\ \vdots \\ 1/5 \end{bmatrix}$ = $\begin {bmatrix} x \\ y \\ \vdots \\ z \end{bmatrix}$

Initially on right side we had $[2 \ 12 \ 2 ]^T$. Using inverse, we want to place $[x \ y \ z ]^T$ on right side.

An another way is to think $[2 \ 12 \ 2 ]^T$ as a point represented using three vectors $[1 \ 3 \ 0 ]^T$, $[2 \ 8 \ 4 ]^T$, $[1 \ 1 \ 1 ]^T$. $x,y,z$ was the scaling factors. Now we transformed same point into $[x \ y \ z ]^T$ using vectors associated with column vectors of $A^{-1}$.

Answer 4

It depends on the method you use to calculate the inverse. For instance, if you use the LU decomposition. Then given the matrix $A$ we have

$$ \underbrace{L_{m-1}, \cdots L_{2} ,L_{1}}_{L^{-1}} A = U \tag{1} $$

that is $A = LU$ is composed of a lower and upper triangular matrix. The lower triangular $L$ is a record you use to keep track of eliminating the entries to make the $U$ matrix, or reduced echelon form (sometimes). It is simply a ratio of the rows. When you take the inverse of either you will end up with a lower triangular or upper triangular matrix again. In either event it meant such that

$$ A = LU \implies A^{-1}A = (LU)^{-1}(LU) = U^{-1}L^{-1}LU = I \tag{2}$$

Technically if you are attempting to find that vector you are using two steps back sub and front sub.

If we have the $Ax=b$ problem we have

$$ LUx=b \tag{3}$$

we get two problems

$$ Ly =b \tag{4}$$

$$ y_{1} = \frac{b_{1}}{l_{11}} \tag{5} $$ $$ y_{i} = \frac{1}{l_{ii}} \bigg( b_{i} -\sum_{j=1}^{i-1} l_{ij} y_{j} \bigg) \tag{6} $$ is front sub

$$ Ux = y \tag{7} $$

and back sub is

$$x_{i} = \frac{1}{u_{ii}} \bigg( y_{i} - \sum_{j=i+1}^{N} u_{ij} x_{j} \bigg) \tag{8} $$

The intuition depends on the matrix decomposition. I'd hesitate to call the vector you get a linear combination of the rows or columns because while true it is meaningless.

Answer 5

There are two possible views on what you are doing when computing the inverse of a matrix.

One point of view is the one that edo already mentioned in his answer: The matrix can be imagined as transforming a point in 3D space. For example, you could fill the matrix with the values that describe a rotation about one axis in 3D, like the rotation about $45°$ around the x-axis. Computing the inverse of such a matrix then just describes the inverse transformation. In this case, this would be the rotation about $-45°$ around the x-axis. The inverse lets you "undo" the operation that was done with the original matrix.

Now, this somehow obfuscates the point of the underlying equations. So the other point of view is to describe what you're actually doing with this matrix when it describes a system of equations:

When you have a system of equations expressed as $Ax=b$, you're looking for the vector $x$ that "makes this expression true". You could now use different approaches to find this vector. For example, some sort of iterative approximation, elimination, or even trial-and-error - it does not matter.

The point is:

• For one vector $b = (4,3,9)$, you could find the vector $x = (3.2, 4.5, 2.3)$
• For one vector $b = (1,8,4)$, you could find the vector $x = (2.1, 7.5, 1.5)$
• For one vector $b = (6,1,5)$, you could find the vector $x = (1.6, 6.2, 5.3)$
• For one vector $b = (4,6,1)$, you could find the vector $x = (5.2, 4.6, 5.7)$
• ...

_{(The numbers are made up, only for illustration)}

What you are doing when computing the inverse of the matrix is:

You are solving all these problems simultaneously. You are deriving a rule that tells you how to obtain the matching $x$ for any $b$, without any real computation, just by "looking up" the value of $A^{-1}b$. So instead of solving one problem, you are solving infinitely many problems. That's a nice thing, and pretty useful.

Answer 6

Think about an equation in one variable. For example, 3x = 1. Normally we would divide both sides by $$\frac{1}{3} = 3^{-1}$$ and obtain $$3^{-1}(3x) =3^{-1}1$$ $$(3^{-1}\cdot 3)x =3^{-1} \cdot 1 $$ $$x = 3^{-1}\cdot 1$$

Similarly, we want to do the same thing with a matrix equation $Ax = b$. However, in general $A^{-1}$ does not exist. So in those cases we have to row reduce to show there are either no solutions or infinite number of solutions. If $A^{-1}$ does exist, we can multiply on both sides, thus solving for x.

Answer 7

If you think of the matrix representation as a function from $\mathbb{R}^3$ to itself, maybe that will help? The inverse could then be thought of as the inverse of the function, in the same manner that you would invert, say, $f(x)=x^3+5$. The question you're answering when multiplying the right hand side of the equation with the inverse is "what vector do I put in to my function A(x) so that I get out the RHS".

Answer 8

A matrix is just an array filled with numbers. But you have learnt how to "multipliy" to matrices in order to get a third one. Getting an inverse of a matrix $A$, is just finding an other matrix named $A^{-1}$ such that $$A^{-1}A=id.$$ Now it happens that this multiplication rule, which may seem abstract and a nonsense, is precisely defined to respect finding solution of a system of equations. Namely, if you write as you did your system $A\bar x=\bar y$, then it is equivalent to any system $BA\bar x=B\bar y$ for any invertible matrix $B$, and chosing $B=A^{-1}$ gives you the system $\bar x=A^{-1}\bar y$ because of the (good) way multiplication is defined (associative etc.) which is precisely what you wanted to known: finding $\bar x$ expressed in term of $\bar y$.

Answer 9

I’m not sure which part exactly is unclear to you, so let’s start at the very beginning. The matrix $A$ represents a linear transformation $\mathbb{R}^3 \rightarrow \mathbb{R}^3$ (it’s a theorem that any such linear transformation can be defined by a matrix). The $k$th column of the matrix shows where the transformation puts the $k$th basis vector.

Now the question is, what vector gets mapped to $(2, 12, 2)^\mathrm{T}$? Ways of solving this include finding the inverse of the matrix and replacing equations with equivalent ones till they all become trivial. These two are one and the same: if you want to, say, add the second equation, multiplied by 10, to the first equation, this is equivalent to multiplying both sides of your matrix equation from the left by $$ M=\begin{pmatrix}1 & 10 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{pmatrix}.$$

Each operation on the system of equations has its corresponding matrix, and when you find a sequence of operations that turns the equations into trivial ones, you will have found a sequence of linear transformations that undo the original transformation: $M_nM_{n-1}\cdots M_1A=I$. So $\prod M_i$ is one way to compute $A^{-1}$ (if $M_i$ are matrices corresponding to steps of Gaussian elimination, this is almost the same as LU decomposition).

To answer your question of what does $A^{−1}$ mean with respect to these 3 equations:

1. The equations define a linear transformation, and most linear transformations have inverses, and $A^{−1}$ is the matrix form of that inverse. Or
2. The equations can be solved, and steps of solving those involve going from $A\mathbf{x}=B$ to $A'\mathbf{x}=B'$ a number of times, and $A^{−1}$ represents the composition of all the steps required to go from $A$ to $I$. (Applying the same steps to $B$ then gives the solution, $A^{-1}B$.)

Answer 10

That is a good question when learning about matrices.

The inverse of a matrix $A$ is $A^{-1}$ such that $AA^{-1}=I$.

For simplicity, let's deal with a $2\times2$ matrix: $A=\begin{pmatrix}a&b\\c&d\end{pmatrix}$.

Let's ask the question: $$\begin{pmatrix}a&b\\c&d\end{pmatrix}\begin{pmatrix}?&?\\?&?\end{pmatrix}=\begin{pmatrix}1&0\\0&1\end{pmatrix}$$

Although this looks like division, it isn't, since you can't divide a matrix. What this more resembles is something like $5x\equiv2\pmod7$ where division only doesn't work.

As you know how to calculate the inverse, I will not go into details about that.

So, there are two ways to think about the inverse of a matrix.

• The inverse is a matrix such that if you multiply it with the original matrix, you get the identity matrix. Imagine $\frac12$ written as $2^{-1}$.

• It also means that for an equation $A\mathbf x=\mathbf b$, the inverse is such that if you multiply it by the values on the RHS of the equation (namely $\mathbf b$), then you get the original matrix!