Let's take a second look at the "book" method of adding $-5$ and $3$ in four-bit two's-complement arithmetic.
In four-bit two's-complement binary, $-5$ is represented by $1011$ and $3$ is represented by $0011.$ The sum is $1011 + 0011 = 1110,$ which is the four-bit two's-complement representation of $-2.$ And that's the answer: $-2.$
The only reason to do any further operations on the result is if you want to convert the result back to ordinary decimal notation and don't instantly recognize that $1110$ represents $-2.$
One way to do that is to take the two's complement of $1110,$ which is $0010.$ Taking the two's complement of a number changes the sign of the number in the two's-complement representation.
If reversing the sign of $1110$ gives us $0010,$ which represents $2,$ then what does $1110$ represent? It can only represent $-2.$
Again, that 's the answer: $-2.$
The rest of the question presents an interesting procedure for adding binary representations of integers. Instead of using two's-complement exclusively, however, this method begins with the two operands ($-5$ and $3$) in a four-bit signed-magnitude representation, and ends with the result ($-2$) in four-bit signed-magnitude representation.
The method has three major steps:
- Convert the signed-magnitude representation to two's complement.
- Add the two's-complement representations.
- Convert the result from two's complement to signed magnitude.
The two's-complement addition is performed in the conventional way, so we merely have to check that the conversions from signed-magnitude to two's-complement and back again are correct.
Positive numbers are "converted" by leaving their bits completely unchanged.
Only negative numbers have their bits changed by the conversion.
In order to be able to reason about the conversion mathematically, I'll evaluate each four-bit binary number as an unsigned integer, although it can still also be interpreted as signed magnitude or two's complement.
If a four-bit signed-magnitude binary number represents $x,$ where $x < 0,$
then the bits of that representation have unsigned value
$8 - \lvert x\rvert.$ (The sign bit has place value $8,$ and the rightmost three bits have value $\lvert x\rvert.$)
The first step is to take the one's complement of the three magnitude bits,
which is equivalent to subtracting them from the three-bit number $111,$
that is, from $7.$ The sign bit remains unchanged, so after this step
we have a four-bit number with unsigned value
$8 + (7 - \lvert x\rvert) = 15 - \lvert x\rvert.$
The second step is to add $1.$ The resulting number has unsigned value
$(15 - \lvert x\rvert) + 1 = 16 - \lvert x\rvert = 16 + x,$
since $x < 0.$
And $16 + x$ is the unsigned value of the four-bit two's-complement representation of a number $x$ when $x < 0.$
Hence the conversion is always correct.
More generally, for an $n$-bit signed-magnitude representation of a
negative number $x,$ the
unsigned value of the signed-magnitude representation is
$2^{n-1} + \lvert x\rvert,$ after taking the one's complement of the magnitude we have
$2^{n-1} + (2^{n-1} - 1 - \lvert x\rvert) = 2^n - 1 - \lvert x\rvert,$
and after adding $1$ we have
$(2^n - 1 - \lvert x\rvert) + 1 = 2^n - \lvert x\rvert = 2^n + x,$
the two's complement representation of $x.$ Since the signed-magnitude
representation can only represent a number $x$ if
$\lvert x\rvert \leq 2^{n-1} - 1,$ there is no possibility of overflow,
and this conversion will always work.
Note that if the signed-magnitude number is $-0$ (sign bit $1$ with magnitude $0$), the procedure above produces a number with all bits set to $1$ after the one's complement, and adding $1$ to this results in all bits set to $0.$
That is, this signed-magnitude representation is correctly converted to $0$ in two's complement.
To convert from two's complement to signed magnitude, again positive numbers are not changed, and only negative numbers require manipulation of their bits.
Clearly the procedure cannot work when $x = -2^{n-1},$ which is the most negative possible number that can be represented in $n$-bit two's complement,
because the least number that can be represented in $n$-bit signed magnitude is $-(2^{n-1} - 1).$
So let's assume that $-(2^{n-1} - 1) \leq x \leq -1,$ that is, $x$ can be
any of the possible negative integers in $n$-bit two's-complement binary representation except $-2^{n-1}$
Then the $n$-bit two's-complement representation of $x$ is a binary number
with unsigned value
$2^n + x = 2^n - \lvert x \rvert \geq 2^{n-1} + (2^{n-1} - \lvert x \rvert).$
That is, the rightmost $n-1$ bits have unsigned value
$2^{n-1} - \lvert x \rvert.$
We take the one's complement of these bits,
$(2^{n-1} - 1) - (2^{n-1} - \lvert x \rvert) = \lvert x \rvert - 1,$
and put these to the right of the sign bit, so we have a binary number
with unsigned value $2^{n-1} + \lvert x \rvert - 1.$
The next step is to add $1$ to the this, with the result
$2^{n-1} + \lvert x \rvert,$ which is the $n$-bit signed-magnitude representation of $x$ when $x < 0.$
In summary, the proposed method does indeed give correct results in all cases except when overflow occurs, and those are precisely the cases where it is not possible to represent the correct result in signed magnitude
(that is, those are the cases in which no algorithm can give the correct answer).
