Explanation of setting the overflow flag

Question

In Kip Irvine's book he talks about the mechanism to set the overflow flag, he writes:

The CPU uses an interesting mechanism to determine the state of the Overflow flag after an addition or subtraction operation. The Carry flag is exclusive ORed with the high bit of the result. The resulting value is placed in the Overflow flag.

I wrote a simple program that shows that adding 5 to 127 in the al register sets the overflow flag :

0111 1111 + 0000 0101 = 1000 0100 carry flag = 0, high bit = 1, overflow = 1

However adding 5 to 255 in the al register does not set the overflow flag:

1111 1111 + 0000 0101 = 0000 0100 carry flag = 1, high bit = 0, overflow = 0

Why isn't the overflow flag set in the second example isn't the conditional statement satisfied ie; carry flag = 1 and high bit = 0?

Answer 1

The reason why the overflow flag is not set in your second example 255 + 5 is because it is concerned with signed arithmetic. You are adding -1 + 5 which gives 4. There is no overflow: the result is correct. As @Jester commented, there must be a mistake in your book.

My 8086 book says of the Overflow flag:

It will be set if there is an internal carry from bit 6 to bit 7 (the sign bit) and there is no external Carry. It will also be set when there is no internal carry from bit 6 to bit 7 and there is an external Carry.

EDIT

The paragraph continues:

For the technically minded reader, the overflow flag is set by XOR-ing the carry-in and the carry-out of bit 7 (the sign bit).

In the second example, 1 is carried out (to the Carry flag), and because all the bits were set, there must have been an internal carry-in during the internal addition, even if the resultant b7 is 0.

Answer 2

you can craft a simple program to examine what the logic would do with 3 bit addition/subtraction.

#include <stdio.h>
int main ( void )
{
    unsigned int ra;
    unsigned int rb;
    unsigned int rc;
    unsigned int rd;
    unsigned int re;
    unsigned int cy;
    unsigned int ov;
    unsigned int ovx;
    unsigned int ovy;
    int sa;
    int sb;
    int sc;
    int sd;
    for(ra=0;ra<8;ra++)
    {
        for(rb=0;rb<8;rb++)
        {
            printf("%u%u%u",(ra>>2)&1,(ra>>1)&1,(ra>>0)&1);
            printf(" + ");
            printf("%u%u%u",(rb>>2)&1,(rb>>1)&1,(rb>>0)&1);
            printf(" :");
            if(ra&4) sa=(ra|((-1)<<3)); else sa=ra;
            if(rb&4) sb=(rb|((-1)<<3)); else sb=rb;
            sc = sa + sb;
            //printf("%u(%2d) + %u(%2d)",ra,sa,rb,sb);
            printf("%2d + %2d = %2d",sa,sb,sc);
            printf("  :");
            rc=rb;
            printf("%u%u%u",(ra>>2)&1,(ra>>1)&1,(ra>>0)&1);
            printf(" + ");
            printf("%u%u%u",(rc>>2)&1,(rc>>1)&1,(rc>>0)&1);
            printf(" + 0 = ");
            rd=ra+rc+0;
            if(rd&4) sd=(rd|((-1)<<3)); else sd=rd;
            re=(ra&3)+(rc&3)+0;
            ov=0;
            if((ra&4)==(rc&4)) ov = ((rd>>3)&1) ^ ((rd>>2)&1);
            ovy=0;
            if((ra&4)==(rc&4)) if((rd&4) != (ra&4)) ovy=1;
            ovx = ((rd>>3)&1) ^ ((re>>2)&1);
            printf("%u%u%u",(rd>>2)&1,(rd>>1)&1,(rd>>0)&1);
            printf(" C %u O %u %u %u ",(rd>>3)&1,ov,ovx,ovy);
            if(sc>3) printf("X");
            if(sc<(-4)) printf("X");
            printf("\n");
        }
    }
    for(ra=0;ra<8;ra++)
    {
        for(rb=0;rb<8;rb++)
        {
            printf("%u%u%u",(ra>>2)&1,(ra>>1)&1,(ra>>0)&1);
            printf(" - ");
            printf("%u%u%u",(rb>>2)&1,(rb>>1)&1,(rb>>0)&1);
            printf(" :");
            if(ra&4) sa=(ra|((-1)<<3)); else sa=ra;
            if(rb&4) sb=(rb|((-1)<<3)); else sb=rb;
            sc = sa - sb;
            //printf("%u(%2d) - %u(%2d)",ra,sa,rb,sb);
            printf("%2d - %2d = %2d",sa,sb,sc);
            printf(" : ");
            rc=(~rb)&7;
            printf("%u%u%u",(ra>>2)&1,(ra>>1)&1,(ra>>0)&1);
            printf(" + ");
            printf("%u%u%u",(rc>>2)&1,(rc>>1)&1,(rc>>0)&1);
            printf(" + 1 = ");
            rd=ra+rc+1;
            if(rd&4) sd=(rd|((-1)<<3)); else sd=rd;
            re=(ra&3)+(rc&3)+1;
            ov=0;
            if((ra&4)==(rc&4)) ov = ((rd>>3)&1) ^ ((rd>>2)&1);
            ovx = ((rd>>3)&1) ^ ((re>>2)&1);
            ovy=0;
            if((ra&4)==(rc&4)) if((rd&4) != (ra&4)) ovy=1;
            printf("%u%u%u",(rd>>2)&1,(rd>>1)&1,(rd>>0)&1);
            printf(" C %u O %u %u %u ",(rd>>3)&1,ov,ovx,ovy);
            sc = sa - sb;
            if(sc>3) printf("X");
            if(sc<(-4)) printf("X");
            printf("\n");
        }
    }
}

giving

000 + 000 : 0 +  0 =  0  :000 + 000 + 0 = 000 C 0 O 0 0 0 
000 + 001 : 0 +  1 =  1  :000 + 001 + 0 = 001 C 0 O 0 0 0 
000 + 010 : 0 +  2 =  2  :000 + 010 + 0 = 010 C 0 O 0 0 0 
000 + 011 : 0 +  3 =  3  :000 + 011 + 0 = 011 C 0 O 0 0 0 
000 + 100 : 0 + -4 = -4  :000 + 100 + 0 = 100 C 0 O 0 0 0 
000 + 101 : 0 + -3 = -3  :000 + 101 + 0 = 101 C 0 O 0 0 0 
000 + 110 : 0 + -2 = -2  :000 + 110 + 0 = 110 C 0 O 0 0 0 
000 + 111 : 0 + -1 = -1  :000 + 111 + 0 = 111 C 0 O 0 0 0 
001 + 000 : 1 +  0 =  1  :001 + 000 + 0 = 001 C 0 O 0 0 0 
001 + 001 : 1 +  1 =  2  :001 + 001 + 0 = 010 C 0 O 0 0 0 
001 + 010 : 1 +  2 =  3  :001 + 010 + 0 = 011 C 0 O 0 0 0 
001 + 011 : 1 +  3 =  4  :001 + 011 + 0 = 100 C 0 O 1 1 1 X
001 + 100 : 1 + -4 = -3  :001 + 100 + 0 = 101 C 0 O 0 0 0 
001 + 101 : 1 + -3 = -2  :001 + 101 + 0 = 110 C 0 O 0 0 0 
001 + 110 : 1 + -2 = -1  :001 + 110 + 0 = 111 C 0 O 0 0 0 
001 + 111 : 1 + -1 =  0  :001 + 111 + 0 = 000 C 1 O 0 0 0 
010 + 000 : 2 +  0 =  2  :010 + 000 + 0 = 010 C 0 O 0 0 0 
010 + 001 : 2 +  1 =  3  :010 + 001 + 0 = 011 C 0 O 0 0 0 
010 + 010 : 2 +  2 =  4  :010 + 010 + 0 = 100 C 0 O 1 1 1 X
010 + 011 : 2 +  3 =  5  :010 + 011 + 0 = 101 C 0 O 1 1 1 X
010 + 100 : 2 + -4 = -2  :010 + 100 + 0 = 110 C 0 O 0 0 0 
010 + 101 : 2 + -3 = -1  :010 + 101 + 0 = 111 C 0 O 0 0 0 
010 + 110 : 2 + -2 =  0  :010 + 110 + 0 = 000 C 1 O 0 0 0 
010 + 111 : 2 + -1 =  1  :010 + 111 + 0 = 001 C 1 O 0 0 0 
011 + 000 : 3 +  0 =  3  :011 + 000 + 0 = 011 C 0 O 0 0 0 
011 + 001 : 3 +  1 =  4  :011 + 001 + 0 = 100 C 0 O 1 1 1 X
011 + 010 : 3 +  2 =  5  :011 + 010 + 0 = 101 C 0 O 1 1 1 X
011 + 011 : 3 +  3 =  6  :011 + 011 + 0 = 110 C 0 O 1 1 1 X
011 + 100 : 3 + -4 = -1  :011 + 100 + 0 = 111 C 0 O 0 0 0 
011 + 101 : 3 + -3 =  0  :011 + 101 + 0 = 000 C 1 O 0 0 0 
011 + 110 : 3 + -2 =  1  :011 + 110 + 0 = 001 C 1 O 0 0 0 
011 + 111 : 3 + -1 =  2  :011 + 111 + 0 = 010 C 1 O 0 0 0 
100 + 000 :-4 +  0 = -4  :100 + 000 + 0 = 100 C 0 O 0 0 0 
100 + 001 :-4 +  1 = -3  :100 + 001 + 0 = 101 C 0 O 0 0 0 
100 + 010 :-4 +  2 = -2  :100 + 010 + 0 = 110 C 0 O 0 0 0 
100 + 011 :-4 +  3 = -1  :100 + 011 + 0 = 111 C 0 O 0 0 0 
100 + 100 :-4 + -4 = -8  :100 + 100 + 0 = 000 C 1 O 1 1 1 X
100 + 101 :-4 + -3 = -7  :100 + 101 + 0 = 001 C 1 O 1 1 1 X
100 + 110 :-4 + -2 = -6  :100 + 110 + 0 = 010 C 1 O 1 1 1 X
100 + 111 :-4 + -1 = -5  :100 + 111 + 0 = 011 C 1 O 1 1 1 X
101 + 000 :-3 +  0 = -3  :101 + 000 + 0 = 101 C 0 O 0 0 0 
101 + 001 :-3 +  1 = -2  :101 + 001 + 0 = 110 C 0 O 0 0 0 
101 + 010 :-3 +  2 = -1  :101 + 010 + 0 = 111 C 0 O 0 0 0 
101 + 011 :-3 +  3 =  0  :101 + 011 + 0 = 000 C 1 O 0 0 0 
101 + 100 :-3 + -4 = -7  :101 + 100 + 0 = 001 C 1 O 1 1 1 X
101 + 101 :-3 + -3 = -6  :101 + 101 + 0 = 010 C 1 O 1 1 1 X
101 + 110 :-3 + -2 = -5  :101 + 110 + 0 = 011 C 1 O 1 1 1 X
101 + 111 :-3 + -1 = -4  :101 + 111 + 0 = 100 C 1 O 0 0 0 
110 + 000 :-2 +  0 = -2  :110 + 000 + 0 = 110 C 0 O 0 0 0 
110 + 001 :-2 +  1 = -1  :110 + 001 + 0 = 111 C 0 O 0 0 0 
110 + 010 :-2 +  2 =  0  :110 + 010 + 0 = 000 C 1 O 0 0 0 
110 + 011 :-2 +  3 =  1  :110 + 011 + 0 = 001 C 1 O 0 0 0 
110 + 100 :-2 + -4 = -6  :110 + 100 + 0 = 010 C 1 O 1 1 1 X
110 + 101 :-2 + -3 = -5  :110 + 101 + 0 = 011 C 1 O 1 1 1 X
110 + 110 :-2 + -2 = -4  :110 + 110 + 0 = 100 C 1 O 0 0 0 
110 + 111 :-2 + -1 = -3  :110 + 111 + 0 = 101 C 1 O 0 0 0 
111 + 000 :-1 +  0 = -1  :111 + 000 + 0 = 111 C 0 O 0 0 0 
111 + 001 :-1 +  1 =  0  :111 + 001 + 0 = 000 C 1 O 0 0 0 
111 + 010 :-1 +  2 =  1  :111 + 010 + 0 = 001 C 1 O 0 0 0 
111 + 011 :-1 +  3 =  2  :111 + 011 + 0 = 010 C 1 O 0 0 0 
111 + 100 :-1 + -4 = -5  :111 + 100 + 0 = 011 C 1 O 1 1 1 X
111 + 101 :-1 + -3 = -4  :111 + 101 + 0 = 100 C 1 O 0 0 0 
111 + 110 :-1 + -2 = -3  :111 + 110 + 0 = 101 C 1 O 0 0 0 
111 + 111 :-1 + -1 = -2  :111 + 111 + 0 = 110 C 1 O 0 0 0 
000 - 000 : 0 -  0 =  0 : 000 + 111 + 1 = 000 C 1 O 0 0 0 
000 - 001 : 0 -  1 = -1 : 000 + 110 + 1 = 111 C 0 O 0 0 0 
000 - 010 : 0 -  2 = -2 : 000 + 101 + 1 = 110 C 0 O 0 0 0 
000 - 011 : 0 -  3 = -3 : 000 + 100 + 1 = 101 C 0 O 0 0 0 
000 - 100 : 0 - -4 =  4 : 000 + 011 + 1 = 100 C 0 O 1 1 1 X
000 - 101 : 0 - -3 =  3 : 000 + 010 + 1 = 011 C 0 O 0 0 0 
000 - 110 : 0 - -2 =  2 : 000 + 001 + 1 = 010 C 0 O 0 0 0 
000 - 111 : 0 - -1 =  1 : 000 + 000 + 1 = 001 C 0 O 0 0 0 
001 - 000 : 1 -  0 =  1 : 001 + 111 + 1 = 001 C 1 O 0 0 0 
001 - 001 : 1 -  1 =  0 : 001 + 110 + 1 = 000 C 1 O 0 0 0 
001 - 010 : 1 -  2 = -1 : 001 + 101 + 1 = 111 C 0 O 0 0 0 
001 - 011 : 1 -  3 = -2 : 001 + 100 + 1 = 110 C 0 O 0 0 0 
001 - 100 : 1 - -4 =  5 : 001 + 011 + 1 = 101 C 0 O 1 1 1 X
001 - 101 : 1 - -3 =  4 : 001 + 010 + 1 = 100 C 0 O 1 1 1 X
001 - 110 : 1 - -2 =  3 : 001 + 001 + 1 = 011 C 0 O 0 0 0 
001 - 111 : 1 - -1 =  2 : 001 + 000 + 1 = 010 C 0 O 0 0 0 
010 - 000 : 2 -  0 =  2 : 010 + 111 + 1 = 010 C 1 O 0 0 0 
010 - 001 : 2 -  1 =  1 : 010 + 110 + 1 = 001 C 1 O 0 0 0 
010 - 010 : 2 -  2 =  0 : 010 + 101 + 1 = 000 C 1 O 0 0 0 
010 - 011 : 2 -  3 = -1 : 010 + 100 + 1 = 111 C 0 O 0 0 0 
010 - 100 : 2 - -4 =  6 : 010 + 011 + 1 = 110 C 0 O 1 1 1 X
010 - 101 : 2 - -3 =  5 : 010 + 010 + 1 = 101 C 0 O 1 1 1 X
010 - 110 : 2 - -2 =  4 : 010 + 001 + 1 = 100 C 0 O 1 1 1 X
010 - 111 : 2 - -1 =  3 : 010 + 000 + 1 = 011 C 0 O 0 0 0 
011 - 000 : 3 -  0 =  3 : 011 + 111 + 1 = 011 C 1 O 0 0 0 
011 - 001 : 3 -  1 =  2 : 011 + 110 + 1 = 010 C 1 O 0 0 0 
011 - 010 : 3 -  2 =  1 : 011 + 101 + 1 = 001 C 1 O 0 0 0 
011 - 011 : 3 -  3 =  0 : 011 + 100 + 1 = 000 C 1 O 0 0 0 
011 - 100 : 3 - -4 =  7 : 011 + 011 + 1 = 111 C 0 O 1 1 1 X
011 - 101 : 3 - -3 =  6 : 011 + 010 + 1 = 110 C 0 O 1 1 1 X
011 - 110 : 3 - -2 =  5 : 011 + 001 + 1 = 101 C 0 O 1 1 1 X
011 - 111 : 3 - -1 =  4 : 011 + 000 + 1 = 100 C 0 O 1 1 1 X
100 - 000 :-4 -  0 = -4 : 100 + 111 + 1 = 100 C 1 O 0 0 0 
100 - 001 :-4 -  1 = -5 : 100 + 110 + 1 = 011 C 1 O 1 1 1 X
100 - 010 :-4 -  2 = -6 : 100 + 101 + 1 = 010 C 1 O 1 1 1 X
100 - 011 :-4 -  3 = -7 : 100 + 100 + 1 = 001 C 1 O 1 1 1 X
100 - 100 :-4 - -4 =  0 : 100 + 011 + 1 = 000 C 1 O 0 0 0 
100 - 101 :-4 - -3 = -1 : 100 + 010 + 1 = 111 C 0 O 0 0 0 
100 - 110 :-4 - -2 = -2 : 100 + 001 + 1 = 110 C 0 O 0 0 0 
100 - 111 :-4 - -1 = -3 : 100 + 000 + 1 = 101 C 0 O 0 0 0 
101 - 000 :-3 -  0 = -3 : 101 + 111 + 1 = 101 C 1 O 0 0 0 
101 - 001 :-3 -  1 = -4 : 101 + 110 + 1 = 100 C 1 O 0 0 0 
101 - 010 :-3 -  2 = -5 : 101 + 101 + 1 = 011 C 1 O 1 1 1 X
101 - 011 :-3 -  3 = -6 : 101 + 100 + 1 = 010 C 1 O 1 1 1 X
101 - 100 :-3 - -4 =  1 : 101 + 011 + 1 = 001 C 1 O 0 0 0 
101 - 101 :-3 - -3 =  0 : 101 + 010 + 1 = 000 C 1 O 0 0 0 
101 - 110 :-3 - -2 = -1 : 101 + 001 + 1 = 111 C 0 O 0 0 0 
101 - 111 :-3 - -1 = -2 : 101 + 000 + 1 = 110 C 0 O 0 0 0 
110 - 000 :-2 -  0 = -2 : 110 + 111 + 1 = 110 C 1 O 0 0 0 
110 - 001 :-2 -  1 = -3 : 110 + 110 + 1 = 101 C 1 O 0 0 0 
110 - 010 :-2 -  2 = -4 : 110 + 101 + 1 = 100 C 1 O 0 0 0 
110 - 011 :-2 -  3 = -5 : 110 + 100 + 1 = 011 C 1 O 1 1 1 X
110 - 100 :-2 - -4 =  2 : 110 + 011 + 1 = 010 C 1 O 0 0 0 
110 - 101 :-2 - -3 =  1 : 110 + 010 + 1 = 001 C 1 O 0 0 0 
110 - 110 :-2 - -2 =  0 : 110 + 001 + 1 = 000 C 1 O 0 0 0 
110 - 111 :-2 - -1 = -1 : 110 + 000 + 1 = 111 C 0 O 0 0 0 
111 - 000 :-1 -  0 = -1 : 111 + 111 + 1 = 111 C 1 O 0 0 0 
111 - 001 :-1 -  1 = -2 : 111 + 110 + 1 = 110 C 1 O 0 0 0 
111 - 010 :-1 -  2 = -3 : 111 + 101 + 1 = 101 C 1 O 0 0 0 
111 - 011 :-1 -  3 = -4 : 111 + 100 + 1 = 100 C 1 O 0 0 0 
111 - 100 :-1 - -4 =  3 : 111 + 011 + 1 = 011 C 1 O 0 0 0 
111 - 101 :-1 - -3 =  2 : 111 + 010 + 1 = 010 C 1 O 0 0 0 
111 - 110 :-1 - -2 =  1 : 111 + 001 + 1 = 001 C 1 O 0 0 0 
111 - 111 :-1 - -1 =  0 : 111 + 000 + 1 = 000 C 1 O 0 0 0

so for addition you would have one like this (the signed overflow flag by definition is for when you interpret the bits as signed)

011 + 001 : 3 +  1 =  4  :011 + 001 + 0 = 100 C 0 O 1 1 1 X

so 1 + 3 (001 + 011) = the bit pattern 100, which in a three bit twos complement world has the value -4 so 1 + 3 = -4 which is wrong so signed overflow, we cannot represent a +4 with three bits. On an x86 this would be the equivalent of an 8 bit addition of 127+1 (0x7F + 0x01). And basically all of the combinations of positive numbers that result in 128 (or larger) 126+2, 125+3 124+4 and so on. all have this problem.

010 + 010 : 2 +  2 =  4  :010 + 010 + 0 = 100 C 0 O 1 1 1 X

I did addition and subtraction. Subtraction logically comes from the twos complement notion invert and add 1. So subtraction c = a - b uses instead c = a + (-b), and from twos complement we know this means c = a + ((~b)+1) or c = a + ~b + 1. Addition is c = a + b + 0. the latter 1 or 0 being the carry in of the lsbit.

Now take this one step further addition c = a + b + cin, subtraction c = a + ~b + ~cin. You invert the second operand and the carry in. BUT that is processor specific as some processors invert the carry out (I think x86 is one) making it a "borrow" instead of a "carry" for subtraction. Then that messes up the notion of carry in for add with carry or subtract with borrow if you have those instructions (the logic for those would simply not invert cin on a sbb)

I computed the overflow flag three different (really?) ways.

    ov=0;
    if((ra&4)==(rc&4)) ov = ((rd>>3)&1) ^ ((rd>>2)&1);
    ovx = ((rd>>3)&1) ^ ((re>>2)&1);
    ovy=0;
    if((ra&4)==(rc&4)) if((rd&4) != (ra&4)) ovy=1;

ov is like in your text you are reading, if the signs are the same going into the adder then the overflow is carry out xored with the msbit of the result.

ovx is the definition of overflow carry in compared to carry out of the msbit

and ovy is a shortcut you can use if you want to figure out overflow but dont have an N+1 bit for registers (how would you figure out overflow using 32 bit variables in a language where you cannot see the carry out? many ways as I have shown in the code, but simply examining the msbits work as well).

and then the X at the end is also the definition of overflow if the result does not fit into the number of bits available then you overflowed. true for unsigned (carry) and signed (overflow) overflow. since this is about overflow then this is about signed numbers so for my three bit system you can only get from -4 to +3 anything above +3 or less than -4 wont fit and the X at the end of the printout shows that, so that is a fourth way to show overflow in this simplified example.

Again the output above is how the generic logic would do it, you then have processor family nuances with the carry out flag that some processors invert the carry out to make it a borrow and some do not when doing a subtract. true real logic would cascade a bunch of 3 input (two operands and carry in) two output (result and carry out) adders together, although in HDL languages you can use a plus operator and get around that (and in those languages also need one of these shortcuts that doesnt examine carry in vs carry out)

If you work the booloean equations you should be able to find that the three ways shown to compute the overflow are equivalent, not just experimentally as here, but mathematically.