VBA: Why would the Not operator stop working? [duplicate]

Question

This has me utterly baffled.

Sub testChangeBoolean()   Dim X As Boolean       ' default value is False   X = True               ' X is now True   X = Not X              ' X is back to False End Sub 

But I'm trying to toggle the .FitText property in a table cell (in Word). .FitText starts as True.

If I assign it to X:

Sub testChangeBoolean()   Dim X As Boolean                  ' again, default is False   X = Selection.Cells(1).FitText    ' does set X to True   X = Not X                         ' X is still True! End Sub 

I just don't understand what I'm doing wrong.

Answer 1

I believe the explanation has to do with how older programming languages (WordBasic and early VBA) stored the integer values of True and False. In those days, True = -1 and False = 0.

Newer programming languages still use 0 for False, but 1 for True.

The majority of Word's Boolean type properties continue to use -1 for True (Font.Bold, for example), which has been cause for confusion and frustration for programmers working with the Interop in newer languages. So, at some point, some developers at Microsoft decided to use the new way and assigned the integer value of 1 to True for some new functionality. Such as FitText.

Considering the following code sample, where X is of type Boolean and y of type Integer:

• If FitText is True, the integer value is 1
• If reversing the values, using Not shows that the Boolean remains "True" because its integer value is not 0, it's -2
• Setting the integer value directly to True gives -1

This is confusing, indeed, but does explain why Not is not giving the expected result.

Sub testChangeBoolean()   Dim X As Boolean                  ' again, default is False   Dim Y As Integer   X = Selection.Cells(1).FitText    ' does set X to True   Y = Selection.Cells(1).FitText   Debug.Print X, Y                  ' result: True    1   X = Not X                         ' X is still True!   Y = Not Y   Debug.Print X, Y                  ' result: True   -2   X = False   Y = True   Debug.Print X, Y                  ' result: False  -1 End Sub 

Answer 2

To add on to Cindy's excellent answer, I want to point out that while VBA normally has safeguards to coerce the values when assigning to a Boolean data type, this can be circumvented. Basically, if you write a random value to a memory address that's not yours, then you should expected undefined behavior.

To help demonstrate this, we'll (ab)use LSet which essentially allow us to copy the value without actually assigning.

Private Type t1   b As Boolean End Type  Private Type t2   i As Integer End Type  Private Sub Demo()   Dim i1 As t2   Dim b1 As t1   Dim b As Boolean    i1.i = 1    LSet b1 = i1    b = b1.b    Debug.Print b, b1.b, i1.i   Debug.Print CInt(b), CInt(b1.b), i1.i  End Sub 

Note the line b = b1.b is basically equivalent to what we did in the OP code

X = Selection.Cells(1).FitText 

That is, assigning a Boolean to another Boolean. However, because I wrote to the b1.b using LSet, bypassing VBA runtime checks, it doesn't get coerced. When reading the Boolean, VBA does implicitly coerce it into either True or False, which seems misleading but is correct because any falsy results is one that equals 0 (aka False), and any truthy results is one that doesn't. Note that the negative for truthy means that both 1 and -1 are truthy.

Had I assigned the 1 to a Boolean variable directly, VBA would have had coerced it into -1/True and thus there'd be no problem. But evidently with FitText or LSet, we are basically writing to the memory address in an uncontrolled fashion, so that VBA start to behave strangely with this particular variable since it expects the Boolean variable to already had its contents coerced but wasn't.