Python's pyparsing: Implementing grammar to parse logical AND expression

Question

I am trying to parse and evaluate expressions, given to me as input from a file, of the form:

var[3] = 0 and var[2] = 1
var[0] = 1 and var[2] = 0 and var[3] = 1
...

(actually I also allow "multibit access" (i.e. var[X:Y]) but let's ignore it for now...)
Where var is an integer, and the [] indicates bit access.
For example, for var = 0x9, the first expression above should be evaluated to False, and the second should be evaluated to True since 0x9 = b1001.
and and = are the only binary operators I allow, and for the = operator, the left operand is always var[X] and the right operand is always a number.
I tried to look around a bit and found that this could be achieved with Python's pyparsing, but I ran into some difficulties trying to implement it.
Here's what I've tried so far, based roughly on this example (which is one of many examples provided here):

#!/usr/bin/env python
from pyparsing import Word, alphas, nums, infixNotation, opAssoc

class BoolAnd():
    def __init__(self, pattern):
        self.args = pattern[0][0::2]

    def __bool__(self):
        return all(bool(a) for a in self.args)

    __nonzero__ = __bool__


class BoolEqual():
    def __init__(self, pattern):
        self.bit_offset = int(pattern[0][1])
        self.value = int(pattern[0][-1])

    def __bool__(self):
        return True if (0xf >> self.bit_offset) & 0x1 == self.value else False # for now, let's assume var == 0xf

    __nonzero__ = __bool__




variable_name   = 'var'
bit_access      = variable_name + '[' + Word(nums) + ']'
multibit_access = variable_name + '[' + Word(nums) + ':' + Word(nums) + ']'
value = Word(nums)

operand = bit_access | multibit_access | value

expression = infixNotation(operand,
    [
    ('=',   2, opAssoc.LEFT,  BoolEqual),
    ('AND', 2, opAssoc.LEFT,  BoolAnd),
    ])


p = expression.parseString('var[3] = 1 AND var[1] = 0', True)

print 'SUCCESS' if bool(p) else 'FAIL'

I have three problems I need help with.

1. For multibit access of the form var[X:Y] = Z, how do I enforce that:
   a. X > Y
   b. Z < 2^{X - Y + 1}
   I assume this can't be enforced by the grammar itself (for example, for single-bit access of the form var[X] = Y, I can enforce by the grammar that Y will be either 0 or 1, and this will cause the expression.parseString() to fail with exception if Y != 0/1).
2. Most importantly: Why does it always print SUCCESS? What am I doing wrong?
   For the input var[3] = 1 AND var[1] = 0 it should be print FAIL (you can see in my example that I hardcoded var to be 0xf, so var[3] = 1 is True but var[1] = 0 is False).
3. Which brings me to my third problem: var is not a class member of BoolEqual nor is it global... is there a way to somehow send it to BoolEqual's __init__ function?

Answer 1

Before getting to the problem solving, I suggest some minor changes to your grammar, primarily the inclusion of results names. Adding these names will make your resulting code much cleaner and robust. I'm also using some of the expressions that were added in recent pyparsing versions, in the pyparsing_common namespace class:

from pyparsing import pyparsing_common

variable_name   = pyparsing_common.identifier.copy()
integer = pyparsing_common.integer.copy()
bit_access      = variable_name('name') + '[' + integer('bit') + ']'
multibit_access = variable_name('name') + '[' + integer('start_bit') + ':' + integer('end_bit') + ']'

Part 1a: Enforcing valid values in "var[X:Y]"

This kind of work is best done using parse actions and conditions. Parse actions are parse-time callbacks that you can attach to your pyparsing expressions to modify, enhance, filter the results, or raise an exception if a validation rule fails. These are attached using the method:

expr.addParseAction(parse_action_fn)

And parse_action_fn can have any of the following signatures:

def parse_action_fn(parse_string, parse_location, matched_tokens):
def parse_action_fn(parse_location, matched_tokens):
def parse_action_fn(matched_tokens):
def parse_action_fn():

(See more at https://pythonhosted.org/pyparsing/pyparsing.ParserElement-class.html#addParseActio)n)

Parse actions can return None, return new tokens, modify the given tokens, or raise an exception.

If all the parse action does is evaluate some condition based on the input tokens, you can write it as a simple function that returns True or False, and pyparsing will raise an exception if False is returned. In your case, your first validation rule could be implemented as:

def validate_multibit(tokens):
    return tokens.end_bit > tokens.start_bit
multibit_access.addCondition(validate_multibit,
                            message="start bit must be less than end bit", 
                            fatal=True)

Or even just as a Python lambda function:

multibit_access.addCondition(lambda t: t.end_bit > t.start_bit, 
                            message="start bit must be less than end bit", 
                            fatal=True)

Now you can try this with:

multibit_access.parseString("var[3:0]")

And you will get this exception:

pyparsing.ParseFatalException: start bit must be less than end bit (at char 0), (line:1, col:1)

Part 1b: Enforcing valid values in "var[X:Y] = Z"

Your second validation rule deals not only with the var bit ranges but also the value it is being compared to. This will require a parse action that is attached to the complete BoolEqual. We could put this in BoolEqual's __init__ method, but I prefer to separate independent functions when possible. And since we will be adding our validation by attaching to the infixNotation level, and infixNotation only accepts parse actions, we will need to write your second validation rule as a parse action that raises an exception. (We will also use a new feature that was only recently released in pyparsing 2.2.0, attaching multiple parse actions at a level in infixNotation.)

Here is the validation that we will wish to perform:

• if single bit expression, value must be 0 or 1

• if multibit expression var[X:Y], value must be < 2**(Y-X+1)

  def validate_equality_args(tokens):
      tokens = tokens[0]
      z = tokens[-1]
      if 'bit' in tokens:
          if z not in (0,1):
              raise ParseFatalException("invalid equality value - must be 0 or 1")
      else:
          x = tokens.start_bit
          y = tokens.end_bit
          if not z < 2**(y - x + 1):
              raise ParseFatalException("invalid equality value")

And we attach this parse action to infixNotation using:

expression = infixNotation(operand,
    [
    ('=',   2, opAssoc.LEFT,  (validate_equality_args, BoolEqual)),
    ('AND', 2, opAssoc.LEFT,  BoolAnd),
    ])

Part 3: Supporting other var names and values than 0xf

To deal with vars of various names, you can add a class-level dict to BoolEqual:

class BoolEqual():
    var_names = {}

and set this ahead of time:

BoolEqual.var_names['var'] = 0xf

And then implement your __bool__ method as just:

return (self.var_names[self.var_name] >> self.bit_offset) & 0x1 == self.value

(This will need to be extended to support multibit, but the general idea is the same.)