Modifying python's string Formatter

Question

I have a function which takes no arguments and returns a string, which I want to be called using a string format. Here it is, and here's how I've tried to use format:

def cabbages():
    return 'hello'

In [2]: '{cabbages} world'.format(**locals())
Out[2]: '<function cabbages at 0x101f75578> world'

In [3]: '{cabbages()} world'.format(**locals())
KeyError: 'cabbages()'

So neither of which is quite what I want, i.e. the value of cabbages().

PEP 3101 describes some way in which string.Formatter can be overwritten but it doesn't seem to give many examples. How can I subclass/customise the string Formatter class to do this?

A hacky thing I considered would be to overwrite the __getattr__ method of cabbages, and I really don't want be "considered pathological" (or, at least, *that* pathological).

Answer 1

I guess the main deficiency I see in your answer is that it doesn't also handle the original compound field name syntaxes like 0.name for 'getattr'/'dot' operator access nor, 0[name] for 'getitem' retrieval specified in PEP 3101.

Here's a version that works in both Python 2.7 and 3.3. A major implementation difference is that it overrides the get_value() method instead of get_field().

While it's a little hacky about how it detects calls in the get_value() method, I don't think it's to the point where it would be considered pathological. ;-)

from __future__ import print_function

from string import Formatter

class CallFormatter(Formatter):
    try:  # deal with Py 2 & 3 difference
        NUMERICS = (int, long)
    except NameError:
        NUMERICS = int

    def get_value(self, key, args, kwargs):
        if key.endswith('()'):  # call?
            return kwargs[key[:-2]]()
        elif isinstance(key, self.NUMERICS):
            return args[key]
        else:
            return kwargs[key]

if __name__ == '__main__':
    fmt = CallFormatter()

    def cabbages():
        return 'hello'

    d = dict(name='Fred')

    class Thing(object):
        def __init__(self, value):
            self.attr = value
    th = Thing(42)

    print('d[name]:{d[name]}, th.attr:{th.attr}, '
          'cabbages:{cabbages}'.format(**locals()))
    print(fmt.format('d[name]:{d[name]}, th.attr:{th.attr}, '
                     'cabbages:{cabbages}, cabbages():{cabbages()}', 
                     **locals()))

Output:

d[name]:Fred, th.attr:42, cabbages:<function cabbages at 0x00BB05F0>
d[name]:Fred, th.attr:42, cabbages:<function cabbages at 0x00BB05F0>, 
                          cabbages():hello

Answer 2

You can overwrite the get_field method of Formatter as follows:

from string import Formatter
class CallFormatter(Formatter):
    def get_field(self, field_name, *args, **kwargs):
        obj, used_key = Formatter.get_field(self, field_name, *args, **kwargs)
        return obj(), used_key  # obj is the function

fmt = CallFormatter()

In [11]: fmt.format('{cabbages} world', **locals())
Out[11]: 'hello world'

Doing something like this comes with a health warning, so I think it's worth pasting the Security Considerations section of that PEP in full (emphasis added):

Historically, string formatting has been a common source of security holes in web-based applications, particularly if the string formatting system allows arbitrary expressions to be embedded in format strings.

The best way to use string formatting in a way that does not create potential security holes is to never use format strings that come from an untrusted source.

Barring that, the next best approach is to ensure that string formatting has no side effects. Because of the open nature of Python, it is impossible to guarantee that any non-trivial operation has this property. What this PEP does is limit the types of expressions in format strings to those in which visible side effects are both rare and strongly discouraged by the culture of Python developers. So for example, attribute access is allowed because it would be considered pathological to write code where the mere access of an attribute has visible side effects (whether the code has invisible side effects - such as creating a cache entry for faster lookup - is irrelevant.)