Is it possible to fully Monkey Patch builtin `str` in python3

Question

I am trying to patch python’s built-in str in order to track the count of all str allocations. I am running into some issues and was wondering if anyone could see what I’m doing wrong, or if this is even possible natively through monkey patching in python3? (the following works fine in python 2.7.12)

$ python
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux

I first naively tried to patch str as if it were a function:

def patch_str_allocations():
  old_str = str
  def mystr(*args, **kwargs):
    return old_str(*args, **kwargs)

  builtins.str = mystr


def test():
  logger = logging.getLogger(__name__)
  patch_str_allocations()
  logger.debug(str('test'))

But of course this fails all sorts of operations that string is used for like isinstance

    logger.debug(route)
  File "/usr/lib/python3.5/logging/__init__.py", line 1267, in debug
    self._log(DEBUG, msg, args, **kwargs)
  File "/usr/lib/python3.5/logging/__init__.py", line 1403, in _log
    fn, lno, func, sinfo = self.findCaller(stack_info)
  File "/usr/lib/python3.5/logging/__init__.py", line 1360, in findCaller
    filename = os.path.normcase(co.co_filename)
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/posixpath.py", line 52, in normcase
    if not isinstance(s, (bytes, str)):
TypeError: isinstance() arg 2 must be a type or tuple of types

I then tried a class based approach:

class StrAllocator(str):
    oldstr = None

    def __new__(cls, *args, **kwargs):
        return StrAllocator.oldstr.__new__(cls, *args, **kwargs)

    @property
    def __class__(self):
        return str


def patch_str_allocations():
    StrAllocator.oldstr = str
    builtins.str = StrAllocator

In normal str construction this is working OK but am still running into some issues:

class StrAllocatorTestCase(unittest.TestCase):

    def test_log(self):
        t1 = str('t1')
        logger = logging.getLogger(__name__)
        patch_str_allocations()
        t2 = str('t2')
        print(type(t1))
        print(type(t2))
        print(isinstance(t1, str))
        print(isinstance(t2, StrAllocator))
        print(isinstance(t2, str))
        logger.debug(str('test'))


$ nosetests tests.test_str_allocator:StrAllocatorTestCase.test_log -s

<class 'str'>
<class 'pythonapm.instruments.allocations.StrAllocator'>
False
True
True
E
======================================================================
ERROR: test_log (tests.instruments.test_str_allocator.StrAllocatorTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/vagrant_data/github.com/dm03514/python-apm/tests/instruments/test_str_allocator.py", line 30, in test_log
    logger.debug(str('test'))
  File "/usr/lib/python3.5/logging/__init__.py", line 1267, in debug
    self._log(DEBUG, msg, args, **kwargs)
  File "/usr/lib/python3.5/logging/__init__.py", line 1403, in _log
    fn, lno, func, sinfo = self.findCaller(stack_info)
  File "/usr/lib/python3.5/logging/__init__.py", line 1360, in findCaller
    filename = os.path.normcase(co.co_filename)
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/posixpath.py", line 54, in normcase
    "not '{}'".format(s.__class__.__name__))
TypeError: normcase() argument must be str or bytes, not 'str'

----------------------------------------------------------------------
Ran 1 test in 0.003s

As well as in sre_compile on isstring check

  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/site-packages/flask/app.py", line 1250, in decorator                                                         [0/9965]
    self.add_url_rule(rule, endpoint, f, **options)
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/site-packages/flask/app.py", line 66, in wrapper_func
    return f(self, *args, **kwargs)
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/site-packages/flask/app.py", line 1216, in add_url_rule
    self.url_map.add(rule)
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/site-packages/werkzeug/routing.py", line 1215, in add
    rule.bind(self)
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/site-packages/werkzeug/routing.py", line 687, in bind
    self.compile()
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/site-packages/werkzeug/routing.py", line 751, in compile
    self._regex = re.compile(regex, re.UNICODE)
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/re.py", line 224, in compile
    return _compile(pattern, flags)
  File "/home/ubuntu/.virtualenvs/papm/lib/python3.5/re.py", line 292, in _compile
    raise TypeError("first argument must be string or compiled pattern")
TypeError: first argument must be string or compiled pattern

---

Can anyone see what's missing? (besides my understanding of descriptors and python classes :p )

---

From the REPL the example above Works, but it does not work within the context of nose and unittests...

⟫ ipython
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import logging

In [2]: import builtins

In [3]: class StrAllocator(str):
   ...:     oldstr = None
   ...:
   ...:     def __new__(cls, *args, **kwargs):
   ...:         return StrAllocator.oldstr.__new__(cls, *args, **kwargs)
   ...:
   ...:     @property
   ...:     def __class__(self):
   ...:         return str
   ...:
   ...:
In [4]: def patch_str_allocations():                                                                                                                                    [6/9733]
   ...:     StrAllocator.oldstr = str
   ...:     builtins.str = StrAllocator
   ...:

In [5]:   def test_log():
   ...:         t1 = str('t1')
   ...:         logger = logging.getLogger(__name__)
   ...:         patch_str_allocations()
   ...:         t2 = str('t2')
   ...:         print(type(t1))
   ...:         print(type(t2))
   ...:         print(isinstance(t1, str))
   ...:         print(isinstance(t2, StrAllocator))
   ...:         print(isinstance(t2, str))
   ...:         logger.debug(str('test'))
   ...:
In [6]: test_log()
<class 'str'>
<class '__main__.StrAllocator'>
False
True
True

Answer 1

If you insist on monkey-patching the built-in str with your own function, why don't you monkey-patch the isinstance() as well to ensure it treats your function as the built-in str? Something like:

def patch_str_allocations():
    old_str = str
    old_isinstance = builtins.isinstance

    def mystr(*args, **kwargs):
        return old_str(*args, **kwargs)

    def my_isinstance(o, t):
        if t is mystr:
            t = old_str
        return old_isinstance(o, t)

    builtins.str = mystr
    builtins.isinstance = my_isinstance

You may also want to check if t of my_isinstance() is a tuple and iterate over it to make sure you replace mystr with old_str as well.