Determine if function includes a specific method

Question

I would like to write some code to check student submissions to ensure that a given function includes np.random.choice.

For example:

import numpy as np

def checkme(z, x=[1,2,3], y=4):
  tmp = np.random.choice(x, size=y)
  if z:
    print("z")
  return(list(tmp))

I have seen that I can use calls like

tmp = inspect.signature(checkme)
for param in tmp.parameters.values():
  print(param.name, ",", param.default)

To determine the parameters and values, which is great, but I want to take this one step further and ensure that the body of the function included a specific function or method. So above, I would want to ensure the students' code included np.random.choice.

How can I access the body of the function to "inspect" and determine if this is True or False?

Answer 1

You can temporarily replace the method you want to check with a wrapper that will let you know if it's called or not through a global variable (or something else of your choosing). I think this is the only real solution, as both checking with string matching and checking the disassembeld code like I suggested in my other answer is error prone and will inevitably miss edge cases.

Here's an example:

class Checker:
    def check(self, func, funcargs):
        real_np_random_choice = np.random.choice
        self.called = False

        def wrapper(*args, **kwargs):
            self.called = True
            return real_np_random_choice(*args, **kwargs)

        np.random.choice = wrapper
        func(*funcargs)
        np.random.choice = real_np_random_choice

        return self.called

Checker().check(checkme, (3, [1,2,3], 4)) # -> True

I'm using a class here only because I need to carry the result out of wrapper somehow. This could also be done with a global variable of course.

A more general solution to check that a given method of a given module is called would be:

class Checker:
    def __init__(self, module, method):
        self.module = module
        self.method = method

    def check(self, func, funcargs):
        real_method = getattr(self.module, self.method)
        self.called = False

        def wrapper(*args, **kwargs):
            self.called = True
            return real_method(*args, **kwargs)

        setattr(self.module, self.method, wrapper)
        func(*funcargs)
        setattr(self.module, self.method, real_method)

        return self.called

c = Checker(np.random, 'choice')
print(c.check(checkme, (3, [1,2,3], 4)))