List all third party packages and theirselves functions used in a Python file

Question

I have many python packages, wrote by my colleagues and I want to write a tool to check which third packages they rely on.

Like this

 #it is my package, need to check,call it example.py
 #We have more than one way to import a package, It is a problem need to consider too

 from third_party_packages import third_party_function

 def my_function(arg):
    return third_party_function(arg)

and the tool should work like this

result = tool(example.py)
#this result should be a dict like this structure
#{"third_party_function":["my_function",]}
#Means "my_function" relies on "third_party_function"

I have no idea how to do that, all I can come up implementation of this tool is read a Python file line by one line as string, and use regex to compare it. Could you give me some advises?

If you don't know what I mean, please comment your question, I will fix it as soon as possible. Thanks!

Answer 1

You can parse your files with the ast module and check all Import and ImportFrom statements.

To give you an idea, here's an example:

>>> import ast
>>> tree = ast.parse('import a; from b import c')
>>> tree.body
[<_ast.Import object at 0x7f3041263860>, <_ast.ImportFrom object at 0x7f3041262c18>]
>>> tree.body[0].names[0].name
'a'
>>> tree.body[1].module
'b'
>>> tree.body[1].names[0].name
'c'

Your script could work like this:

1. Parse the source file via ast.parse
2. Visit each node using ast.walk()
3. If a node is an Import or ImportFrom object, then inspect the names and do what you have to do.

Using ast is way much easier and more robust than regular expressions or a custom parser.