Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python Rope: How to Find all missing imports and errors in all sub modules refactoring

I am trying to find all missing import statements and errors for each module and its sub modules.

Is there a dedicated tool for what I am trying to do?

The code that I wrote, but seems really terrible and maybe something like this exists already?:

import os
def find_missing_imports(walk):
    for items in walk:
        d = items[0]
        f_list = items[1]
        for f in f_list:
            module = f[:-3]
            # posix_path
            module_path = d.lstrip('.').replace('/','.').lstrip('.')
            try:
                __import__(module_path, fromlist=[module])
            except IndentationError, e:
                #print(f,e)
                pass
            except NameError, e:
                print(d,f,e)
                pass
            except Exception, e:
                print(f,e)
                pass

walk = [[root,files] for root,dirs,files in os.walk('.') for fn in files if  fn.endswith('.py')]
find_missing_imports(walk)

Outputs:

.[snip]
('./Sky_Group_Inventory_Scanner-wxpython/display_image/Dialogs', 'ImageSelectionFrame.py', NameError("name 'wx' is not defined",))
('./Sky_Group_Inventory_Scanner-wxpython/display_image/Dialogs', 'ItemSpecificsDialog.py', NameError("name 'wx' is not defined",))
('./Sky_Group_Inventory_Scanner-wxpython/display_image/Dialogs', 'ReturnCorrectWatchTitle.py', NameError("name 'wx' is not defined",))
.[snip]

My project before refactoring was a mess but sort of useful, now its broken after refactoring.

After reading 'The Pragmatic Programmer' based on suggestions from my initial post on codereview:

  • https://codereview.stackexchange.com/questions/46223/wxpython-item-information-scraper

I have been digging around in the source code of:

/usr/local/lib/python2.7/dist-packages/rope

Documentation for ROPE seems a little sparse. I have also been using Ninja-IDE, but haven't been able to find a solution for the problem that I am facing.

Overall I think I missed the boat on what refactoring is all about.

The current parent directory layout can be seen here.

enter image description here

In comparison to what it was before.

before

Any help, on filling in missing terminology, or on what I am even asking would be great.

Solution:

pylint -E /path/to/module

like image 526
jmunsch Avatar asked May 14 '14 02:05

jmunsch


1 Answers

pip install pylint

Point pylint to the folder/module in question:

pylint /path/to/module > pylint_output

Where /path/to/module is the location of the python module you're interested in looking at.

For example:

my_project
   |____  moduleA
             |____  __init__.py
             |____  A.py
   |____  moduleB
             |____  __init__.py
             |____  B.py
  • ./my_project/moduleA
  • ./my_project/moduleB

This will create a file with Global Evaluations on:

  • undefined-variable <-- This is what I was looking for in each file.
  • invalid-name
  • line-too-long
  • superfluous-parens
  • bad-whitespace
  • attribute-defined-outside-init
  • missing-docstring
  • bad-indentation
  • bad-continuation
  • broad-except
  • unused-argument
  • unused-import
  • unused-variable
  • no-self-use
  • no-member
  • fixme
  • unnecessary-pass
  • multiple-statements
  • too-many-statements
  • duplicate-code
  • too-many-locals
  • missing-final-newline
  • too-many-instance-attributes
  • too-many-branches
  • redefined-builtin
  • too-many-public-methods
  • syntax-error
  • relative-import
  • maybe-no-member
  • import-error
  • super-on-old-class
  • bare-except
  • undefined-loop-variable
  • too-many-return-statements
  • too-many-arguments
  • too-few-public-methods
  • star-args
  • reimported
  • indexing-exception
  • unreachable
  • too-many-lines
  • redefined-outer-name
  • property-on-old-class
  • pointless-string-statement
  • pointless-statement
  • old-style-class
  • no-name-in-module
  • global-variable-undefined
  • expression-not-assigned
  • bad-except-order
  • assignment-from-none

Of interest and the direct answer to my question is that within the pylint results there will be lines that have this sort of layout:

************* Module module_name.sub_module.class_name.method_name
R: line_no, column: Issue description 'some_name' (issue-type)
C: line_no, column: Issue description 'some_name' (issue-type)
W: line_no, column: Issue description 'some_name' (issue-type)
E: line_no, column: Issue description 'some_name' (issue-type)
F: line_no, column: Issue description 'some_name' (issue-type)
************* Module module_name.sub_module.class_name.method_name
R: line_no, column: Issue description 'some_name' (issue-type)
C: line_no, column: Issue description 'some_name' (issue-type)
W: line_no, column: Issue description 'some_name' (issue-type)
E: line_no, column: Issue description 'some_name' (issue-type)
F: line_no, column: Issue description 'some_name' (issue-type)    
  • [R]efactor for a “good practice” metric violation
  • [C]onvention for coding standard violation
  • [W]arning for stylistic problems, or minor programming issues
  • [E]rror for important programming issues (i.e. most probably bug)
  • [F]atal for errors which prevented further processing

So an issue-type of (undefined-variable) in most of my cases indicate modules that have not been imported. pylint -E /path/to/module will return only the undefined-variable errors.

like image 73
jmunsch Avatar answered Oct 18 '22 22:10

jmunsch