Making python imports more structured?

Question

The code works but looks messy so this might be a code review question where I didn't study enough of pythons conventions to know how to structure and organize the beginning of my file more pythonic. I basically just pasted in imports so they could be duplicates, not needed anymore or wrongly ordered. Can you advice anything how to structure my imports or can I leave code like this to focus on my own functions?

File 1:

from __future__ import with_statement
import logging
import os
from google.appengine.api.users import is_current_user_admin, UserNotFoundError
import time
import cgi
import geo.geotypes
import main
import captcha
from google.appengine import api
from google.appengine.runtime import DeadlineExceededError
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext.blobstore import BlobInfo
from google.appengine.ext.db import djangoforms
from django import forms
from django.core.exceptions import ValidationError
from django.utils import translation
from datetime import datetime, timedelta
os.environ['DJANGO_SETTINGS_MODULE'] = 'conf.settings'
from django.conf import settings
from django.template import RequestContext
from util import I18NHandler
import util
from google.appengine.api import urlfetch, taskqueue
from django.template.defaultfilters import register
from django.utils import simplejson as json
from functools import wraps
from google.appengine.api import urlfetch, taskqueue, users, images
from google.appengine.ext import db, webapp, search, blobstore
from google.appengine.ext.webapp import util, template
from google.appengine.runtime import DeadlineExceededError
from random import randrange
import Cookie
import base64
import cgi
import conf
import datetime
import hashlib
import hmac
import logging
import time
import traceback
import urllib
import twitter_oauth_handler
from twitter_oauth_handler import OAuthClient
from geo.geomodel import GeoModel
from django.utils.translation import gettext_lazy as _
webapp.template.register_template_library('common.templatefilters')

File 2 (there's are several instructions here I don't understand):

from __future__ import with_statement
                # -*- coding: utf-8 -*-
import facebookconf
import os, wsgiref.handlers
os.environ[u'DJANGO_SETTINGS_MODULE'] = u'conf'
import util
import time
import logging
import urllib
import wsgiref.handlers
import appengine_admin
import cgi
import captcha
import re
import hashlib
import string
import hmac
import twitter_oauth_handler
from twitter_oauth_handler import OAuthClient
os.environ['DJANGO_SETTINGS_MODULE'] = 'conf.settings'
from geo.geomodel import GeoModel
from google.appengine.dist import use_library
from google.appengine.ext import blobstore, webapp, db, search
# template import must be run before other Django modules imports
from google.appengine.ext.webapp import blobstore_handlers, util, template
from google.appengine.ext.blobstore import BlobInfo
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.api import files, images, mail, memcache, users
from django.conf import settings
# Force Django reload
settings._target = None
from util import I18NHandler, FacebookBaseHandler
from google.appengine.ext.db import djangoforms
from django.utils import translation
from django.utils import simplejson as json
from django.contrib.formtools.preview import FormPreview
from random import choice
from urllib import quote
from google.appengine.api.users import is_current_user_admin, UserNotFoundError
from google.appengine.api import urlfetch
import random
import datetime
from datetime import timedelta
from django.utils.translation import gettext_lazy as _
from django.template import defaultfilters

How do I know when an import is no longer used since the function was moved or removed? Why can't I specify the same import for multiple files at one place and I must spec the same import in both files? I can imagine moveing handling imports to a separate file i.e. imports.yaml to specify imports for all python files in that directory or likewise.

Answer 1

Adding to the answers above, what we like to do in addition to the PEP8, is organize things alphabetically (but import comes before from) like this:

# 1. standard libraries alphabetical with 'imports' before 'from' like this:
import csv
import logging
from collections import defaultdict
from datetime import date
#(blank line)

# 2. third party packages alphabetical with imports first
# next is the third party packages (also alphabetical as described above)
import numpy as  np 
import pandas as pd
from statsmodels import api as sm
#(blank line again)

# 3. your own stuff (also alphabetical) 
from my_other_folder import my_other_file

Answer 2

PEP 8 - Style Guide for Python code recomends to order your imports in following order:

1. Standard library imports
2. - blank line -
3. google sdk imports
4. - blank line -
5. django imports
6. - blank line -
7. your own code imports

Import only things you use in code. Remove unused imports. You can use one of these tools to detect unused imports: Pydev on Eclipse / pyflakes / pylint

You have quite a lot imports. How big is your actual code? It might be a good idea to split it into few modules.

Why can't you import single time in single file? Well you actually could do it like this:

WARNING: THIS EXAMPLE ILLUSTRATES BAD CODING PRACTICES

import_all.py:

    import a
    import b
    import c

other.py:

     from import_all import *

But please don't do that. It is against all good practices of Python development and against The Zen of Python:

Explicit is better than implicit.

...

Namespaces are one honking great idea -- let's do more of those!

I also recommend you to read the Python documentation on modules and something about Python namespaces.

Answer 3

I wouldn’t worry about it. Thnk of the imports as being there not to be read as such, but to provide a cross-reference: when someone comes across an unqualified identifier further down, and wonders where it came from, they can just search backwards for the first occurrence, and if it’s not defined locally in the file, they should hit a mention in an import statement which will tell them where it came from.

Conversely, you can check which imports are unused by searching forwards from the import to see if there are any mentions of the imported identifier; if not, then it should be safe to remove it.

Note this doesn’t work with wildcard imports. Don’t use wildcard imports.

Answer 4

PEP8 has a section on Imports (that I can't link directly).

Basically, for organizing, here's what you want to do:

Imports should be grouped in the following order:

1. standard library imports
2. related third party imports
3. local application/library specific imports

You should put a blank line between each group of imports.

Oh, I believe PyDev for Eclipse has an "organize imports" command.