Store python requests session in persistent storage

Question

I'm using python requests library. My application performs simple get request from a site and prints results.

The site requires authorization with ntlm. Fortunately I can rely on HttpNtlmAuth, which works fine.

session = requests.Session()
session.auth = HttpNtlmAuth(domain + "\\" + username,
                            password,
                            session)

But if application is executed several times - each time I need to ask for username and password. It is very uncomfortable. Storing credentials is undesirable.

Could I store session object itself and reuse it several times? From server point of view - it should be fine.

Is there a way to pickle and unpickle session?

Answer 1

If you use the dill package, you should be able to pickle the session where pickle itself fails.

>>> import dill as pickle
>>> pickled = pickle.dumps(session)
>>> restored = pickle.loads(pickled)

Get dill here: https://github.com/uqfoundation/dill

Actually, dill also makes it easy to store your python session across restarts, so you could pickle your entire python session like this:

>>> pickle.dump_session('session.pkl')

Then restart python, and pick up where you left off.

Python 2.7.8 (default, Jul 13 2014, 02:29:54) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill as pickle
>>> pickle.load_session('session.pkl')
>>> restored
<requests.sessions.Session object at 0x10c012690>