How to login to a website with python and mechanize

Question

i'm trying to log in to the website http://www.magickartenmarkt.de and do some analyzing in the member-area (https://www.magickartenmarkt.de/?mainPage=showWants). I saw other examples for this, but i don't get why my approaches didn't work. I identified the right forms for the first approach, but it's not clear if it worked. In the second approach the returing webpage shows me that i don't have access to the member area.

I would by glad for any help.

import urllib2 import cookielib import urllib import requests import mechanize from mechanize._opener import urlopen from mechanize._form import ParseResponse  USERNAME = 'Test' PASSWORD = 'bla123' URL      = "http://www.magickartenmarkt.de"  # first approach request = mechanize.Request(URL) response = mechanize.urlopen(request) forms = mechanize.ParseResponse(response, backwards_compat=False) # I don't want to close?! #response.close()  # Username and Password are stored in this form form = forms[1]  form["username"] = USERNAME form["userPassword"] = PASSWORD  #proof entering data has worked user = form["username"]  # a string, NOT a Control instance print user pw = form["userPassword"]  # a string, NOT a Control instance print pw #is this the page where I will redirected after login? print urlopen(form.click()).read ()   #second approach cj = cookielib.CookieJar() opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) login_data = urllib.urlencode({'username' : USERNAME, 'userPassword': PASSWORD})  #login response_web = opener.open(URL, login_data)  #did it work? for me not.... resp = opener.open('https://www.magickartenmarkt.de/?mainPage=showWants') print resp.read() 

Answer 1

Why not use a browser instance to facilitate navigation? Mechanize also has the ability to select particular forms (e.g. nr = 0 will select the first form on the page)

browser = mechanize.Browser() browser.open(YOUR URL) browser.select_form(nr = 0) browser.form['username'] = USERNAME browser.form['password'] = PASSWORD browser.submit()