How can I use python request to login this website?

Question

I'm quit new to Python and web-scraping. And I can't even achieve the first step of scraping a website: login. Before I try to use mechanize or selenium, I want to use requests first. Can someone help me?

The website I have been trying to log into is here.

For those who don't have an account and want to help me, I will provide the information I got by using Chrome.

The first POST: POST_1.JPG. The second POST: POST_2.JPG

And here is my code for the login attempt:

import requests

s = requests.Session()
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36'}
payload = {'username': 'USERNAME',
          'password': 'PASSWORD',
          'submit.x': '21',
          'submit.y': '12'}

s.post('https://ssologin.cuny.edu/oam/server/auth_cred_submit', data = payload, headers = headers)
target_page = s.get('https://home.cunyfirst.cuny.edu/psp/cnyepprd/EMPLOYEE/EMPL/h/?tab=DEFAULT', headers = headers)

if 'login' in target_page.url:
    print("Failed to Login")
else:
    print("Logged in Successfully!")

Thanks!!!

Answer 1

You just have to reproduce the page's behavior as it is:

from requests import Session

login_data = {
    'username' : 'YOUR_USERNAME_HERE',
    'password' : 'YOUR_PASSWORD_HERE',
    'submit.x' : '41',
    'submit.y' : '2',
}

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36',
    'Referer': 'https://ssologin.cuny.edu/cuny.html?resource_url=https%3A%2F%2Fhome.cunyfirst.cuny.edu%252Fpsp%252Fcnyepprd%252FEMPLOYEE%252FEMPL%252Fh%252F%3Ftab%253DDEFAULT',
}

s = Session()

s.get("https://home.cunyfirst.cuny.edu/psp/cnyepprd/EMPLOYEE/EMPL/h/?tab=DEFAULT")
s.post("https://m.addthis.com/live/red_lojson/100eng.json?sh=590&ph=1512&ivh=590&dt=5366&pdt=11272&ict=&pct=1&perf=widget%7C11273%7C1445%2Clojson%7C12846%7C236%2Csh%7C12848%7C79&rndr=render_toolbox%7C13088&cmenu=null&ppd=0&ppl=0&fbe=&xmv=0&xms=0&xmlc=0&jsfw=jquery&jsfwv=jquery-1.7.1&al=men&scr=0&scv=0&apiu=1&ba=3&sid=59646cef3b3fdec3&rev=v7.15.4-wp&pub=cunywebservices&dp=ssologin.cuny.edu&fp=cuny.html%3Fresource_url%3Dhttps%253A%252F%252Fhome.cunyfirst.cuny.edu%25252Fpsp%25252Fcnyepprd%25252FEMPLOYEE%25252FEMPL%25252Fh%25252F%253Ftab%25253DDEFAULT&pfm=0&icns=")
response = s.post("https://ssologin.cuny.edu/oam/server/auth_cred_submit", data=login_data, headers=headers)

if 'Sign out' in response.content:
    print "[*] Logged in successfully!"
else:
    print "[!] Couldn't login..."