How can I use POST from requests module to login to Github?

Question

I have tried logging into GitHub using the following code:

url = 'https://github.com/login'

headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36',
          'login':'username',
          'password':'password',
          'authenticity_token':'Token that keeps changing',
          'commit':'Sign in',
          'utf8':'%E2%9C%93'
}

res = requests.post(url)
print(res.text)

Now, res.text prints the code of login page. I understand that it maybe because the token keeps changing continuously. I have also tried setting the URL to https://github.com/session but that does not work either.

Can anyone tell me a way to generate the token. I am looking for a way to login without using the API. I had asked another question where I mentioned that I was unable to login. One comment said that I am not doing it right and it is possible to login just by using the requests module without the help of Github API.

ME:

So, can I log in to Facebook or Github using the POST method? I have tried that and it did not work.

THE USER:

Well, presumably you did something wrong

Can anyone please tell me what I did wrong?

After the suggestion about using sessions, I have updated my code:

s = requests.Session()
headers = {Same as above}

s.put('https://github.com/session', headers=headers)    
r = s.get('https://github.com/')

print(r.text)

I still can't get past the login page.

Answer 1

I think you get back to the login page because you are redirected and since your code doesn't send back your cookies, you can't have a session.

You are looking for session persistance, requests provides it :

Session Objects The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance, and will use urllib3's connection pooling. So if you're making several requests to the same host, the underlying TCP connection will be reused, which can result in a significant performance increase (see HTTP persistent connection).

s = requests.Session()

s.get('http://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get('http://httpbin.org/cookies')

print(r.text)
# '{"cookies": {"sessioncookie": "123456789"}}'

http://docs.python-requests.org/en/master/user/advanced/

Answer 2

Actually in post method the request parameters should be in request body, not in header.So the login data should be in data parameter.

For github, authenticity token is present in value attribute of an input tag which is extracted using BeautifulSoup library.

This code works fine

import requests
from getpass import getpass
from bs4 import BeautifulSoup

headers = {
 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36'
}

login_data = {
'commit': 'Sign in',
'utf8': '%E2%9C%93',
'login': input('Username: '),
'password': getpass()
}
url = 'https://github.com/session'
session = requests.Session()
response = session.get(url, headers=headers)

soup = BeautifulSoup(response.text, 'html5lib')
login_data['authenticity_token'] = soup.find(
'input', attrs={'name': 'authenticity_token'})['value']
response = session.post(url, data=login_data, headers=headers)
print(response.status_code)
response = session.get('https://github.com', headers=headers)
print(response.text)