Python requests fails to log in

Question

I appreciate the things you're doing here. Usually I am able to figure out my issues with the help of Stackoverflow, but this time I'm stuck. Hopefully you can help me!

The question is fairly simple: how to login on this webpage using Python's requests?

My steps:

1. Get the login url
2. Provide the login details. According to the HTML, I need to provide an 'email' and a 'password'.
3. Create a session and use post to login
4. Check HTML if login was successful

Unfortunately, the easy approach does not seem to work in this case. For example, the output of details is:

<script>
    dataLayer = [{
        'environment': 'production',
        'loggedIn': '0',
        'userCode': '',
        'rank': '',
        'totalBalance': '0',
        'overAgeCasino': '0'
    }];
</script>

Obviously, '0' of 'loggedIn' should change to '1' if the login was successful.

In an example I found that you might have to add a 'csrftoken' and that this can be found as a 'hidden' type in the HTML. However, the name in the type='hidden' part seems to have nothing to do with this and also has no value (link to screenshot of HTML). Somewhere else I read that a CSFR token is also stored in the CookieJar, but it is not there:

<RequestsCookieJar[<Cookie PHPSESSID=5dib6cf6kpvf29dsn725ljcec7 for .napoleongames.be/>, <Cookie locale=en_GB for .napoleongames.be/>, <Cookie user=false for .napoleongames.be/>]>

I find it hard to believe that logging in is not possible, but I've ran out of ideas. If someone knows how to do it with urllib(2), it is also useful. I'd rather not use Selenium as I haven't been able to run it smoothly.

Code:

 import requests
    from bs4 import BeautifulSoup
    from datetime import date

date_str = str(date.today())
login_url = 'https://en-gb.sports.napoleongames.be/user/login'
protected_url = 'proctected_url'

payload = {'email': '[email protected]',
           'password': '*********'}

with requests.Session() as session:
    session.get(login_url)
    login_page = session.post(login_url,
                          data=payload)

    html_body = BeautifulSoup(login_page.content, 'html.parser').find(
        name='body', attrs={'id': 'user_login'})
    details = html_body.findAll('script')[0]
    page = session.get(protected_url)

Headers:

{'Date': 'Wed, 06 Sep 2017 23:45:52 GMT', 'Server': 'Apache', 'Expires': 'Thu, 19 Nov 1981 08:52:00 GMT', 'Cache-Control': 'no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'Pragma': 'no-cache', 'X-Frame-Options': 'SAMEORIGIN', 'X-Cache-Page': 'MISS', 'Set-Cookie': 'locale=en_GB; Expires=Mon, 05-Mar-2018 23:45:52 GMT; Domain=.napoleongames.be; Path=/, user=false; expires=Mon, 05-Mar-2018 23:45:52 GMT; Max-Age=15552000; path=/; domain=.napoleongames.be, loyalty=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=.napoleongames.be', 'Strict-Transport-Security': 'max-age=15768000', 'Vary': 'Accept-Encoding', 'Content-Encoding': 'gzip', 'Content-Length': '10840', 'Connection': 'close', 'Content-Type': 'text/html'}

Answer 1

When submitting a form, you should take in consideration other fields inside the form tags not just the fields you need to fill up. In this case, when you look inside the page source, there is one more field that is being populated upon login.

You can try adding that in your payload:

payload = {'email': '[email protected]',
           'password': '*********'
           'buttons_app_service_user_login[buttonClicked]': 'buttons_app_service_user_login-save'}

Let me know if that worked or if you encounter other issues.