Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python: urllib2 how to send cookie with urlopen request

Tags:

python

urllib2

I am trying to use urllib2 to open url and to send specific cookie text to the server. E.g. I want to open site Solve chess problems, with a specific cookie, e.g. search=1. How do I do it?

I am trying to do the following:

import urllib2 (need to add cookie to the request somehow) urllib2.urlopen("http://chess-problems.prg") 

Thanks in advance

like image 832
Oleg Tarasenko Avatar asked Jul 26 '10 12:07

Oleg Tarasenko


People also ask

What does Urllib request Urlopen do?

request is a Python module for fetching URLs (Uniform Resource Locators). It offers a very simple interface, in the form of the urlopen function. This is capable of fetching URLs using a variety of different protocols.

Is Urllib request the same as request?

Despite the similar name, they are unrelated: they have a different design and a different implementation. urllib was the original Python HTTP client, added to the standard library in Python 1.2. Earlier documentation for urllib can be found in Python 1.4.


2 Answers

Cookie is just another HTTP header.

import urllib2 opener = urllib2.build_opener() opener.addheaders.append(('Cookie', 'cookiename=cookievalue')) f = opener.open("http://example.com/") 

See urllib2 examples for other ways how to add HTTP headers to your request.

There are more ways how to handle cookies. Some modules like cookielib try to behave like web browser - remember what cookies did you get previously and automatically send them again in following requests.

like image 164
Messa Avatar answered Sep 19 '22 13:09

Messa


Maybe using cookielib.CookieJar can help you. For instance when posting to a page containing a form:

import urllib2 import urllib from cookielib import CookieJar  cj = CookieJar() opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) # input-type values from the html form formdata = { "username" : username, "password": password, "form-id" : "1234" } data_encoded = urllib.urlencode(formdata) response = opener.open("https://page.com/login.php", data_encoded) content = response.read() 

EDIT:

After Piotr's comment I'll elaborate a bit. From the docs:

The CookieJar class stores HTTP cookies. It extracts cookies from HTTP requests, and returns them in HTTP responses. CookieJar instances automatically expire contained cookies when necessary. Subclasses are also responsible for storing and retrieving cookies from a file or database.

So whatever requests you make with your CookieJar instance, all cookies will be handled automagically. Kinda like your browser does :)

I can only speak from my own experience and my 99% use-case for cookies is to receive a cookie and then need to send it with all subsequent requests in that session. The code above handles just that, and it does so transparently.

like image 20
Morten Jensen Avatar answered Sep 21 '22 13:09

Morten Jensen