Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python requests urlencode not working?

EDIT: I figured out the problem. The # in #user_sex is not being converting to %23 by python requests. Is there a way to force python requests to convert # to %23 or will I just have to hand code that part?

I'm trying to make a facebook fql multiquery. When I use the fql_url below

fql_url = (
    'https://graph.facebook.com/fql?q='
    '{"user_sex":"SELECT sex FROM user WHERE uid=me()",'
    '"friends":"SELECT uid, name FROM user WHERE uid IN '
    '(SELECT uid2 FROM friend WHERE uid1 = me()) '
    'AND not (sex in (SELECT sex FROM #user_sex)) '
    ' ORDER BY name"}'
    '&access_token='+access_token
)

and run requests.get(fql_url), the json returned is

{u'error': {
    u'code': 601,
    u'message': u"(#601) Parser error: unexpected '{' at position 0.",
    u'type': u'OAuthException'}
}

However when I hand code the fql_url as this

fql_url = (
    'https://graph.facebook.com/fql?q=%7B%22'
    'user_sex%22:%22SELECT%20sex%20FROM%20user%20WHERE%20uid=me()%22,%22'
    'friends%22:%22SELECT%20uid,%20name%20FROM%20user%20WHERE%20uid%20IN%20'
    '(SELECT%20uid2%20FROM%20friend%20WHERE%20uid1%20=%20me())%20'
    'AND%20not%20(sex%20in%20(select%20sex%20from%20%23user_sex))%20%20'
    'ORDER%20BY%20name%22%7D&'
    'access_token='+access_token
)

everything works (the json has the desired data).

I've compared both the first fql_url and the hand coded fql_url and both should result in the same url being used to get the json. Is the requests urlencode not working or am I doing something wrong here?

like image 679
bab Avatar asked Jun 29 '13 00:06

bab


People also ask

Does Python request URL encode?

In Python, we can URL encode a query string using the urlib. parse module, which further contains a function urlencode() for encoding the query string in URL. The query string is simply a string of key-value pairs.

How do you escape a URL in Python?

Use urllib.parse. quote(input) to escape a URL input string.

What is Urlencode in Python?

urlencode() function. This is a convenience function which takes a dictionary of key value pairs or a sequence of two-element tuples and uses the quote_plus() function to encode every value. The resulting string is a series of key=value pairs separated by & character.

What is the use of Urlencode?

The urlencode() function is an inbuilt function in PHP which is used to encode the url. This function returns a string which consist all non-alphanumeric characters except -_.


1 Answers

The problem is, that # is indeed a valid character in an URL. It denotes the fragment part. As the fragment is always resolved by the useragent it is never sent to the server. You can try this:

>>> import urllib3
>>> urllib3.util.parse_url(fql_url)
Url(scheme='https', auth=None, host='graph.facebook.com', port=None, path='/fql',
    query='q={"user_sex":"SELECT sex FROM user WHERE uid=me()","friends":"SELECT uid, name FROM user WHERE uid IN (SELECT uid2 FROM friend WHERE uid1 = me()) AND not (sex in (SELECT sex FROM ',
    fragment='user_sex))  ORDER BY name"}')

As you can see, the last part of your URL ended up being parsed as the fragment.

Edit:

The most convenient way would probably be to let requests do all the encoding.

import requests
s = requests.Session()
s.params = {'access_token': 'foobarbaz'} # so you don't have to specify it every time
query = ('{"user_sex":"SELECT sex FROM user WHERE uid=me()",'
         '"friends":"SELECT uid, name FROM user WHERE uid IN '
         '(SELECT uid2 FROM friend WHERE uid1 = me()) '
         'AND not (sex in (SELECT sex FROM #user_sex)) '
         ' ORDER BY name"}')
s.get('https://graph.facebook.com/fql', params={'q': query})
like image 127
t-8ch Avatar answered Sep 27 '22 20:09

t-8ch