I need to add custom parameters to an URL query string using Python
Example: This is the URL that the browser is fetching (GET):
/scr.cgi?q=1&ln=0
then some python commands are executed, and as a result I need to set following URL in the browser:
/scr.cgi?q=1&ln=0&SOMESTRING=1
Is there some standard approach?
The results of urlparse() and urlsplit() are actually namedtuple instances. Thus you can assign them directly to a variable and use url_parts = url_parts. _replace(query = …) to update it.
Query parameters are a defined set of parameters attached to the end of a url. They are extensions of the URL that are used to help define specific content or actions based on the data being passed. To append query params to the end of a URL, a '? ' Is added followed immediately by a query parameter.
compile("/title/tt")) for link in all_links: print(link. get("href")) all_urls = link. get("href") url_test = 'http://www.imdb.com/{}/' for i in all_urls: urls = url_test. format(i) print(urls) this is the code to scrape the urls of all the 250 movies from the main url.
You can use urlsplit()
and urlunsplit()
to break apart and rebuild a URL, then use urlencode()
on the parsed query string:
from urllib import urlencode from urlparse import parse_qs, urlsplit, urlunsplit def set_query_parameter(url, param_name, param_value): """Given a URL, set or replace a query parameter and return the modified URL. >>> set_query_parameter('http://example.com?foo=bar&biz=baz', 'foo', 'stuff') 'http://example.com?foo=stuff&biz=baz' """ scheme, netloc, path, query_string, fragment = urlsplit(url) query_params = parse_qs(query_string) query_params[param_name] = [param_value] new_query_string = urlencode(query_params, doseq=True) return urlunsplit((scheme, netloc, path, new_query_string, fragment))
Use it as follows:
>>> set_query_parameter("/scr.cgi?q=1&ln=0", "SOMESTRING", 1) '/scr.cgi?q=1&ln=0&SOMESTRING=1'
Use urlsplit()
to extract the query string, parse_qsl()
to parse it (or parse_qs()
if you don't care about argument order), add the new argument, urlencode()
to turn it back into a query string, urlunsplit()
to fuse it back into a single URL, then redirect the client.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With