Would it be possible to route a GET request through a proxy by specifying the host as the proxy? Or would you have to set the destination of the packet?
I am trying to generate an HTTPRequestMessage and route it through a proxy. However, I do not have fine level control of setting the destination of the request being sent out.
An HTTP proxy acts as a high-performance content filter on traffic received by an HTTP client and HTTP server. The HTTP proxy protocol routes client requests from web browsers to the internet and supports rapid data caching.
To use a proxy in Python, first import the requests package. Next create a proxies dictionary that defines the HTTP and HTTPS connections. This variable should be a dictionary that maps a protocol to the proxy URL. Additionally, make a url variable set to the webpage you're scraping from.
A proxy server lessens network traffic by rejecting unwanted requests, forwarding requests to balance and optimize server workload, and fulfilling requests by serving data from cache rather than unnecessarily contacting the true destination server. HTTP Server has proxy server capabilities built in.
I was able to add a proxy to HttpClient, HttpWebRequest and HttpRequestMessage. They do not have to be used together, but I just found two ways of making HTTP Requests with proxy. To do this in windows store/metro applications, you would have to implement IWebProxy.
Take a look at this for implementing IWebProxy: http://social.msdn.microsoft.com/Forums/windowsapps/en-US/6e20c2c0-105c-4d66-8535-3ddb9a048b69/bug-missing-type-webproxy-cant-set-proxy-then-where-is-the-appconfig
Then all you need to do is set the proxy for HttpClient or HttpWebRequest:
HttpClient:
HttpClientHandler aHandler = new HttpClientHandler();
IWebProxy proxy = new MyProxy(new Uri("http://xx.xx.xx.xxx:xxxx"));
proxy.Credentials = new NetworkCredential("xxxx", "xxxx");
aHandler.Proxy = proxy;
HttpClient client = new HttpClient(aHandler);
HttpWebRequest:
HttpWebRequest webrequest = (HttpWebRequest)WebRequest.CreateHttp(uri);
IWebProxy proxy = new MyProxy(new Uri("http://xx.xx.xx.xxx:xxxx"));
proxy.Credentials = new NetworkCredential("xxxx", "xxxx");
webrequest.Proxy = proxy;
HttpRequestMessage
Once you construct an HttpRequestMessage, you can use the method above (HttpClient) to send this request message and it will be routed through the proxy without any additional work.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With