Make requests using Python over Tor

Question

I want to make multiple GET requests using Tor to a webpage. I want to use a different ipaddress for each request.

import socks
import socket
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 9150)
socket.socket = socks.socksocket
import requests
print (requests.get('http://icanhazip.com')).content

Using this, I made one request. How can I change the ipaddress to make another?

Answer 1

There are 2 aspects to your question -

1. Making requests using Tor
2. Renewing the connection as per requirement (in your case, after every request)

---

Part 1

The first one is easy to do with the latest (upwards of v2.10.0) requests library with an additional requirement of requests[socks] for using the socks proxy.

Installation -

pip install requests[socks]

Basic usage -

import requests

def get_tor_session():
    session = requests.session()
    # Tor uses the 9050 port as the default socks port
    session.proxies = {'http':  'socks5://127.0.0.1:9050',
                       'https': 'socks5://127.0.0.1:9050'}
    return session

# Make a request through the Tor connection
# IP visible through Tor
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
# Above should print an IP different than your public IP

# Following prints your normal public IP
print(requests.get("http://httpbin.org/ip").text)

---

Part 2

To renew the Tor IP, i.e. to have a fresh visible exit IP, you need to be able to connect to the Tor service through it's ControlPort and then send a NEWNYM signal.

Normal Tor installation does not enable the ControlPort by default. You'll have to edit your torrc file and uncomment the corresponding lines.

ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:05834BCEDD478D1060F1D7E2CE98E9C13075E8D3061D702F63BCD674DE

Please note that the HashedControlPassword above is for the password "password". If you want to set a different password, replace the HashedControlPassword in the torrc by noting the output from tor --hash-password "<new_password>" where <new_password> is the password that you want to set.

................................................................................

Warning for Windows users: see post here.

There is an issue on windows where the setting for the controlport in the torrc file is ignored if tor was installed using the following command:

tor --service install

To resolve the issue, after editing your torrc file, type the following commands:

tor --service remove
tor --service install -options ControlPort 9051

................................................................................

Okay, so now that we have Tor configured properly, you will have to restart Tor if it is already running.

sudo service tor restart

Tor should now be up & running on the 9051 ControlPort through which we can send commands to it. I prefer to use the official stem library to control Tor.

Installation -

pip install stem

You may now renew the Tor IP by calling the following function.

Renew IP -

from stem import Signal
from stem.control import Controller

# signal TOR for a new connection 
def renew_connection():
    with Controller.from_port(port = 9051) as controller:
        controller.authenticate(password="password")
        controller.signal(Signal.NEWNYM)

To verify that Tor has a new exit IP, just rerun the code from Part 1. For some reason unknown to me, you need to create a new session object in order to use the new IP.

session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)

Answer 2

Here is the code you want to use (download the stem package using pip install stem)

from stem import Signal
from stem.control import Controller

with Controller.from_port(port = 9051) as controller:
    controller.authenticate(password='your password set for tor controller port in torrc')
    print("Success!")
    controller.signal(Signal.NEWNYM)
    print("New Tor connection processed")

Good luck and hopefully that works.

Answer 3

You can try pure-python tor protocol implementation Torpy. No need original Tor client or Stem dependency at all.

$ pip3 install torpy[requests]
...

$ python3.7
>>> from torpy.http.requests import TorRequests
>>> with TorRequests() as tor_requests:
...    print("build circuit")
...    with tor_requests.get_session() as sess:
...        print(sess.get("http://httpbin.org/ip").json())
...        print(sess.get("http://httpbin.org/ip").json())
...    print("renew circuit")
...    with tor_requests.get_session() as sess:
...        print(sess.get("http://httpbin.org/ip").json())
...        print(sess.get("http://httpbin.org/ip").json())
...
build circuit
{'origin': '23.129.64.190, 23.129.64.190'}
{'origin': '23.129.64.190, 23.129.64.190'}
renew circuit
{'origin': '198.98.50.112, 198.98.50.112'}
{'origin': '198.98.50.112, 198.98.50.112'}

So each time when you getting new session you get new identity (basically you get new circuit with new exit node). See more examples at readme file https://github.com/torpyorg/torpy

Answer 4

You can use torrequest library (shameless plug). It's available on PyPI.

from torrequest import TorRequest

with TorRequest() as tr:
  response = tr.get('http://ipecho.net/plain')
  print(response.text)  # not your IP address

  tr.reset_identity()

  response = tr.get('http://ipecho.net/plain')
  print(response.text)  # another IP address, not yours

Answer 5

Requests supports proxies using the SOCKS protocol from version 2.10.0.

import requests
proxies = {
    'http': 'socks5://localhost:9050',
    'https': 'socks5://localhost:9050'
}
url = 'http://httpbin.org/ip'
print(requests.get(url, proxies=proxies).text)