Scan every possible port in a host using Python

Question

I'm writing a program which needs to scan all 65535 ports in a host searching for the ones which are open. This is what I have so far, and it works, but it produces different results every time I execute the script, why is this happening?

def check_open_port(host, port):
    s = socket.socket()
    s.settimeout(0.1)
    # the SO_REUSEADDR flag tells the kernel to reuse a local 
    # socket in TIME_WAIT state, without waiting for its natural
    # timeout to expire.
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    try:
        code = s.connect_ex((host, port))
        s.close()

        if code == 0:
            return True
        else:
            return False
    except socket.error:
        return False


def get_open_ports(host, max_port=65535):
    open_ports = []

    def worker(port):
        if check_open_port(host, port):
            open_ports.append(port)


    pool = ThreadPoolExecutor(max_workers=10000)
    [pool.submit(worker, port) for port in range(1, max_port + 1)]
    pool.shutdown(wait=True)

    return open_ports

For example, in a host with ports 22, 80, and 443 open, sometimes I get this response:

[22, 80]

and sometimes I get:

[22, 80, 443]

or even:

[22]

Hosts with more open ports produce more combinations.

I have played with max_workers and settimeout() values, but I can't get it to work well. The only time it worked was without using threads, but obviously it took ages to complete, I need to use them.

Am I missing something? Is there another way to implement this?

Answer 1

2 questions here:

1. Am I missing something
2. Is there another way to implement this?

Am I missing something

I think it’s worth checking error codes here:

if code == 0: 
    return True
else:
    return False

given that you’re trying to run a pool of ! 10K threads all kind of errors might follow here - i.e. reach some system/your user limits (check out ulimit -a) and you would treat such errors as a closed port without a notice. It might explain unstable results you experience.

BTW on my MacBook the results are consistent (checking against my live server on a VPS hosting)

I’d also pick fewer number of threads - 10K is an overkill. For example, here are the default values suggested in the python docs:

Changed in version 3.5: If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5, assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor

Is there another way to implement this?

First of all, there’s no need to use threads/processes - non-blocking sockets + event multiplexors like epoll are around for years so you’d be able to get away without having additional threads/processed whatsoever.

The method to connect/close is also suboptimal, because you just need to check if a port is open or not - you don’t need a full-on TCP connection here.

In the simplest case, you just need to send a SYN segment and check what the server would respond.

Here’s a good article with a dozen of methods using scapy

Scapy is a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more. It can easily handle most classical tasks like scanning, tracerouting, probing, unit tests, attacks or network discovery (it can replace hping, 85% of nmap, arpspoof, arp-sk, arping, tcpdump, tethereal, p0f, etc.).

Here’s one of the methods description (“TCP connect scan”):

The client sends the first handshake using the SYN flag and port to connect to the server in a TCP packet. If the server responds with a RST instead of a SYN-ACK, then that particular port is closed on the server

And one more method (“TCP stealth scan”):

This technique is similar to the TCP connect scan. The client sends a TCP packet with the SYN flag set and the port number to connect to. If the port is open, the server responds with the SYN and ACK flags inside a TCP packet. But this time the client sends a RST flag in a TCP packet and not RST+ACK, which was the case in the TCP connect scan. This technique is used to avoid port scanning detection by firewalls

Of course if just want to play with sockets/threads your approach would also be fine even without pcap/scapy

Answer 2

I tried your code on jupyter notebook and I always get the same set of ports:

get_open_ports('127.0.0.1')

Output:

[133, 200, 144...60700]

Could it be possible that a different number of ports are open at a particular time for the host being queried?

To verify for a small set of ports, I reduced max_port to 10000 and I still get the same set of ports every time:

def get_open_ports(host, max_port=10000):
open_ports = []

def worker(port):
    if check_open_port(host, port):
        open_ports.append(port)

with ThreadPoolExecutor(max_workers=10000) as executor:
    [executor.submit(worker, port) for port in range(1, max_port + 1)]
    executor.shutdown(wait=True)
return open_ports

get_open_ports('127.0.0.1')

Output: [150, 900, 1035, 7789]

Note: I've changed the port numbers for security sake.

EDIT:

def get_open_ports(host, max_port=65535):
    open_ports = []

    def worker(port):
        if check_open_port(host, port):
            open_ports.append(port)

# We can use a with statement to ensure threads are cleaned up promptly
    with ThreadPoolExecutor(max_workers=100) as executor:
        print('main:starting')
        wait_for=[executor.submit(worker,port) for port in range(1, max_port + 1)]
        for f in as_completed(wait_for):
            print('main: result: {}'.format(f.result())) #check result on each thread execution

#         executor.shutdown(wait=True)  #not required when using the 'with' statement
    return len(open_ports)

test = get_open_ports('45.60.112.163') #hostname for www.indracompany.com

#max_workers not defined & max_port=10000
# len(test)     #test1: 148
# len(test)     #test 2: 79

#max_workers = 10000 & max_port=65535
# len(test)      #test1: 1
# len(test)      #test2:1
# len(test)      #test3:1

#max_workers = 20000 & max_port=65535

# len(test)  #test1: 14
# len(test)  #test2:1
# len(test)  #test3: 1
# len(test)  #test4:1

#max_workers not defined & max_port=65535 #quite time-consuming
# len(test)   #test1: 63

EDIT 2: A more reliable solution

As suggested by @Tarun, Python's python-nmap library does a better job at scanning hosts.

The below solution gives an accurate result, however, I observed a significant compromise on performance as the range of port discovery increases. Perhaps, threading could be incorporated into the code to improve performance. I've also imported the time library to get the program execution time in the end. This can be used for comparison purposes when testing for performance.

# The python-nmap library helps to programmatically manipulate scanned results of nmap to automate port scanning tasks. 
# To use this library you must have the Nmap software installed. This can be installed from https://nmap.org/download.html.
# Network Mapper (Nmap) is a free and open-source tool used for network discovery and security auditing. 
# It runs on all major computer operating systems, and official binary packages are available for Linux, Windows, and Mac OS X.
# For Windows 7 and later, you must also upgrade 'NCap' from https://nmap.org/npcap/ 
# For Windows, make sure nmap.exe is added to PATH.
# When you're ready, pip install python-nmap

import time
import nmap
nm = nmap.PortScanner() #initialize PortScanner object
host = '45.60.112.163'  #specify host
nm.scan(host, '1-100') #run the scan, specify host and range of ports to scan

#Optional steps for verification:

#Output: nmap -oX - -p 1-100 -sV 45.60.112.163
print(nm.command_line()) #command_line command to execute on nmap command prompt

#Output: {'tcp': {'method': 'syn', 'services': '1-100'}}
print(nm.scaninfo())   #nmap scan information

#Now we can scan all hosts
#From Official documentation at https://xael.org/pages/python-nmap-en.html
start_time = time.time()   #To get program execution time
for host in nm.all_hosts(): 
    print('----------------------------------------------------')
    print('Host : %s (%s)' % (host, nm[host].hostname()))
    print('State : %s' % nm[host].state())
    for proto in nm[host].all_protocols():
        print('----------')
        print('Protocol : %s' % proto)
        lport = nm[host][proto].keys()
        for key in sorted(lport):
            for port in lport:
                print ('port : %s\tstate : %s' % (port, nm[host][proto][port]['state']))
print('Execution time: %s seconds' % (time.time() - start_time))

    #Output:
    ----------------------------------------------------
    Host : 45.60.112.163 ()
    State : up
    ----------
    Protocol : tcp
    port : 25       state : open
    port : 51       state : open
    port : 53       state : open
    port : 80       state : open
    port : 81       state : open
    port : 85       state : open
    port : 91       state : open
    port : 25       state : open
    port : 51       state : open
    port : 53       state : open
    port : 80       state : open
    port : 81       state : open
    port : 85       state : open
    port : 91       state : open
    port : 25       state : open
    port : 51       state : open
    port : 53       state : open
    port : 80       state : open
    port : 81       state : open
    port : 85       state : open
    port : 91       state : open
    port : 25       state : open
    port : 51       state : open
    port : 53       state : open
    port : 80       state : open
    port : 81       state : open
    port : 85       state : open
    port : 91       state : open
    port : 25       state : open
    port : 51       state : open
    port : 53       state : open
    port : 80       state : open
    port : 81       state : open
    port : 85       state : open
    port : 91       state : open
    port : 25       state : open
    port : 51       state : open
    port : 53       state : open
    port : 80       state : open
    port : 81       state : open
    port : 85       state : open
    port : 91       state : open
    port : 25       state : open
    port : 51       state : open
    port : 53       state : open
    port : 80       state : open
    port : 81       state : open
    port : 85       state : open
    port : 91       state : open
    Execution time: 0.015624761581420898 seconds

To convert output to csv, use:

print(nm.csv())

As a result of this investigation, Nmap was now installed on my computer. Just for fun, I also ran a scan on the command prompt using the below command. This scan ran for range '1-1000' and took more than 15 minutes(I didn't sit through the whole session!).