Errors with Locust load testing scripts - Connection aborted.', RemoteDisconnected

Question

I am new to Locust Load testing framework and in process of migrating my existing Azure cloud based Performance testing C# scripts to Locust's Python based scripts. Our team almost completed migration of scripts. But during our load tests, we are getting errors as below, which fails to create new requests from the machine due to high CPU utilization or because of so many exception on Locust. We are running with Locust web based mode - details are indicated below. These scritps are working fine on smaller loads of 50 to 100 users

"Error 1 -('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))"

"Error 2 : Connection pool is full, discarding connection"

"** **Error 3 :urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 110] Connection timed out****"

Yes, we are using UrlLibs on the utility classes . But first 2 error does seems to be of Locust.

Our Load testing configurations are : "3500 users at a hatch rate of 5 users per second". Running natively(no docker container) on a 8 Core , 16 Gb Linux Ubuntu Virtual machine on Azure. ulimit set as 50,000 on Linux machine.

Please help us with your thoughts

Sample test is as below

import os 
import sys
sys.path.append(os.environ.get('WORKDIR', os.getcwd()))

from locust import HttpLocust, TaskSet, task
from locust.wait_time import between

class ContactUsBehavior(TaskSet):


    wait_time = AppUtil.get_wait_time_function(2)


    @task(1)
    def post_load_test_contact(self):
        data = { "ContactName" : "Mane"
            , "Email" : "[email protected]"
            , "EmailVerifaction" : "[email protected]"
            , "TelephoneContact" : ""
            , "PhoneNumber" : ""
            , "ContactReason" : "Other"            
            , "OtherComment" : "TEST Comments 2019-12-30"
            , "Agree" : "true"
             }
        self.client.post("app/contactform", self.client, 'Contact us submission', post_data = data)


class UnauthenticatedUser(HttpLocust):
    task_set = ContactUsBehavior
    # host is override-able
    host = 'https://app.devurl.com/'

Answer 1

Locust’s default HTTP client uses python-requests which internally use urllib3 . If you are working on a large scale tests, you should consider another HTTP client. The connection pool of urllib 3 (PoolManager ) will reuse connections and limit how many connections are allowed per host at any given time to avoid accumulating too many unused sockets. So you have option to tweak the pool : https://urllib3.readthedocs.io/en/latest/advanced-usage.html#customizing-pool-behavior

Or you can try any other high performance HTTP client . Eg: gevenhttp Locust also provides a built-int client which is faster than the default python-requests: https://docs.locust.io/en/stable/increase-performance.html

You should consider to run Locust in cluster mode in different nodes if the client still couldn't handle the big load.