Unexpectedly low throughput for Network I/O using Scotty

Question

I tried to benchmark Scotty to test the Network I/O efficiency and overall throughput.

For this I set up two local servers written in Haskell. One which doesn't do anything and just acts as an API.

Code for the same is

{-# LANGUAGE OverloadedStrings #-}


import Web.Scotty

import Network.Wai.Middleware.RequestLogger 

import Control.Monad
import Data.Text
import Control.Monad.Trans
import Data.ByteString
import Network.HTTP.Types (status302)
import Data.Time.Clock
import Data.Text.Lazy.Encoding (decodeUtf8)
import Control.Concurrent
import Network.HTTP.Conduit
import Network.Connection (TLSSettings (..))
import Network.HTTP.Client
import Network
main = do 
  scotty 4001 $ do
    middleware logStdoutDev
    get "/dummy_api" $ do
        text $ "dummy response"

I wrote another server which calls this server and returns the response.

{-# LANGUAGE OverloadedStrings #-}


import Web.Scotty

import Network.Wai.Middleware.RequestLogger 

import Control.Monad
import Control.Monad.Trans
import qualified Data.Text.Internal.Lazy as LT
import Data.ByteString
import Network.HTTP.Types (status302)
import Data.Time.Clock
import Data.Text.Lazy.Encoding (decodeUtf8)
import Control.Concurrent
import qualified Data.ByteString.Lazy as LB
import Network.HTTP.Conduit
import Network.Connection (TLSSettings (..))
import Network.HTTP.Client
import Network


main = do 
  let man = newManager defaultManagerSettings 
  scotty 3000 $ do
    middleware logStdoutDev

    get "/filters" $ do
        response <- liftIO $! (testGet man)
        json $ decodeUtf8 (LB.fromChunks response)

testGet :: IO Manager -> IO [B.ByteString]
testGet manager = do
    request <- parseUrl "http://localhost:4001/dummy_api"
    man <- manager
    let req = request { method = "GET", responseTimeout = Nothing, redirectCount = 0}
    a <- withResponse req man $ brConsume . responseBody
    return $! a

With both these servers running, I performed wrk benchmarking and got extremely high throughput.

wrk -t30 -c100 -d60s "http://localhost:3000/filters"
Running 1m test @ http://localhost:3000/filters
  30 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    30.86ms   78.40ms   1.14s    95.63%
    Req/Sec   174.05     62.29     1.18k    76.20%
  287047 requests in 1.00m, 91.61MB read
  Socket errors: connect 0, read 0, write 0, timeout 118
  Non-2xx or 3xx responses: 284752
Requests/sec:   4776.57
Transfer/sec:      1.52MB

While this was significantly higher than other web servers like Phoenix, I realized this meant nothing as majority of responses were 500 errors occuring due to file descriptor exhaustion.

I check the limits which were pretty low.

ulimit -n
256

I increased these limits to

ulimit -n 10240

I ran wrk again and this time clearly enough throughput had been reduced drastically.

wrk -t30 -c100 -d60s "http://localhost:3000/filters"
Running 1m test @ http://localhost:3000/filters
  30 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   105.69ms  161.72ms   1.24s    96.27%
    Req/Sec    19.88     16.62   120.00     58.12%
  8207 requests in 1.00m, 1.42MB read
  Socket errors: connect 0, read 0, write 0, timeout 1961
  Non-2xx or 3xx responses: 1521
Requests/sec:    136.60
Transfer/sec:     24.24KB

Although the amount of 500 errors had reduced, they were not eliminated. I benchmarked Gin and Phoenix and they were way better than Scotty while not giving any 500 responses.

What piece of puzzle I am missing? I suspect there is an issue I'm failing to debug.

I understand that http-conduit has a lot to do with these errors and http-client library uses it under the hood and this has nothing to do with Scotty.

Answer 1

@Yuras's analogy was correct. On running the server again, all the issues related to non 2xx status code were gone.

The first line in the main block was the culprit. I changed the line from

main = do 
  let man = newManager defaultManagerSettings

to

main = do 
  man <- newManager defaultManagerSettings

and voila, there weren't any issues. Also the high memory usage of the program stabilized to 21MB from 1GB earlier.

I don't know the reason though. It would be nice to have an explanation for this.