Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to set a global CURL rate(speed) limit?

Tags:

linux

curl

I had a look around for the CURL settings file but didn't find it in /etc/ and didn't find much on the curl site ether...

So basically what I want to do set a maximum speed limit that curl can upload at (no matter how many instances of it are running) so that my server has some upload capacity left for other tasks.

like image 737
Mint Avatar asked Apr 24 '10 23:04

Mint


1 Answers

Pretty sure you updated your question, as it certainly makes my original answer (TFM, below) obsolete.

I doubt what you are asking for is possible within curl as I would assume that each instance of curl operates independently of each other.

You could write a script that spawns curl instances and sets the limit of each individual job based on the number of total jobs, but this would not be dynamic. You could mimic a global rate by forcing all of your curl commands to operate through a certain port or network interface and then use QOS to throttle it.

However, you probably should just find a download utility that handles job queues and is capable of rate limiting on its own.

From TFM (man curl) --limit-rate Specify the maximum transfer rate you want curl to use. This feature is useful if you have a limited pipe and you'd like your transfer not to use your entire bandwidth.

         The  given speed is measured in bytes/second, unless a suffix is
         appended.  Appending 'k' or 'K' will count the number  as  kilo-
         bytes,  'm'  or M' makes it megabytes, while 'g' or 'G' makes it
         gigabytes. Examples: 200K, 3m and 1G.

         The given rate is the average speed counted  during  the  entire
         transfer. It means that curl might use higher transfer speeds in
         short bursts, but over time it uses no more than the given rate.

         If  you  also  use the -Y/--speed-limit option, that option will
         take precedence and might cripple the rate-limiting slightly, to
         help keeping the speed-limit logic working.

         If this option is used several times, the last one will be used.
like image 162
Indolering Avatar answered Sep 28 '22 03:09

Indolering