I had a look around for the CURL settings file but didn't find it in /etc/ and didn't find much on the curl site ether...
So basically what I want to do set a maximum speed limit that curl can upload at (no matter how many instances of it are running) so that my server has some upload capacity left for other tasks.
Pretty sure you updated your question, as it certainly makes my original answer (TFM, below) obsolete.
I doubt what you are asking for is possible within curl as I would assume that each instance of curl operates independently of each other.
You could write a script that spawns curl instances and sets the limit of each individual job based on the number of total jobs, but this would not be dynamic. You could mimic a global rate by forcing all of your curl commands to operate through a certain port or network interface and then use QOS to throttle it.
However, you probably should just find a download utility that handles job queues and is capable of rate limiting on its own.
From TFM (
man curl
) --limit-rate Specify the maximum transfer rate you want curl to use. This feature is useful if you have a limited pipe and you'd like your transfer not to use your entire bandwidth.The given speed is measured in bytes/second, unless a suffix is appended. Appending 'k' or 'K' will count the number as kilo- bytes, 'm' or M' makes it megabytes, while 'g' or 'G' makes it gigabytes. Examples: 200K, 3m and 1G. The given rate is the average speed counted during the entire transfer. It means that curl might use higher transfer speeds in short bursts, but over time it uses no more than the given rate. If you also use the -Y/--speed-limit option, that option will take precedence and might cripple the rate-limiting slightly, to help keeping the speed-limit logic working. If this option is used several times, the last one will be used.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With