Shell script - loop in batches

Question

I have a sample script like below, where I pass the server names as comma separated stings and execute the functions for each server. Now I need to execute this in batches, e.g. in first batch functions will call with a,b,c,d. Once finished they will take next 4 and call the functions and at last they will call last 2. I can't change the 'servers' variable. Any ides/suggestions.

servers = a,b,c,d,e,f,g,h,i,j
OIFS=$IFS;
IFS=",";
for server in ${servers}
do
 function1 server
done
for server in ${servers}
do
 function2 server
done
for server in ${servers}
do
 function3 server
done
IFS=$OIFS;

Please note that server variable length is not fixed and based on the environment it loads different servers.

Answer 1

Read the server list into an array, then use substring expansion (aka "slicing"). You can also store the names of the functions to call in an array, if I (now) understand your concern over the size of the list of servers.

IFS=, read -a s <<< "$servers"

bs=4  # batch size
for ((i=0; i<=${#s[@]}; i+=bs)); do
    function1 "${s[@]:i:bs}"
    function2 "${s[@]:i:bs}"
    function3 "${s[@]:i:bs}"
done

Answer 2

Looks like a job for GNU Parallel to me. Something like this:

parallel -d ',' -k -j 4 'func1 {1}; func2 {1}; func3 {1}'  <<< $servers

Try it out like this:

parallel -d ',' -k -j 4 'echo {1}; sleep 2'  <<< $servers
a
b
c
d
e
f
g
h
i
j

If they really are functions (rather than shell scripts or executables), remember to export them like this:

func1() {
  echo Doing it for $1
  sleep 2
  echo Done with $1
}

export -f func1