Counter increment in Bash loop not working

Question

First, you are not increasing the counter. Changing COUNTER=$((COUNTER)) into COUNTER=$((COUNTER + 1)) or COUNTER=$[COUNTER + 1] will increase it.

Second, it's trickier to back-propagate subshell variables to the callee as you surmise. Variables in a subshell are not available outside the subshell. These are variables local to the child process.

One way to solve it is using a temp file for storing the intermediate value:

TEMPFILE=/tmp/$$.tmp
echo 0 > $TEMPFILE

# Loop goes here
  # Fetch the value and increase it
  COUNTER=$[$(cat $TEMPFILE) + 1]

  # Store the new value
  echo $COUNTER > $TEMPFILE

# Loop done, script done, delete the file
unlink $TEMPFILE

COUNTER=1
while [ Your != "done" ]
do
     echo " $COUNTER "
     COUNTER=$[$COUNTER +1]
done

TESTED BASH: Centos, SuSE, RH

COUNTER=$((COUNTER+1)) 

is quite a clumsy construct in modern programming.

(( COUNTER++ ))

looks more "modern". You can also use

let COUNTER++

if you think that improves readability. Sometimes, Bash gives too many ways of doing things - Perl philosophy I suppose - when perhaps the Python "there is only one right way to do it" might be more appropriate. That's a debatable statement if ever there was one! Anyway, I would suggest the aim (in this case) is not just to increment a variable but (general rule) to also write code that someone else can understand and support. Conformity goes a long way to achieving that.

HTH

Try to use

COUNTER=$((COUNTER+1))

instead of

COUNTER=$((COUNTER))

I think this single awk call is equivalent to your grep|grep|awk|awk pipeline: please test it. Your last awk command appears to change nothing at all.

The problem with COUNTER is that the while loop is running in a subshell, so any changes to the variable vanish when the subshell exits. You need to access the value of COUNTER in that same subshell. Or take @DennisWilliamson's advice, use a process substitution, and avoid the subshell altogether.

awk '
  /GET \/log_/ && /upstream timed out/ {
    split($0, a, ", ")
    split(a[2] FS a[4] FS $0, b)
    print "http://example.com" b[5] "&ip=" b[2] "&date=" b[7] "&time=" b[8] "&end=1"
  }
' | {
    while read WFY_URL
    do
        echo $WFY_URL #Some more action
        (( COUNTER++ ))
    done
    echo $COUNTER
}

count=0   
base=1
(( count += base ))

Instead of using a temporary file, you can avoid creating a subshell around the while loop by using process substitution.

while ...
do
   ...
done < <(grep ...)

By the way, you should be able to transform all that grep, grep, awk, awk, awk into a single awk.

Starting with Bash 4.2, there is a lastpipe option that

runs the last command of a pipeline in the current shell context. The lastpipe option has no effect if job control is enabled.

bash -c 'echo foo | while read -r s; do c=3; done; echo "$c"'

bash -c 'shopt -s lastpipe; echo foo | while read -r s; do c=3; done; echo "$c"'
3