sed: search and replace values at position for last line of file

Question

I have a fixed width file where the last line of the file needs to be checked at a certain position, and if value is found at that position, it is changed to another value specified by an ARRAY. The input file looks like this for the last line

210204154828000999990000056000000072328100000540000000

The values of interest are at position 27, 40 and 47. The values at those positions are 6, 1, 4 respectively. I'm using an ARRAY for checking these values. If the position of the ARRAY matches the value of the position, replace with the value of the ARRAY location. The code looks like this

#!/bin/bash

chars=( { A B C D E F G H I )

for num in "${!chars[@]}" 
do  
    sed -i -E '$ s%^(.{26})$num%\1${chars[$num]}%' $1
done

I don't get a failure when running this but I don't see it updating the last line either. The expected output if this were working correctly would be the following:

21020415482800099999000005F000000072328A000005D0000000

The command to execute is as follows

# ./convert.sh file1.txt

I'm sure something is wrong with my sed but I just can't figure it out. Any help would be appreciated.

Answer 1

As pointed out in xhienne's answer, your command did not work since $num and ${chars[$num]} were written inside single quotes '...' where bash does no expansion of variables. Use double quotes "..." instead.

In this answer, I just want to point out a trick to make the script more efficient. Right now you do sed -i ten times. That is, the file has to be read over and over again. You can do everything in a single sed command whose command string is dynamically generated:

#!/bin/bash

chars=( '{' {A..I} )
cmd=()
for num in "${!chars[@]}"; do
  for pos in 27 40 47; do
    cmd+=(-e "\$ s%^(.{$((pos-1))})$num%\\1${chars[num]}%")
  done
done
sed -Ei "${cmd[@]}" "$1"

Like your original script, this only works if chars is free of sed control characters like \, %, and &. If you need one of these chars escape them manually as in chars=('\\' '\%' '\&') or automatically with another sed command.

Answer 2

You don't want to have to process the file 10 times for each index.

Here's a single call to awk

tac file | awk '
    BEGIN {
        split("A B C D E F G H I", chars); chars[0] = "{"
        split("27 40 47", idx)
    }
    FNR == 1 {
        line = $0
        for (i in idx) {
            char = substr(line, idx[i], 1)
            repl = chars[char]
            line = substr(line, 1, idx[i]-1) repl substr(line, idx[i]+1)
        }
        $0 = line
    }
    1
' | tac

That reverses the file, operates on the first line, then re-reverses the file

---

To save inplace, use sponge from the moreutils package:

tac file | awk '...' | tac | sponge file

or use a temp file

tmp=$(mktemp)
tac file | awk '...' | tac > "$tmp" && mv "$tmp" file

---

The equivalent perl is unsurprisingly more concise

tac file | perl -pe '
    BEGIN { @c = qw/ { A B C D E F G H I /; }
    $. == 1 or next;
    s/{^(.{26})(\d)(.{12})(\d)(.{6})(\d)/$1$c[$2]$3$c[$4]$5$c[$6]/;
' | ...

Somewhat more readable:

tac file | perl -pe '
    BEGIN { @c = qw/ { A B C D E F G H I /; }
    $. == 1 or next;
    s{^(.{26}) (\d)    (.{12}) (\d)   (.{6}) (\d)   }
     {   $1  . $c[$2]  . $3  . $c[$4] . $5 . $c[$6] }xe;
' | ...