Sed command to parse xml from log

Question

I have a log file that has embedded xml in it and I am trying to parse it out using sed. What is happening is that I am getting the desired xml BUT the line after the desired xml is being picked up. Here is a sample file

2015-05-06 04:07:37.386 [INFO]Process:102 - Application submitted Successfully ==== 1
<APPLICATION> <NAME> test </NAME> </APPLICATION>
2015-05-06 04:07:39.386 [INFO] Process:103 - Application completed Successfully ==== 1

The sed command I am using is

sed -n '/<APPLICATION>/,/<\/APPLICATION>/p' batchlog.txt  >> np.out

As mentioned above, I am getting the desired XML but also getting the line after it. How do I avoid this?

The second part to this question is, if I were to use this in a shell script, what is the most efficient way to process each xml chunk (there can be many instances of "APPLICATION" blocks in the file. The idea is to replace some values within the xml tags contained in each block and re-write the file, maintaining the original content BUT with the new values within the tags. For this part, an example is as follows. Within a shell script, I am parsing the log file and as I come across the APPLICATION tags, I want to mask the values of any tag that has an SSN in it with *. For example:

<APPLICATION><FirstName>Test<FirstName><StudentSSN>123456789</StudentSSN><Address>123 Test Street</Address><ParentSSN>123456780</ParentSSN></APPLICATION>

Now, when the script runs against the log file, it needs to look within any *SSN tag and replace the values with *. Doing the following sed command on the command line I am able to grab the studentSSN tag.

sed -n 's:.*<\StudentSSN>\(.*\)</StudentSSN.*:\1:p'

But I am hoping to make it generic so that both parentSSN and studentSSN are picked up, replaced and written back to the file with the old non xml lines and within the XML, these new values. So the modified file would look something like this:

2015-05-06 04:07:37.386 [INFO]Process:102 - Application submitted Successfully ==== 1
<APPLICATION><FirstName>Test<FirstName><StudentSSN>*********</StudentSSN><Address>123 Test Street</Address><ParentSSN>*********</ParentSSN></APPLICATION>
2015-05-06 04:07:39.386 [INFO] Process:103 - Application completed Successfully ==== 1

Answer 1

The following will fix your sed so it will output just the desired line:

sed -n '/<APPLICATION>.*<\/APPLICATION>/p' batchlog.txt  >> np.out

It prints:

<APPLICATION> <NAME> test </NAME> </APPLICATION>

With the given input

Answer 2

This might work for you (GNU sed):

sed -r '/<(APPLICATION>).*<\/\1/!b;:a;s/(<((Student|Parent)SSN>)\**)[^*](.*<\/\2)/\1*\4/;ta' file

This restricts any sed processing to those lines that contain both a start and end APPLICATION tag.

Each character within Student or Parent SSN tags which is not a * is replaced by an *. This is accomplished by checking for a successful substitution using the ta command and looping back to the :a placeholder until no more substitutions occur.

N.B. The regexp uses coupious back references and even back references within back references (hence the need for the -r switch to make the final solution more visibly tolerable).