Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Split big file in multiple files based on column

Tags:

awk

I have a file with a semicolon as delimiter and headers. I would like to split that file based on the date column. The file has dates in ascending order.

The name of the output file should be as follows: 01_XX_YYMMDD_YYMMDD.txt

e.g. 01_XX_210920_210920.txt

Example

Here's an example file:

--INPUT 
K;V1.00;;;;;;
P;01.01.2021 00:01;16;EXA;31;TESTA;95.9;XXXX
P;01.01.2021 00:02;33;EXA;31;TESTA;95.9;XYXY
P;02.01.2021 00:54;16;EXB;33;TESTB;94.0;DWAD
P;02.01.2021 00:56;11;EXB;33;TESTB;94.0;DADA
P;03.01.2021 01:00;16;EXC;32;TESTC;94.6;WEWEQ
P;03.01.2021 01:22;16;EXC;32;TESTC;94.6;QEQR
P;04.01.2021 02:39;16;EXD;33;TESTD;94.3;DFAG

The output should be as follows, while taking the previous file as example

--OUTPUT FILES
FILE1: 01_XX_210101_210101.txt
P;01.01.2021 00:01;16;EXA;31;TESTA;95.9;XXXX
P;01.01.2021 00:02;33;EXA;31;TESTA;95.9;XYXY

FILE2: 01_XX_210102_210102.txt
P;02.01.2021 00:54;16;EXB;33;TESTB;94.0;DWAD
P;02.01.2021 00:56;11;EXB;33;TESTB;94.0;DADA

FILE3: 01_XX_210103_210103.txt
P;03.01.2021 01:00;16;EXC;32;TESTC;94.6;WEWEQ
P;03.01.2021 01:22;16;EXC;32;TESTC;94.6;QEQR

FILE4: 01_XX_210104_210104.txt
P;04.01.2021 02:39;16;EXD;33;TESTD;94.3;DFAG

I tried AWK but no success because of the timestamp my file has… Thank you!

x

UPDATE: Solution

    awk -F';' '
NR > 1 {
   dt = substr($2,9,2) substr($2,4,2) substr($2,1,2)
   print > ("01_LPR_" dt "_" dt ".txt")
}' input
like image 577
Djabone Avatar asked Dec 18 '25 16:12

Djabone


1 Answers

You may try this awk:

awk -F';' '
NR > 1 {
   dt = substr($2,9,2) substr($2,4,2) substr($2,1,2)
   print > ("01_XX_" dt "_" dt ".txt")
}' input

For the updated requirements in comments below:

awk -F';' '
NR == 1 {
   hdr = $0
   next
}
{
   dt = substr($2,9,2) substr($2,4,2) substr($2,1,2)
}
dt != pdt {
   if (pdt) {
      print "END" > fn
      close(fn)
   }
   fn = "01_XX_" dt "_" dt ".txt"
   print hdr > fn
}
{
   print > fn
   pdt = dt
}
END {
   print "END" > fn
   close(fn)
}' input
like image 157
anubhava Avatar answered Dec 24 '25 11:12

anubhava



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!