Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Cartesian product of two files (as sets of lines) in GNU/Linux

Tags:

shell

How can I use shell one-liners and common GNU tools to concatenate lines in two files as in Cartesian product? What is the most succinct, beautiful and "linuxy" way?

For example, if I have two files:

$ cat file1
a
b
$ cat file2
c
d
e

The result should be

a, c
a, d
a, e
b, c
b, d
b, e
like image 683
P Shved Avatar asked Oct 25 '09 13:10

P Shved


People also ask

How do I get the common line from two files in Linux?

Use comm -12 file1 file2 to get common lines in both files. You may also needs your file to be sorted to comm to work as expected. Or using grep command you need to add -x option to match the whole line as a matching pattern. The F option is telling grep that match pattern as a string not a regex match.

What is the meaning of || in Linux?

The OR Operator (||) is much like an 'else' statement in programming. The above operator allow you to execute second command only if the execution of first command fails, i.e., the exit status of first command is '1'.

What is &2 in bash script?

and >&2 means send the output to STDERR, So it will print the message as an error on the console. You can understand more about shell redirecting from those references: https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Redirections.


1 Answers

Here's shell script to do it

while read a; do while read b; do echo "$a, $b"; done < file2; done < file1

Though that will be quite slow. I can't think of any precompiled logic to accomplish this. The next step for speed would be to do the above in awk/perl.

awk 'NR==FNR { a[$0]; next } { for (i in a) print i",", $0 }' file1 file2

Hmm, how about this hacky solution to use precompiled logic?

paste -d, <(sed -n "$(yes 'p;' | head -n $(wc -l < file2))" file1) \
          <(cat $(yes 'file2' | head -n $(wc -l < file1)))
like image 68
pixelbeat Avatar answered Sep 28 '22 07:09

pixelbeat