Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

join two files based on two columns

Tags:

bash

awk

Believe it or not, I've searched all over the internet and haven't found a working solution for this problem in AWK.

I have two files, A and B:

File A:

chr1   pos1   
chr1   pos2
chr2   pos1
chr2   pos2

File B:

chr1 pos1
chr2 pos1
chr3 pos2

Desired Output:

chr1 pos1
chr2 pos1

I'd like to join these two files to basically get the intersection between the two files based on the first AND second columns, not just the first. Since this is the case, most simple scripts won't work and join doesn't seem to be an option.

Any ideas?

EDIT: sorry, I didn't mention that there are more columns than just the two I showed. I've only shown two in my example because I'm only interested in the first two columns between both files being identical, the rest of the data aren't important (but are nonetheless in the file)

like image 270
Nick Avatar asked Sep 12 '11 18:09

Nick


2 Answers

The awk solution is:

awk 'FILENAME==ARGV[1] {pair[$1 " " $2]; next} ($1 " " $2 in pair)' fileB fileA

Place the smaller file first since you have to basically hold it in memory.

like image 94
glenn jackman Avatar answered Oct 05 '22 03:10

glenn jackman


I would write it like this:

awk 'NR == FNR {
  k[$1, $2]
  next
  }
($1, $2) in k
  ' filea fileb  

The order of the input files might need to be adapted based on the exact requirement.

like image 38
Dimitre Radoulov Avatar answered Oct 05 '22 04:10

Dimitre Radoulov