Merge data frames and overwrite values

Question

How do I merge 2 similar data frames but have one with greater importance?

For example:

Dataframe 1

Date      Col1    Col2
jan         2      1
feb         4      2
march       6      3
april       8      NA

Dataframe 2

Date      Col2    Col3
jan         9      10
feb         8      20
march       7      30
april       6      40

merge these by Date with dataframe 1 taking precedence but dataframe 2 filling blanks

DataframeMerge

Date      Col1    Col2    Col3
jan         2       1      10
feb         4       2      20
march       6       3      30
april       8       6      40

EDIT - SOLUTION

commonNames <- names(df1)[which(colnames(df1) %in% colnames(df2))]
commonNames <- commonNames[commonNames != "key"]
dfmerge<- merge(df1,df2,by="key",all=T)
for(i in commonNames){
  left <- paste(i, ".x", sep="")
  right <- paste(i, ".y", sep="")
  dfmerge[is.na(dfmerge[left]),left] <- dfmerge[is.na(dfmerge[left]),right]
  dfmerge[right]<- NULL
  colnames(dfmerge)[colnames(dfmerge) == left] <- i
}

Answer 1

merdat <- merge(dfrm1,dfrm2, by="Date")  # seems self-documenting

#  explanation for next line in text below.
merdat$Col2.y[ is.na(merdat$Col2.y) ] <- merdat$Col2.x[ is.na(merdat$Col2.y) ]

Then just rename 'merdat$Col2.y' to 'merdat$Col2' and drop 'merdat$Col2.x'.

In reply to request for more comments: One way to update only sections of a vector is to construct a logical vector for indexing and apply it using "[" to both sides of an assignment. Another way is to devise a logical vector that is only on the LHS of an assignment but then make a vector using rep() that has the same length as sum(logical.vector). The goal is both instances is to have the same length (and order) for assignment as the items being replaced.

Answer 2

Update using v1.9.6 of data.table's on= argument (which allows for adhoc joins:

setDT(df1)[df2, `:=`(Col2 = ifelse(is.na(Col2), i.Col2, Col2), 
                     Col3 = i.Col3), on="Date"][]

---

Here's a data.table solution. Make sure your df1 and df2's Date column is factor with desired levels (for ordering)

require(data.table)
dt1 <- data.table(df1, key="Date")
dt2 <- data.table(df2, key="Date")
# Col2 refers to the Col2 of dt1 and i.col2 refers to that of dt2
dt1[dt2, `:=`(Col3 = Col3, Col1 = Col1, 
        Col2 = ifelse(is.na(Col2), i.Col2, Col2))]

# the result is stored in dt1
> dt1
#     Date Col1 Col2 Col3
# 1:   jan    2    1   10
# 2:   feb    4    2   20
# 3: march    6    3   30
# 4: april    8    6   40

Answer 3

Here is a dplyr solution. Credit to @docendo discimus

df1 <- data.frame(y = c("A", "B", "C", "D"), x1 = c(1,2,NA, 4)) 

  y x1
1 A  1
2 B  2
3 C NA
4 D  4

df2 <- data.frame(y = c("A", "B", "C"), x1 = c(5, 6, 7))

  y x1
1 A  5
2 B  6
3 C  7

dplyr

left_join(df1, df2, by="y") %>% 
transmute(y, x1 = ifelse(is.na(x1.y), x1.x, x1.y))

  y x1
1 A  5
2 B  6
3 C  7

Answer 4

Consider this example:

> d1 <- data.frame(x=1:4, a=2:5, b=c(3,4,5,NA))
> d1
  x a  b
1 1 2  3
2 2 3  4
3 3 4  5
4 4 5 NA
> d2 <- data.frame(x=1:4, b=c(6,7,8,9), c=11:14)
> d2
  x b  c
1 1 6 11
2 2 7 12
3 3 8 13
4 4 9 14

Now use merge and within, with ifelse:

> within(merge(d1, d2, by="x"), {b <- ifelse(is.na(b.x),b.y,b.x); b.x <- NULL; b.y <- NULL})
  x a  c b
1 1 2 11 3
2 2 3 12 4
3 3 4 13 5
4 4 5 14 9