Databricks Delta Table Merge statement using R

Question

I have recently started working on Databricks and I have been trying to find a way to perform a merge statement on a Delta table, though using an R api (preferably sparklyr). The ultimate purpose is to somehow impose a 'duplicate' constraint as described here. The aforementioned documentation describes the Python workflow :

deltaTable.alias("logs").merge(
    newDedupedLogs.alias("newDedupedLogs"),
    "logs.uniqueId = newDedupedLogs.uniqueId") \
  .whenNotMatchedInsertAll() \
  .execute()

however, I was wondering whether there is a straight-forward way to achieve this through R. Any assistance/ideas on the matter will be much appreciated since I am a new user (as mentioned above). Thanks in advance!

Answer 1

Providing this answer since the you commented that there is no R Delta Lake API support. There is now a new R package that provides an R API for Delta Lake: dlt. The syntax is very similar to that of the Python API for Delta Lake.

In the case of your example:

# Install and laod the `dlt` package
remotes::install_gitlab("zero323/dlt")
library(dlt)
...

# Use the Delta Lake R API from the dlt package
deltaTable <- dlt_for_path("<path to table>")

deltaTable %>%
  dlt_alias("logs") %>%
  dlt_merge(alias(newDedupedLogs, "newDedupedLogs"), expr("newDedupedLogs.uniqueId = logs.uniqueId")) %>%
  dlt_when_not_matched_insert_all() %>%
  dlt_execute()