How to install a jar in databricks using ADF

Question

We are able to install the jar file using the UI method to a particular cluster. But our requirement to install it on all the ondemand clusters in the workspace. We are using the below shell script to download the jar file to DBFS. Not sure how we can refer/install this jar in all cluster using a global init script

curl https://repo1.maven.org/maven2/com/databricks/spark-xml_2.12/0.12.0/spark-xml_2.12-0.12.0.jar >/dbfs/FileStore/jars/maven/com/databricks/spark_xml_2_12_0_12_0.jar

Any help would be really appreciated!!

Answer 1

There is an alternate solution for adding jar library to the job cluster which is called from Azure data factory while running our job.

In ADF, while calling the notebook we have the option to include the jar directory in DBFS or we can able to give the Maven coordinates.

ADF SETTINGS