Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Azure Data Factory and SharePoint

I have some Excel files stored in SharePoint online. I want copy files stored in SharePoint folders to Azure Blob storage.

To achieve this, I am creating a new pipeline in Azure Data factory using Azure Portal. What are possible ways to copy files from SharePoint to Azure blob store using Azure Data Factory pipelines?

I have looked at all linked services types in Azure data factory pipeline but couldn't find any suitable type to connect to SharePoint.

like image 208
Jamal Mustafa Avatar asked Feb 25 '19 15:02

Jamal Mustafa


People also ask

Can we connect SharePoint to Azure Data Factory?

Use the following steps to create a linked service to a SharePoint Online List in the Azure portal UI. Browse to the Manage tab in your Azure Data Factory or Synapse workspace and select Linked Services, then click New: Azure Data Factory.

Can Azure Connect to SharePoint?

For connections to an on-premises SharePoint server, you need to install and set up the on-premises data gateway on a local computer and a data gateway resource that's already created in Azure. You can then select the gateway resource to use when you create the SharePoint Server connection from your workflow.

What can Azure data/factory connect to?

Multiple Azure Data Factories can connect to a single Microsoft Purview to push lineage information. The current limit allows you to connect up to 10 Data Factory accounts at a time from the Microsoft Purview management center.


2 Answers

Rather than directly accessing the file in SharePoint from Data Factory, you might have to use an intermediate technology and have Data Factory call that. You have a few of options:

  1. Use a Logic App to move the file
  2. Use an Azure Function
  3. Use a custom activity and write your own C# to copy the file.

To call a Logic App from ADF, you use a web activity. You can directly call an Azure Function now.

like image 66
mmarie Avatar answered Sep 27 '22 02:09

mmarie


We can create a linked service of type 'File system' by providing the directory URL as 'Host' value. To authenticate the user, provide username and password/AKV details. Note: Use Self-hosted IR

like image 24
Srinivas Avatar answered Sep 23 '22 02:09

Srinivas