Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Make sure the ACL and firewall rule is correctly configured in the Azure Data Lake Store account

I'm coping CSV files from Azure blob to Azure Data Lake using Azure data factory using Copy data tool. I'm following this link: https://learn.microsoft.com/en-us/azure/data-factory/quickstart-create-data-factory-copy-data-tool

Fron Copy data tool my source configuration and test connection successed. However, the destination connection (that is Data lake) is creating problem.

I'm getting error : Make sure the ACL and firewall rule is correctly configured in the Azure Data Lake Store account.

I followed this link for Fairwall setting: https://learn.microsoft.com/en-us/azure/data-lake-store/data-lake-store-secure-data (Set IP address range for data access)

Enabled fairwall and Allow access to Azure service "ON"

Still, I'm getting same error. Could any one please suggest. How to fix this?

like image 725
AskMe Avatar asked Jun 12 '18 08:06

AskMe


People also ask

What does ACL stand for in Azure Data lake?

Each file and directory in your storage account has an access control list. When a security principal attempts an operation on a file or directory, An ACL check determines whether that security principal (user, group, service principal, or managed identity) has the correct permission level to perform the operation.

How do I turn on my Azure storage firewall?

Sign in to the Azure portal to get started. Locate your storage account and display the account overview. Select Networking to display the configuration page for networking. Under Firewalls and virtual networks, for Selected networks, select to allow access.

What is firewall rules in Azure?

Azure Firewall allows any port in the 1-65535 range in network and application rules, however NAT rules only support ports in the 1-63999 range. This is a current limitation. An Azure Firewall configuration update can take three to five minutes on average, and parallel updates aren't supported.


2 Answers

Get your Managed Identity Application ID from Azure Data Factory properties.

Go to Azure Data Lake Storage and navigate to Data Explorer -> Access -> Add and then provide the ID in the 'Select User or group' field.
It will identify your Azure Data Factory instance/resource and then provide ACLs(R/W/X) as per your requirement.

like image 109
Siva Muthi Avatar answered Nov 02 '22 11:11

Siva Muthi


Except the firewall setting, please also be sure that your account has necessary permission on the target ADLS account. Please refer to this doc for more details: https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-data-lake-store#linked-service-properties

like image 21
Yingqin Avatar answered Nov 02 '22 12:11

Yingqin