I have the following code to return a list of containers using the WindowsAzure.Storage
nuget package:
public static class AzureBlobStorageClient
{
public static CloudBlobClient GetClient(string AccountName = "foo", string AccountKey = "bar" )
{
try
{
var connectionString = $"DefaultEndpointsProtocol=https;AccountName={AccountName};AccountKey={AccountKey};EndpointSuffix=core.windows.net";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
IRetryPolicy exponentialRetryPolicy = new ExponentialRetry(TimeSpan.FromSeconds(2), 10);
blobClient.DefaultRequestOptions.RetryPolicy = exponentialRetryPolicy;
return blobClient;
}
catch (StorageException ex)
{
Console.WriteLine("Error returned from the service: {0}", ex.Message);
throw;
}
}
public static void DeleteContainer(CloudBlobContainer container)
{
var result = container.DeleteIfExistsAsync().Result;
}
public static List<CloudBlobContainer> GetContainers()
{
var client = GetClient();
BlobContinuationToken continuationToken = null;
List<CloudBlobContainer> results = new List<CloudBlobContainer>();
do
{
var response = client.ListContainersSegmentedAsync(continuationToken).Result;
continuationToken = response.ContinuationToken;
results.AddRange(response.Results);
}
while (continuationToken != null);
return results;
}
}
when i run this, i get the following error on client.ListContainersSegmentedAsync(continuationToken).Result :
System.AggregateException: 'One or more errors occurred. (This request is not authorized to perform this operation.)'
and I can't see how to set the authorization for the request.
My question is how to get past this error message
To resolve this issue, you need to verify the access permissions for the ADF and user type: Note: Storage Blob Data Contributor : Use to grant read/write/delete permissions to Blob storage resources. ADF permissions: Kindly check the permissions on the Storage account. Check the user permission on the storage account.
Navigate to your storage account overview in the Azure portal. Under Data storage on the menu blade, select Blob containers. Select the containers for which you want to set the public access level. Use the Change access level button to display the public access settings.
Select binary as format and click on continue. Name your dataset, then select the linked service, then provide the path where we have to download the data, and then click on ok. Now go back to your main pipeline and click on Debug, it will copy the data from the Amazon S3 bucket to our Azure blob storage.
To copy the data from Azure to S3 you also need to have Azure CLI and AWS CLI installed first. While it is easy to use Rclone and it is a recommended option when you don't want to write any code and use Rclone commands to copy the data.
Thanks to @gaurav Mantri for this answer.
The issue was my client IP was not added to the firewall rules for the storage account.
To change this go to :
Storage accounts > {yourAccount} > Firewalls and Virtual networks
and add your IP address
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With