EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to HdfsBridge_IsDirExist. Java exception message:

Question

I am trying to create external table through polybase with below syntax on Visual Studio 2015. Its giving me the below error. Can some one pls help on this

CREATE EXTERNAL TABLE dbo.DimDate2External (
    DateId INT NOT NULL,
    CalendarQuarter TINYINT NOT NULL,
    FiscalQuarter TINYINT NOT NULL
)
WITH (
    LOCATION='/textfiles/DimDate2.txt',
    DATA_SOURCE=AzureStorage,
    FILE_FORMAT=TextFile
);

CREATE EXTERNAL DATA SOURCE AzureStorage
WITH ( 
    TYPE = HADOOP, 
    LOCATION = 'wasbs://<blob_container_name>@<azure_storage_account_name>.‌​blob.core.windows.ne‌​t', 
    CREDENTIAL = AzureStorageCredential
    ); 

CREATE EXTERNAL FILE FORMAT TextFile WITH ( FORMAT_TYPE = DelimitedText, FORMAT_OPTIONS (FIELD_TERMINATOR = ',') );

EXTERNAL TABLE access failed due to internal error:

'Java exception raised on call to HdfsBridge_IsDirExist. Java exception message: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.: Error [com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.] occurred while accessing external file.'

Answer 1

In the 'LOCATION' syntax I mistakenly misplaced the Blob container and Storage account and got this error. Now its fixed.

CREATE EXTERNAL DATA SOURCE AzureStorage WITH ( TYPE = HADOOP, LOCATION = 'wasbs://@.‌​blob.core.windows.ne‌​t', CREDENTIAL = AzureStorageCredential )

Answer 2

I can reproduce this error if the Azure Storage account element of your external data source is incorrect (XXX in my example):

CREATE EXTERNAL DATA SOURCE eds_dummy
WITH (  
    TYPE = Hadoop,
    LOCATION = 'wasbs://[email protected]',
    CREDENTIAL = sc_tpch
);

If the blob container name is incorrect (dummy in my example) but the storage account is correct, you get a very specific error message when trying to create the table:

Msg 105002, Level 16, State 1, Line 27 EXTERNAL TABLE access failed because the specified path name '/test.txt' does not exist. Enter a valid path and try again.

There appears to be some kind of validation on the blob container. However if the Azure Storage Account name is incorrect, you do not get an error when you create the external data source, only when you try and create the table:

Msg 105019, Level 16, State 1, Line 35 EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to HdfsBridge_IsDirExist. Java exception message: com.microsoft.azure.storage.StorageException: The server encountered an unknown failure: : Error [com.microsoft.azure.storage.StorageException: The server encountered an unknown failure: ] occurred while accessing external file.'

To correct, please make sure the Azure Storage Account and Blob container exist.

The easiest way to do this is copy the URL of your file or folder from the portal and fix it up for external tables, ie from this:

https://yourStorageAccountName.blob.core.windows.net/yourBlobContainerName

to this:

wasbs://yourBlobContainerName@yourStorageAccountName.blob.core.windows.net

Good luck.