Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Cannot create EXTERNAL TABLE on Azure SQL Databse

Can you not create an External table on an Azure SQL Database with a format file? I'm trying to create an external table to a table I dumped into blob storage.

From this page: https://msdn.microsoft.com/en-us/library/dn935021.aspx

-- Create a new external table  
CREATE EXTERNAL TABLE [ database_name . [ schema_name ] . | schema_name. ] table_name   
    ( <column_definition> [ ,...n ] )  
    WITH (   
        LOCATION = 'folder_or_filepath',  
        DATA_SOURCE = external_data_source_name,  
        FILE_FORMAT = external_file_format_name  
        [ , <reject_options> [ ,...n ] ]  
    )  
[;]  

Is the documentation incorrect or am I missing something? I can't seem to create a FORMAT FILE and keep receiving

"Incorrect syntax near 'EXTERNAL'." error.

CREATE EXTERNAL FILE FORMAT [DelimitedText] 
WITH (
    FORMAT_TYPE = DELIMITEDTEXT, 
    FORMAT_OPTIONS (
        FIELD_TERMINATOR = N'~¶~', 
        USE_TYPE_DEFAULT = False
    ), 
    DATA_COMPRESSION = N'org.apache.hadoop.io.compress.GzipCodec')
GO
like image 529
BamBamBeano Avatar asked Jan 20 '17 17:01

BamBamBeano


People also ask

How do I create an external table in Azure?

You can create external tables in Synapse SQL pools via the following steps: CREATE EXTERNAL DATA SOURCE to reference an external Azure storage and specify the credential that should be used to access the storage. CREATE EXTERNAL FILE FORMAT to describe format of CSV or Parquet files.

What is external table in PolyBase?

Use an external table with an external data source for PolyBase queries. External data sources are used to establish connectivity and support these primary use cases: Data virtualization and data load using PolyBase. Bulk load operations using SQL Server or SQL Database using BULK INSERT or OPENROWSET.


1 Answers

The problem is (probably) that you are trying to use PolyBase on an Azure SQL Database, but PolyBase is only supported on on SQL Service 2016 on-premise. It is, however, supported on Azure SQL Datawarehouse: PolyBase Versioned Feature Summary

If you instead of an Azure SQL Database create an Azure SQL Datawarehouse you should have the PolyBase features available, including creating an external file format.

SQL data warehouse in Azure portal

Running this:

CREATE EXTERNAL FILE FORMAT TextFormat 
WITH (
    FORMAT_TYPE = DELIMITEDTEXT, 
    FORMAT_OPTIONS (
        FIELD_TERMINATOR = N'~¶~', 
        USE_TYPE_DEFAULT = False
    ), 
    DATA_COMPRESSION = N'org.apache.hadoop.io.compress.GzipCodec')
GO  

On an Azure SQL Database will give you an error like:

Msg 102, Level 15, State 1, Line 1
Incorrect syntax near 'EXTERNAL'.

Running the same thing on Azure SQL Data warehouse will work

Command(s) completed successfully.

You will not be able to work with Hadoop databases using Azure SQL data warehouse, but working with Azure blob Storage is supported.

like image 130
yoape Avatar answered Sep 20 '22 11:09

yoape