SQL Linked server query very very slow

Question

I am extracting large amount of data via linked server from VIEWS. I am using SQL Server 2012 and linked server is SQL Server 2008

My select statement is

SELECT * INTO MY_LOCAL_TABLE
FROM 
(    SELECT * FROM LINKEDSERVER.DB.TABLE.VIEW
     WHERE DATE>'2012-01-01' AND ID IN (SELECT ID FROM MY_LOCAL_VIEW) 
) Q

I am expecting 300K rows for nearly 700+ IDs. before it used to take couple of hours but now its take more than a 20 hr!!

Could you please suggest any alternative solution for this PAIN??

Very many thanks in advance!

Answer 1

When you use a 4-part name such as [server].db.dbo.table, especially in a join, often times the entire table is copied over the wire to the local machine, which is obviously not ideal.

A better approach is to use an OPENQUERY -- which is handled at the source (linked server).

Try:

SELECT * FROM OPENQUERY([LINKEDSERVER], 'SELECT * FROM DB.TABLE.VIEW WHERE DATE>'2012-01-01') AND ID IN (SELECT ID FROM MY_LOCAL_VIEW)  

With this approach the linked server will return all rows for date > x, and then the local server will filter that by ID's in your local table.

Of course, indexing will still play a factor for doing SELECT * FROM DB.TABLE.VIEW WHERE DATE>'2012-01-01.

Another approach, which I use on large subsets, is to dump the local ID's to the remote server, THEN handle it all remotely, such as:

    -- copy local table to linked server by executing remote query      DECLARE @SQL NVARCHAR(MAX)     SET @SQL = 'SELECT ID INTO db.dbo.tmpTable FROM [SERVER].DB.DBO.MY_LOCAL_VIEW'     EXEC(@SQL) AT [LINKEDSERVER]     -- index remote table?!?     DECLARE @SQL NVARCHAR(MAX)     SET @SQL = 'CREATE INDEX [IXTMP] ON db.dbo.tmpTable (ID)'     EXEC(@SQL) AT [LINKEDSERVER]      -- run query on local machine against both remote tables     SELECT *     -- INTO sometable     FROM OPENQUERY([LINKEDSERVER], 'SELECT *                                      FROM DB.TABLE.VIEW                                     WHERE DATE>''2012-01-01''                                     AND ID IN (SELECT ID FROM db.dbo.tmpTable)')      -- now drop remote temp table of id's     DECLARE @SQL NVARCHAR(MAX)     SET @SQL = 'DROP TABLE db.dbo.tmpTable'     EXEC(@SQL) AT [LINKEDSERVER] 

If the local view is also large, then you may consider executing a remote query that uses an openquery back to the local machine (assuming the remote machine has the local as a link).

-- copy local table to linked server by executing remote query  DECLARE @SQL NVARCHAR(MAX) SET @SQL = 'SELECT ID INTO db.dbo.tmpTable FROM OPENQUERY([SERVER], ''SELECT ID FROM DB.DBO.MY_LOCAL_VIEW'')' EXEC(@SQL) AT [LINKEDSERVER] 

Answer 2

Others have already suggested about indexing. So I am not going there. suggest another option, if you could change that inner query

 SELECT * FROM LINKEDSERVER.DB.TABLE.VIEW
 WHERE DATE>'2012-01-01' AND ID IN (SELECT ID FROM MY_LOCAL_VIEW)

To a joined query using inner join since you said having 700+ inlist elements. give it a try.

   SELECT lnv.* FROM LINKEDSERVER.DB.TABLE.VIEW lnv
   inner join MY_LOCAL_VIEW mcv
   on lnv.ID = mcv.ID
   and lnv.DATE > '2012-01-01'