How do I select just a portion of huge binary (file)?

Question

My problem is this: I have the potential for huge files being stored in a binary (image) field on SQL Server 2008 (> 1GB).

If I return the entire binary using a regular select statement, the query takes more than a minute to return results to my .NET program and my client apps time out. What I'm looking for is TSQL code that will limit the size of the data returned (maybe 300mb), allowing me to iterate through the remaining chunks and prevent timeouts.

This has to happen in the SQL query, not in the processing after the data is returned.

I've tried SubString, which MS says works with binary data, but all I get back is 8000 bytes maximum. The last thing I tried looked like this:

select substring(Package,0,300000000) 'package', ID from rplPackage where ID=0
--where package is the huge binary stored in a image field

Data Streaming isn't really an option either, because of the client apps.

Any ideas?

Answer 1

OK, I figured it out. The way to do this is with the substring function, which MS accurately says works with binaries. What they don't say is that substring will return only 8,000 bytes, which is what threw me.

In other words, if the blob data type is image and you use this:

 select substring(BlobField,0,100000000) 
 from TableWithHugeBlobField
 where ID = SomeIDValue

 --all you'll get is the first 8K bytes (use DataLength function to get the size)

However, if you declare a variable of varbinary(max) and the blob field data type is varbinary(max) - or some size that's useful to you - then use the substring function to bring back the partial binary into the variable you declared. This works just fine. Just like this:

 Declare @PartialImage varbinary(max) 
 select @PartialImage = substring(BlobField, 0, 100000000) --1GB
 from TableWithHugeBlobField
 where ID = SomeIDValue

 select DataLength(@PartialImage) -- should = 1GB

The question was posed earlier, why use SQL to store file data? It's a valid question; imagine you're having to replicate data as files to hundreds of different client devices (like iPhones), each package unique from the other because different clients have different needs, then storing the file packages as blobs on a database is a lot easier to program against than it would be to programmatically dig through folders to find the right package to stream out to the client.