Exporting data from SQL Server Express to CSV (need quoting and escaping)

Question

I've spent 2 days trying to export a 75,000 row table containing a large text field of user input data from a SQL server installation. This data contains every plain ascii character, tabs, and newlines. I need to export CSV where every field is quoted, and quotes within the quoted columns are properly escaped ("").

Here is what I've tried so far: - Right clicking on the database from Management Studio and exporting to Excel: fails due to the field being too long. - Data Export from Management Studio to flat file with " text separator and comma separation - completely useless, does not escape quotes within a field making the file completely ambiguous. - BCP from command line - also does not support quoting fields.

I need to import with the FasterCSV ruby library. It does not allow the quote delimiter to be a non-standard ascii character or more than one character. It also does not allow \n or \r in unquoted columns.

Any help is greatly appreciated.

Answer 1

It can be done! However you have to specifically configure SSMS to use quoted output, because for some daft reason it is not the default.

In the query window you want to save go to Query -> Query Options...

Check the box "quote strings containing list separators when saving .csv results".

then

select 'apple,banana,cookie' as col1,1324 as col2,'one two three' as col3,'a,b,"c",d' as col4 

will output

col1,col2,col3,col4 "apple,banana,cookie",1324,one two three,"a,b,""c"",d" 

which is what we all want.

Answer 2

I've been trying to figure this out as well. Not sure if this will work for you since your table is much larger than mine, but this is what I did just out of a whim:

1. I pulled up my table in Express by doing a SELECT * statement
2. Simply selected the resulting rows and Ctrl + C
3. Opened Excel
4. Highlighted the amount of columns the table I was pasting had
5. Pasted, and it friggin' worked!
6. Now just have to Export Excel as CSV and done.

I know it probably sounds stupid, but it actually worked for me.