Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Transform SQL insert script into CSV format

I'm looking for an awk command, or similar tool, to transform standard well formatted SQL insert script into csv file.
By standard I mean there is no database vendor specific stuff anywhere.
By well formatted I mean the case where each line of the sql script has a full column set to insert, even if there are NULLs. Also the order of fields to insert is the same.
Sample input SQL script:

INSERT INTO tbl VALUES (1, 'asd', 923123123, 'zx');
INSERT INTO tbl VALUES (1, NULL, 923123123, 'zxz');
INSERT INTO tbl VALUES (3, 'asd3', 923123123, NULL);

Optionally:

INSERT INTO tbl (colA, colB, colC, colD) VALUES (1, 'asd', 923123123, 'zx');

Expected output should be a csv file:

1,'asd',923123123,'zx'
1,,923123123,'zxz'
3,'asd3',923123123,

Looking for a performance efficient solution.

like image 243
jangorecki Avatar asked Aug 15 '15 15:08

jangorecki


People also ask

How do you add a script to a csv file?

If you import the CSV file into Excel, you can create a formula which creates an INSERT statement by using string concatenation in the formula. So - if your CSV file has 3 columns that appear in columns A, B, and C in Excel, you could write a formula like...

How do I insert a CSV file in SQL Server?

Then, the BULK INSERT command includes the target table and the CSV file. The location of the CSV should follow the rules of the Universal Naming Convention (UNC). You also must tell SQL Server what file it’s dealing with. In this case, FORMAT=CSV because the file is in CSV format.

How to convert a CSV file to a data source?

Paste your CSV data, or click Upload CSVto upload a CSV file, or drag-and-drop a CSV file to the Data Sourcepanel, the CSV converter will execute the conversion magic immediately. Don’t worry about the CSV delimiter, the converter will automatically determine the delimiter, it supports comma, tab, colon, semicolon, pipe, slash, octothorpe and more.

How do I import data from a CSV file into SSMS?

Option 2 : CSV to SQL using the SSMS ‘Import Flat File’ wizard. 1 Step 1: select the csv file. You can now select the csv file that you want to import. Note that the wizard will automatically populate the table name ... 2 Step 2: preview the data. 3 Step 3: modify the columns as required. 4 Step 5: check the data.

Is it possible to save the results to a CSV file?

EDIT - Is it possible to save the results to a CSV file using only SQL ? Use the BCP tool. It is part of SQL Server. Also, 'csv' is not a well-defined format. You need to think about how you want to handle fields that contain your field and row delimiters (usually comma and newline).


2 Answers

$ awk -F' *[(),]+ *' -v OFS=, '{for (i=2;i<NF;i++) printf "%s%s", ($i=="NULL"?"":$i), (i<(NF-1)?OFS:ORS)}' file
1,'asd',923123123,'zx'
1,,923123123,'zxz'
3,'asd3',923123123,

I'd recommend you test all potential solutions with this input:

$ cat file
INSERT INTO tbl VALUES (1, NULL, 923123123, 'foo NULL bar');

$ awk -F' *[(),]+ *' -v OFS=, '{for (i=2;i<NF;i++) printf "%s%s", ($i=="NULL"?"":$i), (i<(NF-1)?OFS:ORS)}' file
1,,923123123,'foo NULL bar'

to make sure the string NULL and blank chars are not deleted when they appear as part of a literal string.

like image 176
Ed Morton Avatar answered Oct 16 '22 11:10

Ed Morton


Try this with GNU grep and sed:

grep -oP '\(\K[^)]*(?=\);)' file | sed 's/NULL//g;s/ //g'

Output from all four lines:

1,'asd',923123123,'zx'
1,,923123123,'zxz'
3,'asd3',923123123,
1,'asd',923123123,'zx'

or only with GNU sed:

sed 's/.*(\([^)]*\));/\1/;s/NULL//g;s/ //g' file

Output from all four lines:

1,'asd',923123123,'zx'
1,,923123123,'zxz'
3,'asd3',923123123,
1,'asd',923123123,'zx'
like image 27
Cyrus Avatar answered Oct 16 '22 13:10

Cyrus