Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Bulk insert rowterminator issue

I have this csv named test.csv with the content below

1,"test user",,,4075619900,[email protected],"Aldelo for Restaurants","this is my deal",,"location4"
2,"joe johnson",,"32 bit",445555519,[email protected],"Restaurant Pro Express","smoe one is watching u",,"some location"

Here is my SQL FILE to do the BULK insert

USE somedb
GO

CREATE TABLE CSVTemp
(id INT,
name VARCHAR(255),
department VARCHAR(255),
architecture VARCHAR(255),
phone VARCHAR(255),
email VARCHAR(255),
download VARCHAR(255),
comments TEXT,
company VARCHAR(255),
location VARCHAR(255))
GO

BULK
INSERT CSVTemp
FROM 'c:\test\test.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '''+CHAR(124)+''+CHAR(10)+'''
)
GO
--Check the content of the table.
SELECT *
FROM CSVTemp
GO

but whats happening is its only inserting one record and all the info from the second record is getting inserted in the location field on the first record

  id,name,department,architecture,phone,email,download,comments,company,location
  1,"test user",NULL,NULL,4075619900,[email protected],"Aldelo for Restaurants","this is my deal",NULL,"""location4""2,""joe johnson"",,""32 bit"",445555519,[email protected],""Restaurant Pro Express"",""smoe one is watching u"",,""some location"""

I assume the problem is the ROWTERMINATOR but i tried all these

ROWTERMINATOR = '\n'
ROWTERMINATOR = '\r\n'
ROWTERMINATOR = '\r'

and all the same results ...any ideas on how to FIX this

I am creating the csv like this via PHP

like image 261
Matt Elhotiby Avatar asked Sep 13 '11 14:09

Matt Elhotiby


People also ask

How to specify rowterminator without using a format file?

INSERT INTO STAGING_TABLE SELECT * FROM OPENROWSET (BULK 'FILE_LOCATION' ...); But that doesn't allow you to specify a ROWTERMINATOR without using a Format File. Hence you need the Format File in either case.

What is the default row Terminator for bulk import?

When you specify as a row terminator for bulk import, or implicitly use the default row terminator, bcp and the BULK INSERT statement expect a carriage return-line feed combination (CRLF) as the row terminator.

What's wrong with \L as the rowterminator?

The weird thing with \l as the rowterminator is that even though it load successfully it still doesn't respect the rowterminator... The rows just get loaded into all 7 columns and split on seemingly random intervals. Anyone have any idea?

What does rowterminator mean?

Specifies the field terminator to be used for character and Unicode character data files. The default is t (tab character). ROWTERMINATOR ='row_terminator'. Specifies the row terminator to be used for character and Unicode character data files.


2 Answers

I think problem is that your csv file uses \n as EOL (unix way). BULK INSERT in SQL Server is "smart" and even if you specify ROWTERMINATOR as \n, which in theory should resolve your problem, it prepends it with \r so you end up with \r\n as row terminator.

Try using ROWTERMINATOR='0x0A'. In this case SQL Server doesn't perform any magic tricks and just uses the value you've set as row terminator.
Works for me. :)

like image 184
Przemek Ptasznik Avatar answered Sep 28 '22 22:09

Przemek Ptasznik


CHAR(124) is | and CHAR(10) is \n

You probably need CHAR(13) and CHAR(10) as Row Terminator

http://www.techonthenet.com/ascii/chart.php

like image 39
Icarus Avatar answered Sep 28 '22 21:09

Icarus