Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

SSIS is dropping a record on flat file source import

am experiencing a very strange issue in SSIS (2008).

Basic workflow is as follows..

Using a flatfile source (CSV), bring into SSIS, push into SQL.

When process is run on dev environment, everything works perfectly.

When the dtsx package is placed in production.. using the exact same flat file source, the last record in the file is dropped by the time it gets to the start of the SQL proc.

Have gone over everything i can possibly think of including line delimiters, column delimeters, rebuilding the flat file source connection.

Has anyone seen anything like this before?

The CSV file contains 10 records and has 4 columns. It is comma delimited and line delimited by {CR}{LF}. The file was produced by Excel and has a blank line at the end of the CSV file.

Let me know if more information is required. Im not sure what else i can offer.

like image 883
Grant Avatar asked May 27 '11 05:05

Grant


People also ask

How do I import a flat file into SSIS?

1. Create a new SSIS package and a new Data Flow Task within the package. 2. Drag the Flat File Source adapter from the Toolbox onto the Data Flow workspace and then double-click the Flat File Source to open the Flat File Source Editor.

How do I parameterize a Flat File connection manager in SSIS?

This is actually really easy in SSIS. Select the connection manager in the connection managers pane and hit F4 to open the properties pane. Find Expressions, and open that up. Then you can change the values of the properties you need in there, and use the parameters/variables to define the values.

How you can skip initial 200 rows while uploading a flat file in SSIS?

For SSIS, in the Flat File Connection Manager, you have an option that mentions Header rows to skip. On that option, you need to set it to 2 and check the Column names in the first row checkbox. That should handle the file if the row delimiters are consistent.


2 Answers

I had this exact same problem. What I found out was that when I copy an SSIS package from one server to another sometimes the text qualifiers for flat file sources get messed up, so instead of <none> it will have _x003C_none_x003E_. Once I fixed that no records were dropped.

like image 130
Dan Avatar answered Sep 30 '22 14:09

Dan


I had the same problem, and I resolved it yesterday by making sure the individual columns were not TextQualified. Setting them all to false magically made mine work. I hope that helps.

like image 36
Eric Lynes Avatar answered Sep 30 '22 14:09

Eric Lynes