I have an approximately 10 GB Excel CSV that I am trying to import into a SQL Database. When I use the import flat file option / wizard the file starts to import for about 10-15 seconds before running into a System Out of Memory Exception Error. The follow is the error output and screenshots.
Any information is greatly appreciated!
Detailed Error Output:
Error inserting data into table. (Microsoft.SqlServer.Import.Wizard)
------------------------------
Program Location:
at Microsoft.SqlServer.Import.Wizard.InsertData.ResultCheck(Result result)
at Microsoft.SqlServer.Import.Wizard.InsertData.DoWork()
at Microsoft.SqlServer.Management.TaskForms.SimpleWorkItem.Run()
===================================
Error inserting data into table. (Microsoft.SqlServer.Prose.Import.SDS)
------------------------------
Program Location:
at Microsoft.SqlServer.Prose.Import.BcpTextSynthesis.InsertIntoDB(String inputFilePath, String tableName, String schemaName, IReadOnlyList`1 columnInfo, SqlConnection connection, Int32 batchSize, SqlTransaction transaction, IList`1 allFinalTransformations, IList`1 allFinalTransformationColumns, IList`1 allFinalColNames)
at Microsoft.SqlServer.Prose.Import.BcpProcess.CreateTableAndInsertDataIntoDb(String connectionString, Int32 batchSize, String azureAccessToken)
===================================
Exception of type 'System.OutOfMemoryException' was thrown. (mscorlib)
------------------------------
Program Location:
at System.Text.StringBuilder.ToString()
at Microsoft.ProgramSynthesis.Read.FlatFile.Semantics.Semantics.<SplitLines>d__17.MoveNext()
at Microsoft.ProgramSynthesis.Read.FlatFile.Semantics.Semantics.<Csv>d__9.MoveNext()
at System.Linq.Enumerable.<SkipIterator>d__31`1.MoveNext()
at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at Microsoft.SqlServer.Prose.Import.BcpTextSynthesis.InsertIntoDB(String inputFilePath, String tableName, String schemaName, IReadOnlyList`1 columnInfo, SqlConnection connection, Int32 batchSize, SqlTransaction transaction, IList`1 allFinalTransformations, IList`1 allFinalTransformationColumns, IList`1 allFinalColNames)
[Image of Import Wizard][1]
[Error Image 'System.OutOfMemoryException' was thrown][2]
I used the Bulk insert command as per below and please make sure that the files to be load is on the SQL server and not on your local/remote machine.
BULK INSERT dbo.TABLE
FROM 'E:\YOUR_FILES.csv'
WITH ( FORMAT = 'CSV',
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n' );
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With