Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Error Importing flat files into Microsoft SQL Server Management Studio 18 -- "System Out of Memory"

Tags:

sql-server

I have an approximately 10 GB Excel CSV that I am trying to import into a SQL Database. When I use the import flat file option / wizard the file starts to import for about 10-15 seconds before running into a System Out of Memory Exception Error. The follow is the error output and screenshots.

Any information is greatly appreciated!

Detailed Error Output:
Error inserting data into table. (Microsoft.SqlServer.Import.Wizard)

------------------------------
Program Location:

   at Microsoft.SqlServer.Import.Wizard.InsertData.ResultCheck(Result result)
   at Microsoft.SqlServer.Import.Wizard.InsertData.DoWork()
   at Microsoft.SqlServer.Management.TaskForms.SimpleWorkItem.Run()

===================================

Error inserting data into table. (Microsoft.SqlServer.Prose.Import.SDS)

------------------------------
Program Location:

   at Microsoft.SqlServer.Prose.Import.BcpTextSynthesis.InsertIntoDB(String inputFilePath, String tableName, String schemaName, IReadOnlyList`1 columnInfo, SqlConnection connection, Int32 batchSize, SqlTransaction transaction, IList`1 allFinalTransformations, IList`1 allFinalTransformationColumns, IList`1 allFinalColNames)
   at Microsoft.SqlServer.Prose.Import.BcpProcess.CreateTableAndInsertDataIntoDb(String connectionString, Int32 batchSize, String azureAccessToken)

===================================

Exception of type 'System.OutOfMemoryException' was thrown. (mscorlib)

------------------------------
Program Location:

   at System.Text.StringBuilder.ToString()
   at Microsoft.ProgramSynthesis.Read.FlatFile.Semantics.Semantics.<SplitLines>d__17.MoveNext()
   at Microsoft.ProgramSynthesis.Read.FlatFile.Semantics.Semantics.<Csv>d__9.MoveNext()
   at System.Linq.Enumerable.<SkipIterator>d__31`1.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Microsoft.SqlServer.Prose.Import.BcpTextSynthesis.InsertIntoDB(String inputFilePath, String tableName, String schemaName, IReadOnlyList`1 columnInfo, SqlConnection connection, Int32 batchSize, SqlTransaction transaction, IList`1 allFinalTransformations, IList`1 allFinalTransformationColumns, IList`1 allFinalColNames)





[Image of Import Wizard][1]


[Error Image 'System.OutOfMemoryException' was thrown][2]
like image 823
jchian Avatar asked Oct 16 '25 08:10

jchian


1 Answers

I used the Bulk insert command as per below and please make sure that the files to be load is on the SQL server and not on your local/remote machine.

BULK INSERT dbo.TABLE
 FROM 'E:\YOUR_FILES.csv'
WITH ( FORMAT = 'CSV',
 FIELDTERMINATOR = ',',
   ROWTERMINATOR = '\n' );
like image 70
mytkavish Avatar answered Oct 19 '25 01:10

mytkavish



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!