Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

C# Very Large String Manipulation (Out of Memory Exception)

Tags:

string

c#

file-io

I have a need to read in a 1gb raw text file from disk to ram to do some string manipulation in C#.

string contents = File.ReadAllText(path)

is throwing out of memory exceptions (unsurprisingly)

What is the best way to go about this?

like image 741
Luke Belbina Avatar asked May 09 '11 22:05

Luke Belbina


People also ask

What C is used for?

C programming language is a machine-independent programming language that is mainly used to create many types of applications and operating systems such as Windows, and other complicated programs such as the Oracle database, Git, Python interpreter, and games and is considered a programming foundation in the process of ...

What is C in C language?

What is C? C is a general-purpose programming language created by Dennis Ritchie at the Bell Laboratories in 1972. It is a very popular language, despite being old. C is strongly associated with UNIX, as it was developed to write the UNIX operating system.

What is the full name of C?

In the real sense it has no meaning or full form. It was developed by Dennis Ritchie and Ken Thompson at AT&T bell Lab. First, they used to call it as B language then later they made some improvement into it and renamed it as C and its superscript as C++ which was invented by Dr.

Is C language easy?

Compared to other languages—like Java, PHP, or C#—C is a relatively simple language to learn for anyone just starting to learn computer programming because of its limited number of keywords.


2 Answers

Possibly also look at using a memory-mapped file

like image 161
Dave Avatar answered Sep 23 '22 17:09

Dave


If you REALLY want to do this huge string manipulation in memory then you are NOT out of luck anymore, provided you can meet the following requirements

  1. Compile targeting x64
  2. Run in a x64 system
  3. Target .NET 4.5

This will lift all the memory limitations you're facing. Your process memory will be limited only by your computer memory, and there is not a 2GiB limit on a single .NET object starting in .NET 4.5 for x64.

like image 22
Loudenvier Avatar answered Sep 23 '22 17:09

Loudenvier