very large string in memory

Question

I am writing a program for formatting 100s of MB String data (nearing a gig) into xml == And I am required to return it as a response to an HTTP (GET) request .

I am using a StringWriter/XmlWriter to build an XML of the records in a loop and returning the

using (StringWriter writer = new StringWriter())
using (writer = XmlWriter.Create(writer, settings)) //where settings are the xml props

writer.ToString() 

during testing I saw a few --out of memory exceptions-- and quite clueless on how to find a solution? do you guys have any suggestions for a memory optimized delivery of the response?

is there a memory efficient way of encoding the data? or maybe chunking the data -- I just can not think of how to return it without building the whole thing into one HUGE string object

thanks

-- a few clarifications -- this is an asp .net webservices app over a gigabit ethernet link as josh noted. I am not very familiar with it so still a bit of a learning curve.

I am using XMLWriter to create the XML and create a string out of it using String

some stats -- response xml size = about 385 megs (my data size will grow very quickly to way more than this)

string object size as calculated by a memory profiler = peaked at 605MB

and thanks to everyone who responded...

Answer 1

Use XmlTextWriter wrapped around Reponse.OutputStream to send the XML to the client and periodically flush the response. This way you never have to have more than a few mb in memory at any one time (at least for sending to the client).