Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Fast way to change txt file and save it

I have txt file(65mb)i need to read line by line and change each line, For example i have many lines

User=value Password=value  Phone=123456789
User=value Password=value  Phone=123456789
User=value Password=value  Phone=123456789

and i need to change first number of credit card/Phone to*(security reason), and get text like this and save it, or just to change origin text file.

User=value Password=value  Phone=*****6789
User=value Password=value  Phone=*****6789
User=value Password=value  Phone=*****6789

I build new string and add to there line(changed) by line than save, but it take me to many time this is my code

 string NewPath = "";
        string lineOfText;
        string NewTextFile = "";
        using (var filestream = new FileStream(FilePath,
                             FileMode.Open,
                             FileAccess.Read,
                             FileShare.ReadWrite))
        {
            var file = new StreamReader(filestream, Encoding.UTF8, true, 128);

            while ((lineOfText = file.ReadLine()) != null)//here i reading line by line
            {
                NewTextFile += lineOfText.Substring(0, 124) + "************" +
                lineOfText.Substring(136, lineOfText.Length - 136);
                NewTextFile += Environment.NewLine;//here i make new string
            }
        }

        NewPath = FilePatharr[1] + "\\temp.txt";
        System.IO.File.WriteAllText(NewPath, NewTextFile);//here i save him

Do any one know better way to do this,my code is taking to long to save this big file.
UPDATE

Why do i get -2 for this question? Whats wrong with this question? I see here only wrong answers about how to pass sensitive data and more things that not belong to this questions When the question was -->Fast way to change txt file and save it

Any way i find out how to do this speed of saving file speedUp from 100kb\sec to 3MB\sec now it taking me 20sec and not 20min like before

like image 457
user3567884 Avatar asked May 26 '14 12:05

user3567884


People also ask

How do I create a .TXT file in Windows?

Windows 10 Right-click in the folder and go to New > Text Document. The text file is given a default name, New Text Document. txt, but the file name is highlighted. Type a new name for the file and press Enter.

How do I create a TXT file in Word?

Another way to create a text file is to right-click an empty area on the desktop, and in the pop-up menu, select New, and then select Text Document. Creating a text file this way opens your default text editor with a blank text file on your desktop. You can change the name of the file to anything you want.


2 Answers

Your primary problem here is that you're appending to a string. And that gets expensive very quickly. You should be able to process that 65 MB in about five seconds. Here's what I would do:

string outputFileName = "temp.txt";
using (var outputFile = new StreamWriter(outputFileName))
{
    foreach (var line in File.ReadLines(inputFileName))
    {
        var newLine = line.Substring(0, 124) + "************" +
                    line.Substring(136, lineOfText.Length - 136);
        outputFile.WriteLine(newLine);
    }
}

This is going to be a lot faster than appending strings. If you really want to do it all in memory, then use a StringBuilder. Instead of

string NewTextFile = "";

Use

StringBuilder NewTextFile = new StringBuilder();

And when you're composing the output, replace the string concatenation with:

NewTextFile.AppendLine(
    lineOfText.Substring(0, 124) + "************" +
    lineOfText.Substring(136, lineOfText.Length - 136));

Finally, to write it to the file:

System.IO.File.WriteAllText(NewPath, NewTextFile.ToString());
like image 178
Jim Mischel Avatar answered Oct 05 '22 07:10

Jim Mischel


You could keep WriteAllLines and use ReadAllLines:

  1. ReadAllLines to get all lines as an array of strings
  2. Enumerate the array and use string.Replace to replace its content, line by line
  3. WriteAllLines and overwrite existing file

Or you could use stream reader/writer and read/write line by line. You currently are building a string with millions of concatenations, that's the performance issue.

That being said, you probably have an obvious security issue here. Why was critical information such as credit card numbers stored as plain text in the first place?

like image 26
ken2k Avatar answered Oct 05 '22 07:10

ken2k