Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove last line from file with Powershell

I am using

gc FileWithEmptyLines.txt | where {$_ -ne ""} > FileWithNoEmptyLines.txt

to remove the empty lines that SSRS puts at the bottom of my CSVs.

However, the last line, which has data on it, ends with a CRLF (as viewed in Notepad++) - and this is not removed, so technically there is still a blank line at the bottom of the file.

Is there a way to remove this CRLF from the last line (and keep the data intact, of course)?

like image 237
stuartdotnet Avatar asked Jul 25 '12 05:07

stuartdotnet


People also ask

How do you skip the last line in PowerShell?

PowerShell Get-Content Skip first line In PowerShell, using Get-Content to skip first line of file, use Skip parameter with value to skip line. It will skip those number of lines and display rest of the lines of file.

How do I remove the first line of a file in PowerShell?

Avoid the loop variable $i and the comparison with 0 in every loop. Wrap the execution into a try.. finally block to always close the files in use. Make the solution work for an arbitrary number of lines to remove from the beginning of the file.

How do you end a line in PowerShell?

Using PowerShell newline in Command To add newline in the PowerShell script command, use the` (backtick) character at the end of each line to break the command into multiple lines.

How do you delete contents of a file in PowerShell?

The Clear-Content cmdlet deletes the contents of an item, such as deleting the text from a file, but it does not delete the item. As a result, the item exists, but it is empty. Clear-Content is similar to Clear-Item , but it works on items with contents, instead of items with values.


2 Answers

If you already know that the very last thing of the file is a CRLF you want to get rid of (and you know the encoding too) you can go the quick route:

$stream = [IO.File]::OpenWrite('foo.txt')
$stream.SetLength($stream.Length - 2)
$stream.Close()
$stream.Dispose()

This is an in-place truncation of the file. It works without reading all the file into memory (very nice if you have a very large file). It works for ASCII, Latin-* and UTF-8. It won't work that way for UTF-16 (you'd have to remove four bytes from the end, in that case).

You can include an additional check that the last two bytes are really what you want to remove:

$stream = [IO.File]::Open('foo.txt', [IO.FileMode]::Open)
$stream.Position = $stream.Length - 2
$bytes = 0..1 | %{ $stream.ReadByte() }
$compareBytes = 13,10 # CR,LF
if ("$bytes" -eq "$compareBytes") {
    $stream.SetLength($stream.Length - 2)
}
$stream.Close()
$stream.Dispose()

Again, adapt if you use another encoding, e.g. for UTF-16 you need to compare to either 0,10,0,13 or 10,0,13,0.

Agreed, this is not very PowerShell-ey, but ever since I had to process a 700-MiB database dump I am wary of reading potentially large files into memory completely ;)

like image 151
Joey Avatar answered Sep 17 '22 13:09

Joey


When you read a file using Get-Content it streams each line down the pipe as a string. When Out-File (essentially what > is an alias for) gets these strings, it always appends a line terminator sequence. Try the following if the files are not too big:

$text = [IO.File]::ReadAllText("c:\FileWithEmptyLinesAtEnd.txt")
[IO.File]::WriteAllText("c:\FileWithEmptyLinesAtEnd.txt", $text.TrimEnd())

This is the file before:

14> fhex .\FileWithEmptyLinesAtEnd.txt

Address:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F ASCII
-------- ----------------------------------------------- ----------------
00000000 73 65 72 76 65 72 31 2C 73 65 72 76 65 72 32 2E server1,server2.
00000010 64 6F 6D 61 69 6E 2E 6C 6F 63 61 6C 2C 73 65 72 domain.local,ser
00000020 76 65 72 33 0D 0A 20 20 20 20 20 20             ver3..

and after:

19> fhex .\FileWithEmptyLinesAtEnd.txt

Address:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F ASCII
-------- ----------------------------------------------- ----------------
00000000 73 65 72 76 65 72 31 2C 73 65 72 76 65 72 32 2E server1,server2.
00000010 64 6F 6D 61 69 6E 2E 6C 6F 63 61 6C 2C 73 65 72 domain.local,ser
00000020 76 65 72 33                                     ver3
like image 25
Keith Hill Avatar answered Sep 19 '22 13:09

Keith Hill