Remove last line from file with Powershell

Question

I am using

gc FileWithEmptyLines.txt | where {$_ -ne ""} > FileWithNoEmptyLines.txt

to remove the empty lines that SSRS puts at the bottom of my CSVs.

However, the last line, which has data on it, ends with a CRLF (as viewed in Notepad++) - and this is not removed, so technically there is still a blank line at the bottom of the file.

Is there a way to remove this CRLF from the last line (and keep the data intact, of course)?

Answer 1

If you already know that the very last thing of the file is a CRLF you want to get rid of (and you know the encoding too) you can go the quick route:

$stream = [IO.File]::OpenWrite('foo.txt')
$stream.SetLength($stream.Length - 2)
$stream.Close()
$stream.Dispose()

This is an in-place truncation of the file. It works without reading all the file into memory (very nice if you have a very large file). It works for ASCII, Latin-* and UTF-8. It won't work that way for UTF-16 (you'd have to remove four bytes from the end, in that case).

You can include an additional check that the last two bytes are really what you want to remove:

$stream = [IO.File]::Open('foo.txt', [IO.FileMode]::Open)
$stream.Position = $stream.Length - 2
$bytes = 0..1 | %{ $stream.ReadByte() }
$compareBytes = 13,10 # CR,LF
if ("$bytes" -eq "$compareBytes") {
    $stream.SetLength($stream.Length - 2)
}
$stream.Close()
$stream.Dispose()

Again, adapt if you use another encoding, e.g. for UTF-16 you need to compare to either 0,10,0,13 or 10,0,13,0.

Agreed, this is not very PowerShell-ey, but ever since I had to process a 700-MiB database dump I am wary of reading potentially large files into memory completely ;)

Answer 2

When you read a file using Get-Content it streams each line down the pipe as a string. When Out-File (essentially what > is an alias for) gets these strings, it always appends a line terminator sequence. Try the following if the files are not too big:

$text = [IO.File]::ReadAllText("c:\FileWithEmptyLinesAtEnd.txt")
[IO.File]::WriteAllText("c:\FileWithEmptyLinesAtEnd.txt", $text.TrimEnd())

This is the file before:

14> fhex .\FileWithEmptyLinesAtEnd.txt

Address:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F ASCII
-------- ----------------------------------------------- ----------------
00000000 73 65 72 76 65 72 31 2C 73 65 72 76 65 72 32 2E server1,server2.
00000010 64 6F 6D 61 69 6E 2E 6C 6F 63 61 6C 2C 73 65 72 domain.local,ser
00000020 76 65 72 33 0D 0A 20 20 20 20 20 20             ver3..

and after:

19> fhex .\FileWithEmptyLinesAtEnd.txt

Address:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F ASCII
-------- ----------------------------------------------- ----------------
00000000 73 65 72 76 65 72 31 2C 73 65 72 76 65 72 32 2E server1,server2.
00000010 64 6F 6D 61 69 6E 2E 6C 6F 63 61 6C 2C 73 65 72 domain.local,ser
00000020 76 65 72 33                                     ver3