Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

tee with utf-8 encoding

I'm trying to tee a server's output to both the console and a file in Powershell 4. The file is ending up with a UTF-16 encoding, which is incompatible with some other tools I'm using. According to help tee -full:

Tee-Object uses Unicode enocding when it writes to files.
...
To specify the encoding, use the Out-File cmdlet

So tee doesn't support changing encoding, and the help for both tee and Out-File don't show any examples of splitting a stream and encoding it with UTF-8.

Is there a simple way in Powershell 4 to tee (or otherwise split a stream) to a file with UTF-8 encoding?

like image 475
Angus Avatar asked Jun 02 '15 21:06

Angus


People also ask

What does UTF-8 encoding do?

UTF-8 is an encoding system for Unicode. It can translate any Unicode character to a matching unique binary string, and can also translate the binary string back to a Unicode character. This is the meaning of “UTF”, or “Unicode Transformation Format.”

What is UTF-8 and what problem does it solve?

The problem UTF-8 solves US keyboards can often produce 101 symbols, which suggests 101 symbols would be enough for most English text. Seven bits would be enough to encode these symbols since 27 = 128, and that's what ASCII does.

What is tee object in PowerShell?

Description. The Tee-Object cmdlet redirects output, that is, it sends the output of a command in two directions (like the letter T). It stores the output in a file or variable and also sends it down the pipeline. If Tee-Object is the last command in the pipeline, the command output is displayed at the prompt.


3 Answers

One option is to use Add-Content or Set-Content instead of Out-File.

The *-Content cmdlets use ASCII encoding by default, and have a -Passthru switch so you can write to the file, and then have the input pass through to the console:

Get-Childitem -Name | Set-Content file.txt -Passthru
like image 166
mjolinor Avatar answered Sep 19 '22 16:09

mjolinor


You would have to use -Variable and then write it out to a file in a separate step.

$data = $null
Get-Process | Tee-Object -Variable data
$data | Out-File -Path $path -Encoding Utf8

At first glance it seems like it's easier to avoid tee altogether and just capture the output in a variable, then write it to the screen and to a file.

But because of the way the pipeline works, this method allows for a long running pipeline to display data on screen as it goes along. Unfortunately the same cannot be said for the file, which won't be written until afterwards.

Doing Both

An alternative is to roll your own tee so to speak:

[String]::Empty | Out-File -Path $path  # initialize the file since we're appending later
Get-Process | ForEach-Object {
    $_ | Out-File $path -Append -Encoding Utf
    $_
}

That will write to the file and back to the pipeline, and it will happen as it goes along. It's probably quite slow though.

like image 35
briantist Avatar answered Sep 17 '22 16:09

briantist


Tee-object seems to invoke out-file, so this will make tee output utf8:

$PSDefaultParameterValues = @{'Out-File:Encoding' = 'utf8'}
like image 21
js2010 Avatar answered Sep 17 '22 16:09

js2010