WMIC command in batch outputting non UTF-8 text files

Question

I'm using a WMIC command to output a list of SIDS and accompanying user profile names to text. From the text, I can edit a list of SIDS I need to add a set of registry keys to. However, the script that loops through the edited text file of SIDS is encoded in a format the script doesn't pick up and just fails to run. Using notepad++ I can re-encode from UCS-2 LE BOM to UTF-8 and then I can complete the script hassle free.

How can I make the output from the WMIC text default to UTF-8?.

I noticed this on more that one PC. To cure the problem, as mentioned, I can re-encode in notepad++ but its a step I really need to avoid if possible. Trying to automate things as much as I can. The sole issue is the encoding, all other scripts, commands, codes etc. are fine once I get a UTF-8 text file. I use batch file often and like to output to text files, looking at those they are all defaulting to UTF-8 as expected. Seems specific to WMIC command here.

WMIC Path Win32_UserProfile Where "Special='False' And Not LocalPath='Null'" Get LocalPath,SID>somefile.txt

Gives all the info I need but outputs to UCS-2 LE BOM, not UTF-8

Any assistance would be great, thanks. (was thinking maybe a reg query would bypass the issue?)

Answer 1

The encoding of wmic's output depends on where the output is being sent

• If you send the output to a disk file by using a redirection operator or the /output switch, wmic will use UCS-2 LE BOM
• If you send the output to the console or to a pipe, wmic will use OEM codepage

If your scripts can not process the UCS-2 output one simple option (without third party tools) is to change where wmic writes by using a pipe.

wmic os get localdatetime | find /v "" > someFile.txt 

Here wmic's output is piped to find /v "" (find any non empty line) and then writen to a disk file using your OEM codepage.

[W:\]:# wmic os get localdatetime > file.txt

[W:\]:# hex file.txt
HEX:       +00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  0123456789abcdef
0000000000: FF FE 4C 00 6F 00 63 00  61 00 6C 00 44 00 61 00  .■L.o.c.a.l.D.a.
0000000010: 74 00 65 00 54 00 69 00  6D 00 65 00 20 00 20 00  t.e.T.i.m.e. . .
0000000020: 20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00   . . . . . . . .
0000000030: 20 00 20 00 20 00 20 00  0D 00 0A 00 32 00 30 00   . . . .....2.0.
0000000040: 31 00 39 00 30 00 33 00  32 00 33 00 31 00 30 00  1.9.0.3.2.3.1.0.
0000000050: 31 00 34 00 34 00 30 00  2E 00 30 00 39 00 34 00  1.4.4.0...0.9.4.
0000000060: 30 00 30 00 30 00 2B 00  30 00 36 00 30 00 20 00  0.0.0.+.0.6.0. .
0000000070: 20 00 0D 00 0A 00                                  .....
[W:\]:#
[W:\]:# wmic os get localdatetime | find /v "" > file.txt

[W:\]:# hex file.txt
HEX:       +00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  0123456789abcdef
0000000000: 4C 6F 63 61 6C 44 61 74  65 54 69 6D 65 20 20 20  LocalDateTime
0000000010: 20 20 20 20 20 20 20 20  20 20 20 0D 0D 0A 32 30             ...20
0000000020: 31 39 30 33 32 33 31 30  31 35 30 34 2E 31 35 38  190323101504.158
0000000030: 30 30 30 2B 30 36 30 20  20 0D 0D 0A 0D 0D 0A     000+060  ......
[W:\]:#

If you use this approach then you should note a curious side effect: the lines in the output don't end in a CRLF sequence, but a CRCRLF sequence.

If this is also a problem to your scripts then you can use the type command to read the output file and redirect its output to generate another one with ANSI enconding

[W:\]:# wmic os get localdatetime > file.txt

[W:\]:# hex file.txt
HEX:       +00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  0123456789abcdef
0000000000: FF FE 4C 00 6F 00 63 00  61 00 6C 00 44 00 61 00  .■L.o.c.a.l.D.a.
0000000010: 74 00 65 00 54 00 69 00  6D 00 65 00 20 00 20 00  t.e.T.i.m.e. . .
0000000020: 20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00   . . . . . . . .
0000000030: 20 00 20 00 20 00 20 00  0D 00 0A 00 32 00 30 00   . . . .....2.0.
0000000040: 31 00 39 00 30 00 33 00  32 00 33 00 31 00 30 00  1.9.0.3.2.3.1.0.
0000000050: 32 00 33 00 31 00 31 00  2E 00 39 00 36 00 31 00  2.3.1.1...9.6.1.
0000000060: 30 00 30 00 30 00 2B 00  30 00 36 00 30 00 20 00  0.0.0.+.0.6.0. .
0000000070: 20 00 0D 00 0A 00                                  .....
[W:\]:#
[W:\]:# type file.txt > file2.txt

[W:\]:# hex file2.txt
HEX:       +00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  0123456789abcdef
0000000000: 4C 6F 63 61 6C 44 61 74  65 54 69 6D 65 20 20 20  LocalDateTime
0000000010: 20 20 20 20 20 20 20 20  20 20 20 0D 0A 32 30 31             ..201
0000000020: 39 30 33 32 33 31 30 32  33 31 31 2E 39 36 31 30  90323102311.9610
0000000030: 30 30 2B 30 36 30 20 20  0D 0A                    00+060  ..
[W:\]:#

The problem with this approach appears when the characters in the UCS-2 file don't have a direct equivalent in the ANSI codepage.

But if using a third party tool is a valid option, then aGerman's CONVERTCP tool (including source code if you prefer to compile it) is a good alternative to integrate in this kind of scripts.

[W:\]:# tasklist /fi "pid eq 6232"

Nombre de imagen               PID Nombre de sesión Núm. de ses Uso de memor
========================= ======== ================ =========== ============
Proceso↔Amañado↔.exe          6232 Console                    1     2.596 KB

[W:\]:# wmic process where "processID=6232" get name > file.txt

[W:\]:# hex file.txt
HEX:       +00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  0123456789abcdef
0000000000: FF FE 4E 00 61 00 6D 00  65 00 20 00 20 00 20 00  .■N.a.m.e. . . .
0000000010: 20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00   . . . . . . . .
0000000020: 20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00   . . . . . . . .
0000000030: 20 00 0D 00 0A 00 50 00  72 00 6F 00 63 00 65 00   .....P.r.o.c.e.
0000000040: 73 00 6F 00 94 21 41 00  6D 00 61 00 F1 00 61 00  s.o.ö!A.m.a.±.a.
0000000050: 64 00 6F 00 94 21 2E 00  65 00 78 00 65 00 20 00  d.o.ö!..e.x.e. .
0000000060: 20 00 0D 00 0A 00                                  .....
[W:\]:#
[W:\]:# type file.txt > file2.txt

[W:\]:# hex file2.txt
HEX:       +00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  0123456789abcdef
0000000000: 4E 61 6D 65 20 20 20 20  20 20 20 20 20 20 20 20  Name
0000000010: 20 20 20 20 20 20 20 20  0D 0A 50 72 6F 63 65 73          ..Proces
0000000020: 6F 1D 41 6D 61 A4 61 64  6F 1D 2E 65 78 65 20 20  oAmañado.exe
0000000030: 0D 0A                                             ..
[W:\]:#
[W:\]:# convertcp 1200 65001 /i file.txt /o file2.txt

[W:\]:# hex file2.txt
HEX:       +00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  0123456789abcdef
0000000000: 4E 61 6D 65 20 20 20 20  20 20 20 20 20 20 20 20  Name
0000000010: 20 20 20 20 20 20 20 20  0D 0A 50 72 6F 63 65 73          ..Proces
0000000020: 6F E2 86 94 41 6D 61 C3  B1 61 64 6F E2 86 94 2E  oÔåöAma├▒adoÔåö.
0000000030: 65 78 65 20 20 0D 0A                              exe  ..
[W:\]:#