UTF-16 Hex Decode NodeJS

Question

I am trying to decode a UTF-16 Hex (Hello 世界) to a String in NodeJS. I've tried doing so by making a buffer from the hex:

let vari = new Buffer.from('00 48 00 65 00 6C 00 6C 00 6F 00 20 4E 16 75 4C', 'hex').toString();

But when I console log the 'vari' I do not get any/the correct result. I've tried passing in 'utf8' and 'utf16le' to the toString method but it doesnt seem to work either. Can anyone point me in the right direction?

Answer 1

It's not working because you are creating a new buffer out of the representation of the buffer as a string. This will result in a buffer then when decoded, will be the string of the buffer '00 48 00 65 00 6C 00 6C 00 6F 00 20 4E 16 75 4C', but because of hex the buffer is going to be empty. if you are to console.log(Buffer.from('00 48 00 65 00 6C 00 6C 00 6F 00 20 4E 16 75 4C', 'hex') you will see an empty buffer.

Also, '00 48 00 65 00 6C 00 6C 00 6F 00 20 4E 16 75 4C' is not a UTF-16 hex representation of "Hello 世界". when encoded as a string, it is this: 䠀攀氀氀漀 ᙎ䱵. 48 00 65 00 6c 00 6c 00 6f 00 20 00 16 4e 4c 75 is "Hello 世界" in UTF-16 hex, I got this from running console.log(Buffer.from('Hello 世界', 'utf16le'));.

To answer the question on how you can convert '48 00 65 00 6c 00 6c 00 6f 00 20 00 16 4e 4c 75' back to "Hello 世界" you can do the following:

let hexStrings = '48 00 65 00 6c 00 6c 00 6f 00 20 00 16 4e 4c 75'.split(' '); // split string chunks
let hex = hexStrings.map(x => parseInt(x, 16)); // convert string chunks to hexadecimal
let buffer = Buffer.from(hex); // create buffer from hexadecimal array
let string = buffer.toString('utf16le'); // convert buffer to string
console.log(string); // output -> Hello 世界

Hope this helps!