Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to convert \uXXXX unicode to UTF-8 using console tools in *nix

I use curl to get some URL response, it's JSON response and it contains unicode-escaped national characters like \u0144 (ń) and \u00f3 (ó).

How can I convert them to UTF-8 or any other encoding to save into file?

like image 600
Krzysztof Wolny Avatar asked Jan 09 '12 21:01

Krzysztof Wolny


People also ask

How do I encode Unicode to UTF-8?

Base Convert Unicode symbols to UTF-8 in this base. Set the byte delimiter character here. Add a Prefix Use prefix "0b" for binary, prefix "o" for octal, and prefix "0x" for hex values. Add Padding Add zero padding to small values to make them all the same length.

How do I convert a file to UTF-8?

Click Tools, then select Web options. Go to the Encoding tab. In the dropdown for Save this document as: choose Unicode (UTF-8). Click Ok.

Is UTF-8 the same as Unicode?

The Difference Between Unicode and UTF-8Unicode is a character set. UTF-8 is encoding. Unicode is a list of characters with unique decimal numbers (code points).


1 Answers

Might be a bit ugly, but echo -e should do it:

echo -en "$(curl $URL)" 

-e interprets escapes, -n suppresses the newline echo would normally add.

Note: The \u escape works in the bash builtin echo, but not /usr/bin/echo.

As pointed out in the comments, this is bash 4.2+, and 4.2.x have a bug handling 0x00ff/17 values (0x80-0xff).

like image 199
Kevin Avatar answered Oct 05 '22 17:10

Kevin