Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Convert iso-8859-1 to utf-8 javascript

I try to parse a "iso-8859-1" page and save to my DB with utf-8, this is my code:

var buffer = iconv.encode(data, "iso-8859-1");
data = iconv.decode(buffer, 'utf8');

It doesn't work. All symbols like å or ä convert to �

How can I save these symbols?

like image 572
MrBinWin Avatar asked Nov 26 '14 17:11

MrBinWin


People also ask

Is ISO 8859 the same as UTF 8?

UTF-8 is a multibyte encoding that can represent any Unicode character. ISO 8859-1 is a single-byte encoding that can represent the first 256 Unicode characters. Both encode ASCII exactly the same way.

How do you convert to UTF?

Click Tools, then select Web options. Go to the Encoding tab. In the dropdown for Save this document as: choose Unicode (UTF-8). Click Ok.

What is encoding ISO 8859?

ISO/IEC 8859-1 encodes what it refers to as "Latin alphabet no. 1", consisting of 191 characters from the Latin script. This character-encoding scheme is used throughout the Americas, Western Europe, Oceania, and much of Africa.


2 Answers

You need a third-party library for that task. You are using iconv-lite so you need to follow these steps:

  1. Open input file in binary mode, so JavaScript doesn't assume UTF-8 nor try to convert to its internal encoding:

    var fs = require("fs");
    var input = fs.readFileSync(inputFilePath, {encoding: "binary"});
    
  2. Convert from ISO-8859-1 to Buffer:

    var iconv = require('iconv-lite');
    var output = iconv.decode(input, "ISO-8859-1");
    
  3. Save Buffer to output file:

    fs.writeFileSync(outputFilePath, output);
    

If unsure of encoding names, you can test whether a given encoding is supported with encodingExists():

> iconv.encodingExists("ISO-8859-1");
true
like image 142
Álvaro González Avatar answered Oct 04 '22 10:10

Álvaro González


It's working for me:

var tempBuffer = new Buffer(response.body, 'iso-8859-1');
var iconv = new Iconv('ISO-8859-1', 'UTF-8');
var tempBuffer = iconv.convert(tempBuffer);

used 'iconv' module https://github.com/bnoordhuis/node-iconv

like image 4
Vitalii Andrusishyn Avatar answered Oct 04 '22 12:10

Vitalii Andrusishyn