UTF-8 encoding: Unescape(%e2%80%93) returns â instead of –

Question

I have a search page that has a dropdown of database names that can be searched for. When a database is selected, the name is used in a querystring to get the search results. I then have a function to save the search, which simply takes the querystring of the current URL and saves it to a database. However, one of my database names is causing problems because a hyphen is not getting encoded correctly.

The name of the database is "Barnstable, MA: Probate Records 1685–1789." The querystring generated is

?database=Barnstable%2C%20MA%3A%20Probate%20Records%201685–1789

This is fine, the querystring gets the results that I need. But when I grab the current querystring with 'window.location.search,' the hyphen turns into UTF-8 encoding, "%e2%80%93." This is my Save function:

function SaveSearch(query, url) {
    var title = $("#save-name").val();
    query = removeParam("page", query);
    if (title) {
        $.ajax({
            url: url,
            type: 'post',
            success: function (info) {
                console.log("INFO: ");
                console.log(info);
                if (info == "Success") {
                    // change icon
                    $('#name-search').hide();
                    $('#save-name').val('');
                    $("#search-saved").show();
                } else {
                    $("#not-logged-in").show();
                }
            },
            error: function (info) {
                $("#error").show();
            },
            data: { queryParams: query, title: title }
        });
    }
}

When I debug it, if I hover of 'query' right at the very beginning, it shows it as

database=Barnstable%2C%20MA%3A%20Probate%20Records%201685%e2%80%931789

even though I haven't changed anything in the encoding, and the url at the top of the page has a hyphen in it. So it ends up in my database as %e2%80%93.

The problem occurs when I try to load this search:

 $.ajax({
    url: url,
    type: 'get',
    async: true,
    dataType: 'json',
    success: function (data) {
        for (var i = 0; i < data.length; i++) {
            var query = data[(data.length - (i + 1))].QueryParams;
            var params = (data[(data.length - (i + 1))].QueryParams).split("&");
            console.log(params);
            var paramDisplay = "";
            for (var j = 0; j < params.length; j++) {
                var param = params[j].split("=");
                var label = labelArray[param[0]];
                if (label != undefined) {
                    var paramString = label + ": " + unescape(param[1]);
                    paramDisplay += paramString;
                    if (j < params.length - 1) {
                        paramDisplay += "<br />";
                    }
                }
            }..........

unescape(param[1]) returns "Barnstable, MA: Probate Records 1685â1789"

Then when I try to do this search, it doesn't get any results because the name has a wrong character in it.

Answer 1

Pay attention to what it's giving you.

Specifically, %e2 as a single character is "â", while %80 and %93 are not defined characters within the ISO-8859-1 character set.

That is to say, your UTF-8 encoded character is being processed as three ISO-8859-1 characters instead.

unescape is not the right function to use. Use decodeURIComponent instead.