Symfony2 JsonResponse utf8 encoding issues on Debian Stable php-5.4

Question

I'm having issues with JsonResponse on Debian Stable php5 (5.4.39-0+deb7u1) when returning UTF8 chars.

I developed an app on Debian Testing php5 (5.6.6+dfsg-2) and the following code worked like a charm:

$response = new JsonResponse();
$response->headers->set('Content-Type', 'application/json');
$response->setData($data);
return $response;

but after deploying to the stable prod server I started getting the following exception for the exact same DB/Data charsets etc:

request.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "Malformed UTF-8 characters, 
possibly incorrectly encoded." at /site/vendor/symfony/symfony/src/Symfony/Component/HttpFoundation/JsonResponse.php
 line 123 {"exception":"[object] (InvalidArgumentException(code: 0): 
Malformed UTF-8 characters, possibly incorrectly encoded. at 
/site/vendor/symfony/symfony/src/Symfony/Component/HttpFoundation/JsonResponse.php:123)"} []

The response from DB that is passed as $data DO contains UTF8 chars that I can't control. I just have to display them.

I suppose I hit a bug of 5.4, but how can I easily walk around it? I did tried:

    $response = new JsonResponse();
    $response->headers->set('Content-Type', 'application/json');
    $response->setEncodingOptions(JSON_UNESCAPED_UNICODE);
    $response->setData($data);
    return $response;

but I get the same error.

Ideas?

Answer 1

After some discussing #symfony channel I found a workaround:

    $response = new Response(json_encode($data, JSON_UNESCAPED_UNICODE));
    $response->headers->set('Content-Type', 'application/json');
    return $response;

Other nice solutions are welcome. I consider this solution as a dirty hack...

Answer 2

I think you are not getting a valid UTF-8 string. Try to find out, why there are invalid utf8 byteblocks (http://en.wikipedia.org/wiki/UTF-8#Invalid_byte_sequences).

You can analyze the bytes with unpack: https://stackoverflow.com/a/11466734/4469738