Unable to translate bytes [FC] at index 35 from specified code page to Unicode

Question

I'm trying to send an object like this to my REST API(built with asp net core)

{
    "firstName":"tersü",
    "lastName":"asda"
}

And this is how the headers form SoapUI look:

 Accept-Encoding: gzip,deflate
Content-Type: application/json:charset=UTF-16
Host: localhost:4004
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)

However, my actionContext.ModelState is always invalid because it can not work with the umlaute. The exception is the following:

Unable to translate bytes [FC] at index 35 from specified code page to Unicode

If it's any help, the method signature looks like this:

[ValidateUserData]
public async Task<IActionResult> Update(string userId, [FromBody] UpdateUserRequest updateRequest)

Basically the code never goes over

if (!actionContext.ModelState.IsValid)
{
    actionContext.Result = new BadRequestObjectResult(actionContext.ModelState);
}

inside the [ValidateUserData] attribute

What am I missing here?

Answer 1

You are sending your string encoded in utf-16, but telling (in the Content-Type header's charset) it is utf-8.

The bytes for tersü in utf-8 are:

74,65,72,73,C3,BC

However tersü (in utf-16) contains the bytes (notice the FC there):

74,0,65,0,72,0,73,0,FC,0

(Check it in this fiddle)

So it just can't understand it. So either convert your string to utf-8 in your client before sending it, or set the Content-Type charset to utf-16 .

Answer 2

Although the Content-Type is charset UTF-8 the received byte code FC denotes the extended ASCII character 252 which represents the umlaut "ü".

In a UTF-8 encoding the umlaut "ü" consists of two bytes. So there is a mismatch between the given encoding header and the transmitted data. So you have to check the code which generates the request.