Why is my ajax post being truncated?

Question

I have just updated my mvc service to include greater error and logging. I have now got this exact error several times. But cannot replicate.

Unterminated string. Expected delimiter: ". Path 'Breadcrumbs[18].Params', line 1, position 59740. at Newtonsoft.Json.JsonTextReader.ReadStringIntoBuffer(Char quote) at 

The Path is different each time, depending on what the user is sending to the server.

My ajax requests generally look like this:

$.ajax(myURL("SendBreadcrumbs"), {
            type: "POST",
            cache: false,
            data: { telemetry: telemetry.JSONify(), userID: currentUser, isMyVI: isMyVI }
        })

In these cases, userID and isMyVI (boolean) didnt exist and the telemetry string is truncated.

JSONify method is as follows:

self.JSONify = function () {
    var data = ko.mapping.toJSON(self, TelemetryMapping);
    return data;
}

This is knockoutJSs serializer.

Server Side:

public void SendBreadcrumbs(string telemetry, string userID = null, bool isMyVI = true)
{
    MyServiceAudit audit;
    Guid UserID;
    if (Guid.TryParse(userID, out UserID))
        audit = InsertAudit(UserID);
    else
        audit = InsertAudit(null);
    try
    {
        Models.Telemetry data = JsonConvert.DeserializeObject<Models.Telemetry>(telemetry);
        Controllers.Telemetry.UpdateTelemetry(data, isMyVI);
    }
    catch (Exception ex)
    {
         MyServiceAuditDB.FailAudit(audit.ID, ex.Message + " " + ex.StackTrace);
    }
}

I am completely stumped, i have tried updating the web.config on the server with larger maxvalues etc, to see its truncating on that side.

The only other difference i have in my ajax is a global timeout of 3mins.

Is this as simple as special characters not being handled on client side, or server side limitations, or is the data so large it gets chunked and server side doesn't know to wait for the next chunk?

Answer 1

With help from a few comments, answers and finally got live failed data, I was able to resolve this.

It turned out that not only did the user use special characters, but also some of the static data sent down included this (GIGO - couldn't believe the state of some of our data).

Client Side Solution:

The encodeURIComponent() function encodes a URI component. This function encodes special characters. In addition, it encodes the following characters: , / ? : @ & = + $ #

$.ajax(myURL("SendBreadcrumbs"), {
        type: "POST",
        cache: false,
        data: { telemetry: encodeURIComponent(telemetry.JSONify()), userID: currentUser, isMyVI: isMyVI }
    })

Server Side Solution:

Converts a string to its unescaped representation. Uri.UnescapeDataString()

public void SendBreadcrumbs(string telemetry, string userID = null, bool isMyVI = true)
{
    MyServiceAudit audit;
    Guid UserID;
    if (Guid.TryParse(userID, out UserID))
        audit = InsertAudit(UserID);
    else
        audit = InsertAudit(null);
    try
    {
        Models.Telemetry data = JsonConvert.DeserializeObject<Models.Telemetry>(Uri.UnescapeDataString(telemetry));
        Controllers.Telemetry.UpdateTelemetry(data, isMyVI);
    }
    catch (Exception ex)
    {
         MyServiceAuditDB.FailAudit(audit.ID, ex.Message + " " + ex.StackTrace);
    }
}

Web Application Configuration:

I had a user getting a 500 error, due to trying to send 20 emails. I updated the config to include maximum request length (in kilobytes, example is 1GB) and max content length (in bytes, example is 1GB). Emails came through no issue. Couldn't believe it!

<appSettings>
    <add key="aspnet:MaxJsonDeserializerMembers" value="150000" />
</appSettings>
<system.web>
    <httpRuntime targetFramework="4.5" maxQueryStringLength="32768" maxUrlLength="65536" maxRequestLength="1048576" />
</system.web>
<system.webServer>
    <security>
        <requestFiltering>
            <requestLimits maxQueryString="32768" maxAllowedContentLength="1073741824"  />
        </requestFiltering>
    </security>
</system.webServer>
<system.web.extensions>
    <scripting>
        <webServices>
            <jsonSerialization maxJsonLength="2147483647" />
        </webServices>
    </scripting>
</system.web.extensions>