Weird malformed cookies returned from REST api (sometimes)

Question

I have an interesting problem - sometimes malformed cookies are returned from my web API hosted on AWS (.NET Core 2.2), behind Incapsula WAF and AWS elb. This messes up my .NET framework SDK which throws exception when trying to parse headers complaining about server doing protocol violation (suggestions are usually this: https://weblog.west-wind.com/posts/2007/Mar/29/The-server-committed-a-protocol-violation-with-WebRequest - setting useUnsafeHeaderParsing to true).

In fiddler, I managed to figure out the problem (likely):

Anyone knows where these cookie headers come from? (as you can see some headers are added by AWS/WAF), but they don't seem to be the problem. Important: those __utm* cookies are only set sometimes. Googling it, only points to possible GA cookies, but they seem different (already checked https://www.webtrafficexchange.com/google-analytics-cookies-utma-utmb-utmz)

Answer 1

See this reply we got from Imperva support regarding this Incapsula issue:

The Imperva Cloud WAF injects cookies in client requests, purposely malformed, as part of the client classification process. This is known as the classification cookie or testing cookie. This allows the WAF to see if the client can parse that purposely malformed cookie to determine the client making the request, i.e. Bot or Human (browser) all modern browsers can parse the cookie. This is known to cause issues with APIs & we can disable this on your request. Would you like us to disable this cookie? Please confirm. Its disabled by Imperva admins only, via a back-end configuration.

I believe it should resolve your issue.

Edit for additional info/context (following PST's comment):

Imperva blog post on their anti-bots client classification process: https://www.imperva.com/blog/how-incapsula-client-classification-challenges-bots/

The entire process is based on using challenges where bots are expected to react differently than regular browsers or humans.

Specifically on the topic of the original issue with cookies:

It seems Incapsula first used plain cookies based on the fact that most bots did not return cookies to the server at all. Bots than got smarter and many of them now do return the cookies. So the process was improved to include a malformed cookie that regular browsers discard (allowing Incapsula to again tell them apart from bots). So now they seem to be sending both types of cookies (as can be seen in the original post) for the full solution: A regular one to see if the client supports cookies at all. A malformed one to still tell apart bots even if they do support cookies.

sources:

1. https://github.com/postmanlabs/newman/issues/2747
2. https://www.neptuneenergy.com/cookie-policy (quoted below in case it changes)

__utmv Incapsula Classification cookie: To see how the client reacts and handles a malformed cookie in order to identify what that client is. lives up to 900 seconds