TCP packet won't get from Russia to Canada when data starts with '1c'

Question

We have a TCP stream protocol where we prefix our data payload by the size. So the data can be properly decoded when received. Pretty standard stuff.

This is working fine for thousands of people. Unfortunately we have at least 4 reported cases of clients having connection problems, all in remote countries. A client in Russia has been able to help us run lots of tests and narrowed the problem down. If we send a packet where the prefix size is forced to be 0 then the entire packet will make it through. If the packet data starts with 1c the packet won't make it through.

I have two side by side Wireshark captures from his computer that show this:

Working
-------
Russia -> Toronto [SYN]
Toronto -> Russia [SYN, ACK]
Russia -> Toronto [ACK]
Russia -> Toronto [PSH,ACK] <- data is sent here
00000000000000001c0000000000000000000000000000000000000000000000
Toronto -> Rusion [PSH,ACK] <- server in toronto got the data, sent a reply!


Not-Working
--------
Russia -> Toronto [SYN]
Toronto -> Russia [SYN, ACK]
Russia -> Toronto [ACK]
Russia -> Toronto [PSH,ACK] <- data is sent here
1c000000000000001c0000000000000000000000000000000000000000000000
Russia -> Toronto [PSH,ACK] <- TCP Retransmission
Russia -> Toronto [PSH,ACK] <- TCP Retransmission
Russia -> Toronto [PSH,ACK] <- TCP Retransmission
Russia -> Toronto [PSH,ACK] <- TCP Retransmission

Server in Toronto never gets the packet from Russia!

The actual client and servers use IOCP but my test app uses C# TcpListener and TcpClient with NO custom options flags at all.

Not actual code
--------------
var client = new TcpClient()
client.Connect(host, port)
client.GetStream().Write()
client.GetStream().Read()

var listener = new TcpListener(port);
listener.Start();
var serverClient = listener.AcceptTcpClient();
serverClient.GetStream().Read()
serverClient.GetStream().Write()

Are there any further tests to recommend to get more information / solve this? My hunch was his hardware/drivers are corrupt but he claims to have no issues with any other application or internet in general.

Answer 1

Could it be that one of the end links is over a mobile carrier?

I don't have the data any more, but since we're in the realm of speculations I remember a similar problem with an Italian mobile carrier a while ago: apparently, a certain sequence of bits over a data connection would drop the carrier. Vaguely similar to the +++ATH0 old 'ping' trick.

Could you try sending a similar sequence (1c0000000....) through another medium, say a netcat stream?