Twisted Conch - Flow control

Question

I have a Twisted Conch SSH server and the typical scenario is this:

git via OpenSSH client >>--- WAN1 --->> Twisted conch svr >>--- WAN2 -->> Git server

There will be occassions that the 'git push' is sending data faster over WAN1 than I can proxy it over WAN2, so I need to tell the client to slow down (well before any TCP packet loss causes adjustments the TCP window size) to avoid buffering too much on the Twisted server. Reading the RFC for SSH this is accomplished with not acknowledging via adj window this will then cause the git push to block on syscall write to the pipe backed by openssh.

Looking at conch/ssh/connection.py:L216 in the method def ssh_CHANNEL_DATA(self, packet): I can accomplish this with setting localWindowSize to 0, and inflight data will still land as the predicate on 230 should still pass (give localWindowLeft). I am wondering if this is the correct approach or am I missing something blindly obvious with regards to flow control with Twisted SSH Conch? *

Note: I acknowledge there are methods placeholders for stopWriting and startWriting on (channel) that I can override so I have hooks to control the other side of the transmission 'git pull', but Im interested in the other side. Also IPush/IPull producer dont seem applicable at this level and I cant see how I can tie in these higher abstraction without butchering conch?

Answer 1

I haven't used Twisted and don't know Conch at all, but with nobody else answering, I'll give it a shot.

As a general principle, you probably want to buffer very little if any in the middle of the network. (Jim Gettys' notes on "buffer bloat" are enlightening.) So it's clear that you're asking a sensible question.

I assume Conch calls a function in your code when data arrives from the client. Does it suffice to simply not return from that call until you can deliver the data to the backend server? The kernel is still going to buffer data in both the inbound and outbound sockets, so the condition won't be signalled to the downstream client immediately, but I'd expect it to settle into a steady state.

As an alternative, of course, you could tunnel across this router at a different layer than SSH. If you tunnel at a lower layer so you have one end-to-end TCP connection, then the TCP stack should figure out a good window size.

If you tunnel at a higher layer, by doing a git push to the intermediate server and then using a post-receive hook to push the objects the rest of the way, then you get maximal buffering (it's all spooled to disk) and faster response time to the client, though longer total latency. It has the distinct advantage of being much simpler to implement.