poll() c function on windows

Question

I want to know if I can use the poll() function on a MinGW development chain. I have CodeBlocks+MinGW. Thanks a lot.

Answer 1

Last I heard, poll() was either not supported or provided limited functionality on mingw

That was in 2009.

In 2020... it is supported, but still "flawed", as illustrated with Git 2.25.2 (March 2020), where MinGW's poll() emulation has been improved.

See commit 94f4d01 (17 Feb 2020) by Alexandr Miloslavskiy (SyntevoAlex).
^{(Merged by Junio C Hamano -- gitster -- in commit 1ac37de, 09 Mar 2020)}

mingw: workaround for hangs when sending STDIN

^{Signed-off-by: Alexandr Miloslavskiy}

Explanation

The problem here is flawed poll() implementation.
When it tries to see if pipe can be written without blocking, it eventually calls NtQueryInformationFile() and tests WriteQuotaAvailable.

However, the meaning of quota was misunderstood.
The value of quota is reduced when either some data was written to a pipe, or there is a pending read on the pipe.
Therefore, if there is a pending read of size >= than the pipe's buffer size, poll() will think that pipe is not writable and will hang forever, usually that means deadlocking both pipe users.

I have studied the problem and found that Windows pipes track two values: QuotaUsed and BytesInQueue.

The code in poll() apparently wants to know BytesInQueue instead of quota.
Unfortunately, BytesInQueue can only be requested from read end of the pipe, while poll() receives write end.

The git's implementation of poll() was copied from gnulib, which also contains a flawed implementation up to today.

I also had a look at implementation in cygwin, which is also broken in a subtle way. It uses this code in pipe_data_available():

fpli.WriteQuotaAvailable = (fpli.OutboundQuota - fpli.ReadDataAvailable)

However, ReadDataAvailable always returns 0 for the write end of the pipe, turning the code into an obfuscated version of returning pipe's total buffer size, which I guess will in turn have poll() always say that pipe is writable.
The commit that introduced the code doesn't say anything about this change, so it could be some debugging code that slipped in.

These are the typical sizes used in git:

• 0x2000 - default read size in strbuf_read()
• 0x1000 - default read size in CRT, used by strbuf_getwholeline()
• 0x2000 - pipe buffer size in compat\mingw.c

As a consequence, as soon as child process uses strbuf_read(), poll() in parent process will hang forever, deadlocking both processes.

This results in two observable behaviors:

1. If parent process begins sending STDIN quickly (and usually that's the case), then first poll() will succeed and first block will go through.
   MAX_IO_SIZE_DEFAULT is 8MB, so if STDIN exceeds 8MB, then it will deadlock.
2. If parent process waits a little bit for any reason (including OS scheduler) and child is first to issue strbuf_read(), then it will deadlock immediately even on small STDINs.

The problem is illustrated by git stash push, which will currently read the entire patch into memory and then send it to git apply via STDIN.
If patch exceeds 8MB, git hangs on Windows.

Possible solutions

1. Somehow obtain BytesInQueue instead of QuotaUsed
   I did a pretty thorough search and didn't find any ways to obtain the value from write end of the pipe.

2. Also give read end of the pipe to poll()
   That can be done, but it will probably invite some dirty code, because poll()

   • can accept multiple pipes at once
   • can accept things that are not pipes
   • is expected to have a well known signature.

3. Make poll() always reply "writable" for write end of the pipe
   Afterall it seems that cygwin (accidentally?) does that for years.
   Also, it should be noted that pump_io_round() writes 8MB blocks, completely ignoring the fact that pipe's buffer size is only 8KB, which means that pipe gets clogged many times during that single write.
   This may invite a deadlock, if child's STDERR/STDOUT gets clogged while it's trying to deal with 8MB of STDIN.
   Such deadlocks could be defeated with writing less than pipe's buffer size per round, and always reading everything from STDOUT/STDERR before starting next round.
   Therefore, making poll() always reply "writable" shouldn't cause any new issues or block any future solutions.

4. Increase the size of the pipe's buffer
   The difference between BytesInQueue and QuotaUsed is the size of pending reads. Therefore, if buffer is bigger than size of reads, poll() won't hang so easily. However, I found that for example strbuf_read() will get more and more hungry as it reads large inputs, eventually surpassing any reasonable pipe buffer size.

Chosen solution

Make poll() always reply "writable" for write end of the pipe.
Hopefully one day someone will find a way to implement it properly.

Reproduction

printf "%8388608s" X >large_file.txt 
git stash push --include-untracked -- large_file.txt

I have decided not to include this as test to avoid slowing down the test suite.
I don't expect the specific problem to come back, and chances are that git stash push will be reworked to avoid sending the entire patch via STDIN.

---

MinGW's poll() emulation also plays a role with Git 2.38 (Q3 2022), fix deadlocks between main Git process and subprocess spawned via the pipe_command() API, that can kill "git add -p"^(man) that was reimplemented in C recently.

See commit 24b56ae (17 Aug 2022) by René Scharfe (rscharfe).
See commit 716c1f6, commit c6d3cce, commit 14eab81, commit ec4f39b, commit 10f7433 (17 Aug 2022) by Jeff King (peff).
^{(Merged by Junio C Hamano -- gitster -- in commit a103ad6, 25 Aug 2022)}

pipe_command(): avoid xwrite() for writing to pipe

^{Helped-by: René Scharfe
Signed-off-by: Jeff King}

If xwrite() sees an EAGAIN response, it will loop forever until the write succeeds (or encounters a real error).
This is due to ef1cf01 ("xwrite: poll on non-blocking FDs", 2016-06-26, Git v2.10.0-rc0 -- merge listed in batch #6), with the idea that we won't be surprised by a descriptor unexpectedly set as non-blocking.

But that will make things awkward when we do want a non-blocking descriptor, and a future patch will switch pipe_command() to using one.
In that case, looping on EAGAIN is bad, because the process on the other end of the pipe may be waiting on us before doing another read() on the pipe, which would mean we deadlock.

In practice we're not supposed to ever see EAGAIN here, since poll() will have just told us the descriptor is ready for writing.
But our Windows emulation of poll() will always return "ready" for writing to a pipe descriptor! This is due to 94f4d01 ("mingw: workaround for hangs when sending STDIN", 2020-02-17, Git v2.26.0-rc1 -- merge).

Our best bet in that case is to keep handling other descriptors, as any read() we do may allow the child command to make forward progress (i.e., its write() finishes, and then it read()s from its stdin, freeing up space in the pipe buffer).
This means we might busy-loop between poll() and write() on Windows if the child command is slow to read our input, but it's much better than the alternative of deadlocking.

---

With Git 2.44 (Q1 2024), batch 14, the code that writes to pipes on Windows does not block anymore.

See commit 19ed0df (30 Jan 2024) by Johannes Schindelin (dscho).
^{(Merged by Junio C Hamano -- gitster -- in commit 1f9d274, 06 Feb 2024)}

win32: special-case ENOSPC when writing to a pipe

^{Signed-off-by: Johannes Schindelin}

Since c6d3cce (pipe_command(): handle ENOSPC when writing to a pipe, 2022-08-17, Git v2.38.0-rc0 -- merge listed in batch #15) (pipe_command(): handle ENOSPC when writing to a pipe, 2022-08-17), one write() call that results in an errno value ENOSPC (which typically indicates out of disk space, which makes little sense in the context of a pipe) is treated the same as EAGAIN.

However, contrary to expectations, as diagnosed in https://github.com/python/cpython/issues/101881#issuecomment-1428667015, when writing to a non-blocking pipe on Windows, an errno value of ENOSPC means something else: the write fails.
Completely.
Because more data was provided than the internal pipe buffer can handle.
Somewhat surprising, considering that write() is allowed to write less than the specified amount, e.g. by writing only as much as fits in that buffer.
But it does not, it writes no byte at all in that instance.

Let's handle this by manually detecting when an ENOSPC indicates that a pipe's buffer is smaller than what needs to be written, and re-try using the pipe's buffer size as size parameter.

It would be plausible to try writing the entire buffer in a loop, feeding pipe buffer-sized chunks, but experiments show that trying to write more than one buffer-sized chunk right after that will immediately fail because the buffer is unlikely to be drained as fast as write() could write again.
And the whole point of a non-blocking pipe is to be non-blocking.

Which means that the logic that determines the pipe's buffer size unfortunately has to be run potentially many times when writing large amounts of data to a non-blocking pipe, as there is no elegant way to cache that information between write() calls.
It is the best we can do, though, so it has to be good enough.

The diff is slightly chatty because it extends an already-existing conditional that special-cases a different errno value for pipes, and because this patch needs to account for the fact that _get_osfhandle() potentially overwrites errno.