pexpect can't pass input over 1024 chars?

Question

I'm currently passing some input to a process with pexpect with the following code:

p = pexpect.spawn('cat', timeout=5.0 )
p.maxread = 5000
p.setecho(False) # prevent the process from echoing stdin back to us
INPUT_LEN = 1024
p.sendline('a'*INPUT_LEN)
print p.readline() # pexpect.TIMEOUT: Timeout exceeded in read_nonblocking().

When INPUT_LEN < 1024, everything works fine, but for >= 1024 characters, the process does not receive the full input, causing raising a "pexpect.TIMEOUT" error on p.readline().

I've tried splitting my input into pieces smaller than 1024 characters, but this has the same issue:

p = pexpect.spawn('cat', timeout=5.0 )
p.maxread = 5000
p.setecho(False)
INPUT_LEN = 1024
p.send('a'*1000)
p.sendline('a'*(INPUT_LEN-1000))
print p.readline() # pexpect.TIMEOUT: Timeout exceeded in read_nonblocking().

Does anyone know how to make pexpect work with inputs over 1024 characters? I tried looking at the source, but it just seems to be calling os.write(...).

(As a side note, I've noticed the same truncation error occurs when I run "cat" from a shell and attempt to paste in >=1024 characters with "Cmd+V". However, everything works fine if I run "pbpaste | cat" .)

Thanks!

Update: The call to "os.write()" returns 1025, indicating a successful write, but os.read() returns "\x07" (the single character BEL), and then hangs on the next call, resulting in the timeout.

Dividing the os.write() call into two sub-1024 byte write()s, separated by a call to os.fsync(), does not change anything.

Answer 1

In my case (Debian Linux) the limit (4096 chars) was related to the canonical processing input mode of the terminal. There are some comments about that in the pexpect documentation.

I solved my problem by turning off canon mode before sending my data:

p.sendline('stty -icanon')
p.sendline('a'*5000)

Answer 2

Your problem seems to be MacOS related, take a look at MacOSX 10.6.7 cuts off stdin at 1024 chars.

It basically says that 1024 is your tty buffer limit.

I'm not an expert on Mac OS, but maybe others can give you more informations about this.