Keeping 80 chars margin for long with statement?

Question

What is the pythonic way to PEP-8-ify such as with statement:

with tempfile.NamedTemporaryFile(prefix='malt_input.conll.', dir=self.working_dir, mode='w', delete=False) as input_file, tempfile.NamedTemporaryFile(prefix='malt_output.conll.', dir=self.working_dir, mode='w', delete=False) as output_file:
    pass

I could do this but since the tempfile i/o is not with the with statement, does it automatically close after the with? Is that pythonic?:

intemp = tempfile.NamedTemporaryFile(prefix='malt_input.conll.', dir=self.working_dir, mode='w', delete=False)

outtemp = tempfile.NamedTemporaryFile(prefix='malt_output.conll.', dir=self.working_dir, mode='w', delete=False)

with intemp as input_file,  outtemp as output_file:
    pass

Or I could use slashes:

with tempfile.NamedTemporaryFile(prefix='malt_input.conll.',
dir=self.working_dir, mode='w', delete=False) as input_file, \
tempfile.NamedTemporaryFile(prefix='malt_output.conll.', 
dir=self.working_dir, mode='w', delete=False) as output_file:
    pass

But is that PEP8 comply-able? Is that pythonic?

Answer 1

PEP 0008 does say it's ok to use backslashes for a with line.

Backslashes may still be appropriate at times. For example, long, multiple with -statements cannot use implicit continuation, so backslashes are acceptable.

Though it recommends they be indented, so your line should be like this:

with tempfile.NamedTemporaryFile(prefix='malt_input.conll.',
      dir=self.working_dir, mode='w', delete=False) as input_file, \
      tempfile.NamedTemporaryFile(prefix='malt_output.conll.', 
      dir=self.working_dir, mode='w', delete=False) as output_file:
    pass

It's recommended you keep it indented a distinctly different amount of space to the following block of code, so it's clearer where the with line ends and the block begins.

However you don't actually need to use slashes if you enclose the parameters in brackets

with (tempfile.NamedTemporaryFile(prefix='malt_input.conll.',
      dir=self.working_dir, mode='w', delete=False)) as input_file, (
      tempfile.NamedTemporaryFile(prefix='malt_output.conll.', 
      dir=self.working_dir, mode='w', delete=False)) as output_file:
    pass

That does depend on your exact sentence arrangement of course and your mileage may vary on whether having a bracket at the end of a line is any better than a backslash.

Answer 2

PEP 8 isn’t too clear about that. It only mentions the with statement once for an exception when backslashe continuations are okay:

Backslashes may still be appropriate at times. For example, long, multiple with -statements cannot use implicit continuation, so backslashes are acceptable:

with open('/path/to/some/file/you/want/to/read') as file_1, \
     open('/path/to/some/file/being/written', 'w') as file_2:
    file_2.write(file_1.read())

A way to overcome the problem would be indeed to first make the function call that returns the context manager and then just use that directly, as you already suggested in your question:

file_1 = open('/path/to/some/file/you/want/to/read')
file_2 = open('/path/to/some/file/being/written', 'w')

with file_1, file_2:
    file_2.write(file_1.read())

This works perfectly (because the with statement will just call the context manager’s methods, regardless of where it came from), and also closes the handles correctly.

However, this is explicitely disallowed in PEP 8:

Context managers should be invoked through separate functions or methods whenever they do something other than acquire and release resources. For example:

Yes:

with conn.begin_transaction():
   do_stuff_in_transaction(conn)

No:

with conn:
   do_stuff_in_transaction(conn)

The latter example doesn't provide any information to indicate that the __enter__ and __exit__ methods are doing something other than closing the connection after a transaction. Being explicit is important in this case.

So in the end, there seems to be no real solution that’s allowed by PEP 8. And at that point, I would argue, that it’s perfectly fine to “violate it” and go against it: It’s just a style guide.

While it suggests many good rules, there are also many situations, in which it simply does not make much sense to follow them that strictly. I would argue that your example using slashes is hardly readable, and you could probably read it a lot better if you allowed longer lines and just broke the line once per context manager:

with tempfile.NamedTemporaryFile(prefix='malt_input.conll.', dir=self.working_dir, mode='w', delete=False) as input_file, \
     tempfile.NamedTemporaryFile(prefix='malt_output.conll.', dir=self.working_dir, mode='w', delete=False) as output_file:
    pass

Yes, you may need to scroll for this, but at least you can very clearly see what’s going on.

Another alternative would be to actually initialize the objects directly before the with:

malt_input = tempfile.NamedTemporaryFile(prefix='malt_input.conll.', dir=self.working_dir, mode='w', delete=False)
malt_output = tempfile.NamedTemporaryFile(prefix='malt_output.conll.', dir=self.working_dir, mode='w', delete=False)
with malt_input as input_file, malt_output as output_file:
    pass

Since you do it directly before the with, it should be an acceptable exception from the PEP 8 rule here. After all, it improves readability, and that is what counts.

Btw. note that a context manager may not return self on __enter__, so you should still use the as syntax to assign the return value of the context manager to a variable.

---

Finally, one other option, which would work for your particular situation, would be to wrap your call into a separate function:

def namedTemp(prefix):
    return tempfile.NamedTemporaryFile(prefix=prefix,
        dir=self.working_dir, mode='w', delete=False)

with namedTemp('malt_input.conll.') as input_file, \
     namedTemp('malt_output.conll.') as output_file:
    pass

So basically, abstract everything away, so the with statement becomes readable again.

Answer 3

PEP-8 actually gives examples of two similar situations:

Example 1 (looks most applicable, since it's using with statements):

with open('/path/to/some/file/you/want/to/read') as file_1, \
     open('/path/to/some/file/being/written', 'w') as file_2:
    file_2.write(file_1.read())

Example 2:

class Rectangle(Blob):

    def __init__(self, width, height,
                 color='black', emphasis=None, highlight=0):

It looks like slashes are a permissable way of handling this(but ultimately unnecessary, if you wrap your file params in parentheses).