What share mode is used when files are opened using open()

Question

I can see nothing in the open() function's parameters that allows specification of how the file will be shared. I suspect therefore that the file will be shared as permissively as possible. Specifically:

1. When the file is opened for reading, its sharing mode will allow subsequent open operations to open the file for reading, but not for writing.
2. When the file is opened for writing, its sharing mode will deny subsequent open operations to open the file for reading or writing.

That would seem to me to be the most logical implementation. Are my assumptions correct?

Update: Martijn Pieters states that the answer is OS dependent. So, for the sake of this question, my target OS is Windows.

Answer 1

Python uses _wfopen() (Python 2 open() function) or _wopen() (Python 3 and io.open()) internally when opening files, neither of which allow for specifying any sharing flags. Sharing is thus set to a default, and what that default is does not appear to be documented.

If you want to set the sharing mode yourself, you'll have to use the msvcrt.open_osfhandle() if you want to open files specifying a sharing mode.

There is a patch in the Python issue tracker that implements a sharing module, illustrating how to do this. Just the opener from that patch, somewhat simplified, is:

import os
import msvcrt
import _winapi

CREATE_NEW                  = 1
CREATE_ALWAYS               = 2
OPEN_EXISTING               = 3
OPEN_ALWAYS                 = 4
TRUNCATE_EXISTING           = 5
FILE_SHARE_READ             = 0x00000001
FILE_SHARE_WRITE            = 0x00000002
FILE_SHARE_DELETE           = 0x00000004
FILE_SHARE_VALID_FLAGS      = 0x00000007
FILE_ATTRIBUTE_READONLY     = 0x00000001
FILE_ATTRIBUTE_NORMAL       = 0x00000080
FILE_ATTRIBUTE_TEMPORARY    = 0x00000100
FILE_FLAG_DELETE_ON_CLOSE   = 0x04000000
FILE_FLAG_SEQUENTIAL_SCAN   = 0x08000000
FILE_FLAG_RANDOM_ACCESS     = 0x10000000
GENERIC_READ                = 0x80000000
GENERIC_WRITE               = 0x40000000
DELETE                      = 0x00010000
NULL                        = 0

_ACCESS_MASK = os.O_RDONLY | os.O_WRONLY | os.O_RDWR
_ACCESS_MAP  = {os.O_RDONLY : GENERIC_READ,
                os.O_WRONLY : GENERIC_WRITE,
                os.O_RDWR   : GENERIC_READ | GENERIC_WRITE}

_CREATE_MASK = os.O_CREAT | os.O_EXCL | os.O_TRUNC
_CREATE_MAP  = {0                                   : OPEN_EXISTING,
                os.O_EXCL                           : OPEN_EXISTING,
                os.O_CREAT                          : OPEN_ALWAYS,
                os.O_CREAT | os.O_EXCL              : CREATE_NEW,
                os.O_CREAT | os.O_TRUNC | os.O_EXCL : CREATE_NEW,
                os.O_TRUNC                          : TRUNCATE_EXISTING,
                os.O_TRUNC | os.O_EXCL              : TRUNCATE_EXISTING,
                os.O_CREAT | os.O_TRUNC             : CREATE_ALWAYS}


def os_open(file, flags, mode=0o777,
            *, share_flags=FILE_SHARE_VALID_FLAGS):
    '''
    Replacement for os.open() allowing moving or unlinking before closing
    '''
    if not isinstance(flags, int) and mode >= 0:
        raise ValueError('bad flags: %r' % flags)

    if not isinstance(mode, int) and mode >= 0:
        raise ValueError('bad mode: %r' % mode)

    if share_flags & ~FILE_SHARE_VALID_FLAGS:
        raise ValueError('bad share_flags: %r' % share_flags)

    access_flags = _ACCESS_MAP[flags & _ACCESS_MASK]
    create_flags = _CREATE_MAP[flags & _CREATE_MASK]
    attrib_flags = FILE_ATTRIBUTE_NORMAL

    if flags & os.O_CREAT and mode & ~0o444 == 0:
        attrib_flags = FILE_ATTRIBUTE_READONLY

    if flags & os.O_TEMPORARY:
        share_flags |= FILE_SHARE_DELETE
        attrib_flags |= FILE_FLAG_DELETE_ON_CLOSE
        access_flags |= DELETE

    if flags & os.O_SHORT_LIVED:
        attrib_flags |= FILE_ATTRIBUTE_TEMPORARY

    if flags & os.O_SEQUENTIAL:
        attrib_flags |= FILE_FLAG_SEQUENTIAL_SCAN

    if flags & os.O_RANDOM:
        attrib_flags |= FILE_FLAG_RANDOM_ACCESS

    h = _winapi.CreateFile(file, access_flags, share_flags, NULL,
                           create_flags, attrib_flags, NULL)
    return msvcrt.open_osfhandle(h, flags | os.O_NOINHERIT)

Answer 2

When the file is opened for writing, its sharing mode will deny subsequent open operations to open the file for reading or writing.

False. No locks are acquired. You can try from two different cmd windows:

first

python -c "import time; f = open('tst','wb'); f.write('1'); time.sleep(3); f.write('333'); f.close"

second, while first is running

python -c "with open('tst','wb') as f: f.write('22')"

results in 1333 in tst file. Flushing writes with f.flush() just after first write in first script results in 2333. Missed data is replaced with \x00 charackters, you can check it for example replacing first f.write('1') with f.write('1'*10**6)