Direct I/O in Python with O_DIRECT

Question

I am new to python and for some project, I have to use python for some I/O. I started with a simple python code that uses open() method from os module and the code worked fine.

#!/usr/bin/python

import os, sys, mmap

# Open a file
fd = os.open( "foo.txt", os.O_RDWR|os.O_CREAT)
s = ' ' * 1024
# Write one string
os.write(fd, s)
# Close opened file
os.close( fd )

Now, since I want to use direct I/O, I want to use O_DIRECT flag with open() to bypass all (or some) effects of cache. Unfortunately, if I open a file with O_DIRECT and subsequently, try to write to that file, there is a strange error message:

OSError: [Errno 22] Invalid argument

Now, as suggested in this article: I have to use mmap to map portion of file into memory. Mmap works with granularity of one memory page – 4kb that is. So every memory mapped buffer is naturally memory aligned to 4kb, thus to 512 byte boundary too. The code with mmap is:

#!/usr/bin/python

import os, sys, mmap

# Open a file
fd = os.open( "foo.txt", os.O_RDWR|os.O_CREAT|os.O_DIRECT)
m = mmap.mmap(-1, 1024)
s = ' ' * 1024
m.write(s)
# Write one string
os.write(fd, m)
# Close opened file
os.close( fd )

My questions are:

1. In line m = mmap.mmap(-1, 1024), I understand specifying -1 as file descriptor allocates RAM. What is the significance of 1024?
2. What s = ' ' * 1024 does? Again, what is the significance of 1024 here?

Answer 1

In line m = mmap.mmap(-1, 1024), I understand specifying -1 as file descriptor allocates RAM. What is the significance of 1024?

Via the docs:

class mmap.mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, access=ACCESS_DEFAULT[, offset])

It's the length of the mapped region. Also, you're correct in the significance of -1: "To map anonymous memory, -1 should be passed as the fileno along with the length."

What s = ' ' * 1024 does? Again, what is the significance of 1024 here?

Creates a string of 1024 spaces. 1024 is the length of the mapped region; the data you're writing. (It should, though, preferably be b' ' * 1024 to create a bytestring of 1024 0x20 bytes. An arbitrary text string might not get encoded into exactly 1024 bytes.)