Python, calling mmap with too-large address causes overflow exception

Question

I am converting some C code that accesses a device driver via mmap. I thought I could easily do very similar things in Python. However I've run into this issue. The address that needs to be mmap'ed is provided by the PCI configuration, so I have no control over this address. In this case, the address found is 3451912192. The address field is of course an unsigned integer, and is 32-bits.

This address just happens to be in the range where it fits in an unsigned integer but not a signed integer. When I call mmap, I get the following exception:

OverflowError: long int too large to convert to int

So Python is artificially telling me the number doesn't fit because it thinks it's signed. Is it possible to resolve this somehow, or do I have to do the mmap call in C?

Note that converting the pointer to the equivalent negative value gives:

OverflowError: memory mapped offset must be positive

eryksun points out that the C interface actually specifies a signed value, so Python is correct from a semantic point of view but it won't let me circumvent this restriction which the C program is able to ignore.

Indeed, lspci reveals that the device has resource:

Memory at cdc00000 (32-bit, non-prefetchable) [size=1M]

and 0xcdc00000==3451912192. It would seem at the very least that the check for offset > 0 in mmap is perhaps not correct. Why check this? Let the operating system return an error code if it's not supported.

Answer 1

I had a similar problem once, I also wanted to mmap a bigger-then-4GB chunk of data on a 64bit machine into a 64bit-compiled Python.

If I remember correctly the main reason it it did not work out was, that most fundamental python-object struct contains an object size member, which is only 32bit. And to change this is so difficult, that you could say, you need to overhaul the whole language, and especially fix all compiled extensions.

I am not sure, but this might has been done with Python 3.0.