I'm new to 64-bits architecture. Could you tell me what's MAX file size supported by file mapping in 64 bits linux machine. I want to open more than 20GB files by file mapping, is it available?
I write a sample code. But it causes Bus Error when I get the value of the pointer in GBSIZE offset:
unsigned char* pCur = pBegin + GBSIZE;
//pBegin is the pointer returned by mmap
printf("%c",*pCur);
BTW, printf("%c",*pBegin );
works fine. and my address sizes : 38 bits physical, 48 bits virtual
Here is the full code:
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
//#define FILEPATH "smallfile"
#define FILEPATH "bigfile"
#define GBSIZE (1024L*1024L*1024L)
#define TBSIZE (1024L*GBSIZE)
#define NUMSIZE (20L * GBSIZE)
//#define NUMSIZE (10)
#define FILESIZE (NUMINTS * sizeof(int))
int main(int argc, char *argv[])
{
int i;
int fd;
unsigned char *pBegin;
fd = open(FILEPATH, O_RDONLY);
if (fd == -1) {
perror("Error opening file for reading");
exit(EXIT_FAILURE);
}
pBegin = mmap(0, NUMSIZE, PROT_READ, MAP_SHARED, fd, 0);
if (pBegin == MAP_FAILED) {
close(fd);
perror("Error mmapping the file");
exit(EXIT_FAILURE);
}
/** ERROR happens here!!! **/
unsigned char* pCur = pBegin + GBSIZE;
printf("%c",*pCur);
if (munmap(pBegin, NUMSIZE) == -1) {
perror("Error un-mmapping the file");
}
close(fd);
return 0;
}
Accessing memory mapped files is faster than using direct read and write operations for two reasons. Firstly, a system call is orders of magnitude slower than a simple change to a program's local memory.
A memory-mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory.
Yes. If one thread changes part of the data in the mapping, then all other threads immediately see that change.
Although pointers are 64-bit wide, most processors do not actually support virtual addresses using the full 64 bits. To see what size virtual addresses your processor supports, look in /proc/cpuinfo
(48 bits is typical).
grep "address sizes" /proc/cpuinfo
Additionally, half of the virtual address space is used by the kernel and not available to userspace - leaving 47 bits in the current Linux implementation.
However, even taking this into account, you will still have plenty of room for a 20GB file. 47 bits in theory means a virtual address space of 128TB.
From the mmap(2)
man page:
void *mmap(void *addr, size_t length, int prot, int flags,
int fd, off_t offset);
length
is a size_t
, which on 64-bit machines is 64 bits in length. Therefore yes, you can theoretically map a 20GB file.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With