C low level read write functions 64-bit equivalent?

Question

I have some code that uses the low level i/o read and write system calls, as described on page 170 of the C programming language book Kernighan and Ritchie. The function prototypes are this

int n_read = read  ( int fd, char *buf, int n )
int n_read = write ( int fd, char *buf, int n )

now the two .c file that uses these read and write are called by a larger fortran based program to read and write lots of data.

the C code is simply this, with no #include of any kind, having the underscore after the function name and passing by reference:

int read_ ( int *descriptor, char *buffer, int *nbyte )
{
   return ( read( *descriptor, buffer, *nbyte ) );
}

int write_ ( int *descriptor, char *buffer, int *nbyte )
{
   return ( write( *descriptor, buffer, *nbyte ) );
}

and the larger fortran based program will do something like this

INTEGER nbyte
COMPLEX*16 matrix(*)
INTEGER READ, WRITE
EXTERNAL READ, WRITE

status = READ( fd, matrix, nbyte )
if ( status .eq. -1 ) then
   CALL ERROR('C call read failure')
   stop
endif

As you may have already guessed, this works fine for nbyte values less than 2^31. I have a need to read more than 2 GB of data, so i need nbyte to be a long integer and INTEGER*8 in fortran.

Is there an equivalent read64 and write64, like there is an lseek64 provided by unistd.h and features.h ?

what is the best way to recode this? should i use fread and fwrite ? is the int fd from the low level write the same as FILE *stream from fread() ?

my requirement is being able to pass a long integer of 8 bytes to allow for values up to 100 to 500 gigabytes or an integer having 12 digits, which is all for the value of nbyte

Am i gaining anything or losing out by currently using read and write which is identified as a "system call" ? What does this mean?

Answer 1

Edit: You can't, at least not on Linux. read will never transfer more than what a 32-bit integer can hold.

From the manpages of Linux on read:

On Linux, read() (and similar system calls) will transfer at most 0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both 32-bit and 64-bit systems.)

This is not a contraint of POSIX, it's allowed by POSIX, but in the end it's implementation defined how read behaves. As Andrew Hanle reports, reading a 32GB file works just fine on Solaris. In this case, my old answer is still valid.

Old Answer:

read can work with 64-bit files just fine. It's defined in <unistd.h> as the following:-

ssize_t read(int fd, void *buf, size_t count);

You would have to adjust your routines to work with size_t instead of int, to properly support big files.

You should check SSIZE_MAX (the maximum value supported for count), before using read with a big file, and abort if it's to small (or split into smaller chunks). SSIZE_MAX is an implementation defined value.