Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does Linux's splice(2) work when splicing from a TCP socket?

Tags:

c

linux

pipe

I've been writing a little program for fun that transfers files over TCP in C on Linux. The program reads a file from a socket and writes it to file (or vice versa). I originally used read/write and the program worked correctly, but then I learned about splice and wanted to give it a try.

The code I wrote with splice works perfectly when reading from stdin (redirected file) and writing to the TCP socket, but fails immediately with splice setting errno to EINVAL when reading from socket and writing to stdout. The man page states that EINVAL is set when neither descriptor is a pipe (not the case), an offset is passed for a stream that can't seek (no offsets passed), or the filesystem doesn't support splicing, which leads me to my question: does this mean that TCP can splice from a pipe, but not to?

I'm including the code below (minus error handling code) in the hopes that I've just done something wrong. It's based heavily on the Wikipedia example for splice.

static void splice_all(int from, int to, long long bytes)
{
    long long bytes_remaining;
    long result;

    bytes_remaining = bytes;
    while (bytes_remaining > 0) {
        result = splice(
            from, NULL,
            to, NULL,
            bytes_remaining,
            SPLICE_F_MOVE | SPLICE_F_MORE
        );

        if (result == -1)
            die("splice_all: splice");

        bytes_remaining -= result;
    }
}

static void transfer(int from, int to, long long bytes)
{
    int result;
    int pipes[2];

    result = pipe(pipes);

    if (result == -1)
        die("transfer: pipe");

    splice_all(from, pipes[1], bytes);
    splice_all(pipes[0], to, bytes);

    close(from);
    close(pipes[1]);
    close(pipes[0]);
    close(to);
}

On a side note, I think that the above will block on the first splice_all when the file is large enough due to the pipe filling up(?), so I also have a version of the code that forks to read and write from the pipe at the same time, but it has the same error as this version and is harder to read.

EDIT: My kernel version is 2.6.22.18-co-0.7.3 (running coLinux on XP.)

like image 299
Neil Williams Avatar asked May 19 '09 20:05

Neil Williams


2 Answers

What kernel version is this? Linux has had support for splicing from a TCP socket since 2.6.25 (commit 9c55e01c0), so if you're using an earlier version, you're out of luck.

like image 169
bdonlan Avatar answered Sep 16 '22 23:09

bdonlan


You need to splice_all from pipes[0] to to every time you do a single splice from from to pipes[1] (the splice_all is for the amount of bytes just read by the last single splice) . Reason: pipes represents a finite kernel memory buffer. So if bytes is more than that, you'll block forever in your splice_all(from, pipes[1], bytes).

like image 36
Jonathan Graehl Avatar answered Sep 19 '22 23:09

Jonathan Graehl