setrlimit fails with Operation not permitted when run under valgrind

Question

I may be missing something or may be not but the setrlimit function is failing consistently when run under valgrind

int main()
{
        const struct rlimit file_limits = { .rlim_cur = 65536, .rlim_max = 65536 };
        if ( setrlimit( RLIMIT_NOFILE, &file_limits ) )
        {
                printf(" Failed  %d", errno );
                perror(" More " );
        }
        else
        {
                printf(" Success ");
        }
        printf("\n");
        return 0;
}

Here are the sample runs

Normal run

sh-4.2# ulimit -H -n
800000
sh-4.2# ulimit -S -n
500000
sh-4.2# ./rlimit
 Success
sh-4.2#

under valgrind

sh-4.2#
sh-4.2# valgrind ./rlimit
==28974== Memcheck, a memory error detector
==28974== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==28974== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==28974== Command: ./rlimit
==28974==
 More : Operation not permitted
 Failed  1
==28974==
==28974== HEAP SUMMARY:
==28974==     in use at exit: 0 bytes in 0 blocks
==28974==   total heap usage: 1 allocs, 1 frees, 568 bytes allocated
==28974==
==28974== All heap blocks were freed -- no leaks are possible
==28974==
==28974== For counts of detected and suppressed errors, rerun with: -v
==28974== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
sh-4.2#

Any hints on this would be great. NOTE: this is on CentOS release 7.4 (Final).

Edits #1

With minimal working code:

int main()
{
        const struct rlimit file_limits = { .rlim_cur = 65536, .rlim_max = 65536 };
        setrlimit( RLIMIT_NOFILE, &file_limits ) ;
        perror(" wrong ?? " );
        printf("\n");
        return 0;
}

corresponding outputs:

[root@localhost kk]# valgrind ./rlimit
==29179== Memcheck, a memory error detector
==29179== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==29179== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==29179== Command: ./rlimit
==29179==
 wrong ?? : Operation not permitted

==29179==
==29179== HEAP SUMMARY:
==29179==     in use at exit: 0 bytes in 0 blocks
==29179==   total heap usage: 1 allocs, 1 frees, 568 bytes allocated
==29179==
==29179== All heap blocks were freed -- no leaks are possible
==29179==
==29179== For counts of detected and suppressed errors, rerun with: -v
==29179== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
[root@localhost kk]# ./rlimit
 wrong ?? : Success

this wrong ?? : Operation not permitted is driving me crazy :(

Edit #2

So based on some suggestion I tried to retrieve existing limits and see if there is something wrong with that, turns out the behavior beyond my comprehension

int main()
{
        const struct rlimit file_limits = { .rlim_cur = 65536, .rlim_max = 65536 };
        struct rlimit limit;
        getrlimit(RLIMIT_NOFILE,&limit);
        printf("%d \n",limit.rlim_max);

        setrlimit( RLIMIT_NOFILE, &file_limits ) ;
        perror(" wrong ?? " );
        printf("\n");

        getrlimit(RLIMIT_NOFILE,&limit);
        printf("%d \n",limit.rlim_max);
        return 0;
}

1st run, the limit is set to 65590 and the executable is able to alter the limits to 65536, viz is expected

[root@localhost kk]# ulimit -n
65590
[root@localhost kk]# ./rlimit
65590
 wrong ?? : Success

65536
[root@localhost kk]#

2nd run, under valgrind

[root@localhost kk]# ulimit -n
65590
[root@localhost kk]# valgrind ./rlimit
==17595== Memcheck, a memory error detector
==17595== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==17595== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==17595== Command: ./rlimit
==17595==
65578
 wrong ?? : Operation not permitted

65578
==17595==
==17595== HEAP SUMMARY:
==17595==     in use at exit: 0 bytes in 0 blocks
==17595==   total heap usage: 1 allocs, 1 frees, 568 bytes allocated
==17595==
==17595== All heap blocks were freed -- no leaks are possible
==17595==
==17595== For counts of detected and suppressed errors, rerun with: -v
==17595== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
[root@localhost kk]#

I'm probably inclined to say that may be valgrind reserves a few files descriptors for its use based on discussion here let's say in this case it 12?? Thus setting the limit to 65578?? Which is still more that what the program is trying to reserve 65536.

Any more suggestions to follow?

Answer 1

Of the allowed range of fd 0 .. hard_limit, valgrind reserves a set of fd for its own purposes at the end of the range i.e. the range hard_limit - 11 .. hard_limit, and then simulates a new hard limit which is hard_limit - 12.

It then forbids the guest application to change this (simulated) hard limit. Here is the piece of code that handles the setrlimit simulation:

 if (((struct vki_rlimit *)(Addr)ARG2)->rlim_cur > VG_(fd_hard_limit) ||
      ((struct vki_rlimit *)(Addr)ARG2)->rlim_max != VG_(fd_hard_limit)) {
     SET_STATUS_Failure( VKI_EPERM );
  }

As you can see, if the provided rlim_max is different of the simulated VG_(fd_hard_limit), valgrind makes the setrlimit fails. When accepted, valgrind will change the simulated soft limit.

I am not too sure to understand why the above code does not accept a lower hard limit and set it into the simulated VG_(fd_hard_limit). I think it is because this (not changeable) VG_(fd_hard_limit) is used by valgrind to find the difference between the valgrind reserved fds and the guest fds.

To bypass the problem, you should get the limit, and then only change the soft limit below the hard limit as changing the hard limit will be rejected by the valgrind setrlimit simulation.