What happens in BASH when you do Ctrl-C (hint, it's not simply sending a SIGINT)

Question

A little background first - When I do apt-get install downloads from my company internet it provides a high burst of speed (400-500KB/s) for the first 10 seconds or so before dropping down to a tenth of that (40-50KB/s), and then after a few minutes to a truly miserable (4-5KB/s). This makes me think that the sysadmin has implemented some sort of a network throttling scheme.

Now I know that the network is not simply erratic, because if I start an apt-get install foo, Ctrl-C it after 10 seconds and immediately run apt-get install foo again (by doing an up arrow and enter to use bash history), and then keep repeating this process for a few minutes till all packages are downloaded, I can download even large packages very fast. In particular, even after aborting a download with Ctrl-C, apt-get seems to be able to resume the download in the next invocation.

Of course, staring at the screen doing Ctrl-C Up Enter every 10 seconds gets really boring real fast, so I wrote a shell script -

#!/bin/sh
for i in `seq 1 100` ; do
    sudo apt-get install foo -y &
    sleep 10
    sudo kill -2 $!
done

This seems to work. It spawns apt-get, runs it for 10 seconds and then kills (by sending a SIGINT) it and starts it up again. However, it doesn't really work because now apt-get does not resume downloads on subsequent invocations!

An an experiment I ran sudo apt-get install foo from one terminal and then ran kill -2 <PID of apt-get> from another terminal. And even in that case, when I restart apt-get, it does not resume the download.

So clearly a Ctrl-C is not equivalent to SIGINT. And something else is happening when I do Ctrl-C manually which gives apt-get a chance to save the state of the download. The question is - what is it?

Edit

These are the suggestions I have received so far, but no cigars. The mystery deepens! -

1. On sudo kill -2 $! the signal might be going to sudo instead of apt-get. This is not the reason because as mentioned above I also tried sending SIGINT specifically to apt-get's PID and even that prevented apt-get from saving its state.

2. Sudo catches the signal and sends some other signal to apt-get. I tried sending apt-get all the signals I can think of! It still does not resume the download for any of them. It only resumes downloads when I do Ctrl-C to kill it.

3. Apt-get handles SIGINT differently if it is from a script instead of an interactive shell. Again, the "experiment" above proves that this is not true.

Answer 1

Okay mystery solved! Thanks to the helpful folks over at the Indian Linux Users Group.

The answer here is two-fold -

Firstly, apt-get invokes another program called http for downloading data.

[~] ➔ file /usr/lib/apt/methods/http

/usr/lib/apt/methods/http: ELF 32-bit LSB executable, Intel 80386, version 1
(SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15,
stripped

Note that that's an executable, not even a script, probably to support downloading files during system installation when none of perl/python/ruby etc. are yet available.

Secondly, when you press Ctrl-C after running apt-get, the SIGINT gets sent to http, and not to apt-get. When http receives the SIGINT, it saves the download state before shutting down.

Here's the updated script that works perfectly -

#!/bin/sh
for i in `seq 1 100` ; do
    sudo apt-get install foo -y &
    sleep 10
    sudo kill -2 `ps -ae | grep " http" | awk '{print $1}'`
done

Answer 2

Hint, it's not simply sending a SIGINT).

Yes it is just sending a SIGINT :-) Throttling is what is happening, you've got that right. Here's what I suspect is happening:

• Something is limiting the bandwidth of connections. To track connections it's also including the source port (which is a bad idea IMO) among other parameters

• When you kill apt-get and restart it, it will naturally get a new TCP source port and the evil entity throttling you will think, "Oh, it's a new connection", which it actually is

So how do you speed things up ? Well the real solution would be to use multiple parallel downloads. I never used it myself, but I have heard of a tool called "apt-fast" (actually a bash script itself) which does something like this.

EDIT

After reading the question again I suspect the signal is not sent to apt-get.

sudo apt-get install foo -y &
sudo kill -2 $! # sends signal to sudo, which sends whatever it wants to `apt-get`

So I believe sudo catches the signal and sends something else (sigterm? sighup?) to apt-get.