System calls, shell commands and programs

Question

I'm trying to understand how programs, shell commands and operating systems work. Please excuse my ignorance since I'm new to this.

When I use a C compiler on the command line, when I type cc [filename] , I suppose the shell uses a fork() system call to duplicate its process and then an exec() system call will load the cc compiler executable into the child process' core image. Then the child process containing the cc executable will do its thing while the parent process executing the shell waits, or not. Is it right?

What about shell commands like cp, mv, ls, and others. What are they? are they executable programs that will also be executed in a new child process forked by the shell? What about shell scripts? Suppose I create a simple shell script like this one (please ignore any errors I dont know how to do this yet) :

echo "Hello" 

date

echo

cc -o test file1.c file2.c file3.c

and then I execute this script using the command line. Will the command line fork() a new process and exec() this script in the new process? And then will this new process containing the script fork() other processes to execute date, cc compiler, etc ??

I hope this doesn't sound too confusing, because I am =/.

Answer 1

Yep! You've got the idea.

When I use a C compiler on the command line, when I type cc [filename] , I suppose the shell uses a fork() system call to duplicate its process and then an exec() system call will load the cc compiler executable into the child process' core image. Then the child process containing the cc executable will do its thing while the parent process executing the shell waits, or not. Is it right?

That's right. The parent process (the shell) calls wait() on the child's PID and waits for it to exit.

What about shell commands like cp, mv, ls, and others. What are they? are they executable programs that will also be executed in a new child process forked by the shell?

Same thing. These are binaries just like a compiler, and the shell does the same thing for them.

Now there are some commands which aren't external binaries known as "built-ins". These are commands that the shell recognizes itself and doesn't need to call an external binary for. Why?

• Some have special syntax, like if and while, and so by necessity must be built into the shell.
• Some, like cd and read, change the shell process's state, and so must be built-ins. (It's impossible for an external binary to change the shell's current directory since forked processes can only change their own PWD, not their parent's.)
• Others, like echo and printf, could be separate binaries, and just happen to be implemented by the shell.

Here's a full list of bash builtins I got from typing help:

 job_spec [&]                                                          history [-c] [-d offset] [n] or history -anrw [filename] or histor>
 (( expression ))                                                      if COMMANDS; then COMMANDS; [ elif COMMANDS; then COMMANDS; ]... [>
 . filename [arguments]                                                jobs [-lnprs] [jobspec ...] or jobs -x command [args]
 :                                                                     kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill>
 [ arg... ]                                                            let arg [arg ...]
 [[ expression ]]                                                      local [option] name[=value] ...
 alias [-p] [name[=value] ... ]                                        logout [n]
 bg [job_spec ...]                                                     mapfile [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callbac>
 bind [-lpvsPVS] [-m keymap] [-f filename] [-q name] [-u name] [-r k>  popd [-n] [+N | -N]
 break [n]                                                             printf [-v var] format [arguments]
 builtin [shell-builtin [arg ...]]                                     pushd [-n] [+N | -N | dir]
 caller [expr]                                                         pwd [-LP]
 case WORD in [PATTERN [| PATTERN]...) COMMANDS ;;]... esac            read [-ers] [-a array] [-d delim] [-i text] [-n nchars] [-N nchars>
 cd [-L|[-P [-e]]] [dir]                                               readarray [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callb>
 command [-pVv] command [arg ...]                                      readonly [-aAf] [name[=value] ...] or readonly -p
 compgen [-abcdefgjksuv] [-o option]  [-A action] [-G globpat] [-W w>  return [n]
 complete [-abcdefgjksuv] [-pr] [-DE] [-o option] [-A action] [-G gl>  select NAME [in WORDS ... ;] do COMMANDS; done
 compopt [-o|+o option] [-DE] [name ...]                               set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
 continue [n]                                                          shift [n]
 coproc [NAME] command [redirections]                                  shopt [-pqsu] [-o] [optname ...]
 declare [-aAfFgilrtux] [-p] [name[=value] ...]                        source filename [arguments]
 dirs [-clpv] [+N] [-N]                                                suspend [-f]
 disown [-h] [-ar] [jobspec ...]                                       test [expr]
 echo [-neE] [arg ...]                                                 time [-p] pipeline
 enable [-a] [-dnps] [-f filename] [name ...]                          times
 eval [arg ...]                                                        trap [-lp] [[arg] signal_spec ...]
 exec [-cl] [-a name] [command [arguments ...]] [redirection ...]      true
 exit [n]                                                              type [-afptP] name [name ...]
 export [-fn] [name[=value] ...] or export -p                          typeset [-aAfFgilrtux] [-p] name[=value] ...
 false                                                                 ulimit [-SHacdefilmnpqrstuvx] [limit]
 fc [-e ename] [-lnr] [first] [last] or fc -s [pat=rep] [command]      umask [-p] [-S] [mode]
 fg [job_spec]                                                         unalias [-a] name [name ...]
 for NAME [in WORDS ... ] ; do COMMANDS; done                          unset [-f] [-v] [name ...]
 for (( exp1; exp2; exp3 )); do COMMANDS; done                         until COMMANDS; do COMMANDS; done
 function name { COMMANDS ; } or name () { COMMANDS ; }                variables - Names and meanings of some shell variables
 getopts optstring name [arg]                                          wait [id]
 hash [-lr] [-p pathname] [-dt] [name ...]                             while COMMANDS; do COMMANDS; done
 help [-dms] [pattern ...]                                             { COMMANDS ; }

Aside from built-ins there are also functions and aliases. These are ways of defining new functionality without having to create separate scripts/binaries.

uppercase() {
    tr '[:lower:]' '[:upper:]' <<< "$*"
}

alias ls='ls --color=auto -F'

Functions and aliases are usually for convenience or to add supplementary functionality.

What about shell scripts? ... Will the command line fork() a new process and exec() this script in the new process? And then will this new process containing the script fork() other processes to execute date, cc compiler, etc ??

Yes, exactly right. When a shell script is run the parent shell forks a child process and the script runs there. The commands in the script and therefore forked off of this child process; they are grandchildren of the original shell.

Answer 2

When you execute a shellscript, it does fork off and create a new shell, interpreting each command through a separate fork/exec mechanism. However, there are some shell builtins, for example, echo may be built into some shells even when it is available as an executable in /usr/bin. cp and mv are indeed executables that are executed through fork/exec mechanism. One thing you may have missed is that the executables need to be in a directory contained in your PATH variable. Try renaming a hello world code executable in your current directory as ls and specify your current directory (.) as the first one in your path. You can also find out about the executables using type and which commands.