code coverage tools for validating the scripts

Question

Is there any way to validate the coverage of the shell scripts? I have project having lots of shell scripting, and need to ensure static analysis can be performed on the coverage for the shell scripts. Is there any tool available?

Answer 1

I seriously doubt that there could be any static code analysis performed on shell scripts - especially due to the fact that shell scripts are supposed to call external programs and based on what these external programs return - and there are myriads of external programs and external environment states. It's similar to the problem of static analysis of code heavily relying on eval-like mechanism, but shell scripting is all about eval-style programming.

However, there are some general pointers that could prove useful for "proper" validation, code coverage and documenting of shell scripts as major languages do:

• You can always run a script with -x (AKA xtrace) option - it will output trace looking like that to stderr:

  + log_end_msg 0
  + [ -z 0 ]
  + retval=0
  + log_end_msg_pre 0
  + :
  + log_use_fancy_output
  + TPUT=/usr/bin/tput
  + EXPR=/usr/bin/expr
  + [ -t 1 ]
  + [ xxterm != x ]
  + [ xxterm != xdumb ]
  

• Bash makes it possible to redirect this stream to external FD (using BASH_XTRACEFD variable) - that's much easier to parse in practice.

• It's not trivial, but it's possible to write a program that will find relevant pieces of code being executed using xtrace output and make you a fancy "code coverage" report - like what was called how many times, which pieces of code weren't run at all and thus lack test coverage.

• In fact, there's a wonderful tool named shcov already written that uses this process - albeit it's somewhat simplistic and doesn't handle all possible cases very well (especially when we're talking about long and complex lines)

• Last, but not least - there's minimalistic shelldoc project (akin to javadoc) that helps generating documentation based on comments in shell scripts. Yep, that's a shameless plug :)