What among Bash/Python/Perl/Ruby/Sed/Awk for System administration , coding accessories

Question

I know the question is very subjective. But I cannot form the question in a much more better manner. I would appreciate some guidance.

I often as a developer feel how easier it would have been for me if I could have some tools for doing some reasonably complex task on a log file , or a set of source files, or some data set etc.

Clearly, when the same type of task needs to be done repetitively and when speed is critical, I can think of writing it in C++/Java.

But most of the times, it is some kind of text processing or file searching activity that I want to do only once just to perform a quick check or to do some preliminary analysis etc. In such cases, I would be better off doing the task manually rather than writing in C++/Java. But I could surely doing it in seconds if I knew some language like Bash/Python/Perl/Ruby/Sed/Awk.

I know this whole question is subjective and there is no objective definite answer, but in general what does the developer community feel as a whole? What subset of these languages should I know so that I can do all these kinds of tasks easily and improve my productivity.

Would Perl be a good choice?
It is a super set of Sed/Awk, plus it allows to write terse code. I can get done with fewer lines of code. It is neither readable nor easily maintainable, but I never wanted those features anyway. The only thing that bothers me is the negative publiciity that Perl has got lately and it has been criticized by the Ruby/Python community a lot. Also, I am not sure if it can replace bash scripting totally. If not, then is Perl+Bash a good combination for these kind of tasks?

Answer 1

I tend to do a lot of processing with ruby. It has all the functionality of perl, but I find it to be a little more readable. Both perl and ruby support the -n, -e, and -p options.

-e 'command'    one line of script. Several -e's allowed. Omit [programfile]
-n              assume 'while gets(); ... end' loop around your script
-p              assume loop like -n but print line also like sed

For example in ruby

seq 1 4 | ruby -ne 'BEGIN{ $product = 1 }; $product *= $_.to_i; END { puts $product }'
24

Which is very similar to perl

seq 1 4 | perl -ne 'BEGIN{ $product = 1 }; $product *= $_; END { print $product }'
24

In Python, the same would look like this:

seq 1 4 | python -c 'import sys; print reduce(lambda x,y : int(x)*int(y),  sys.stdin.read().splitlines(True))'
24

While it's possible to do the above in bash/awk/sed, you'll be limited by their lack of more advanced features.

Answer 2

Python is more expressive and readable than bash, but requires more setup: import os and so on. For simple tasks, bash is quicker -- which is the most important for this. And don't underestimate the power of input/output redirection in bash!

Answer 3

I would use Perl over a bash/sed/awk combination. Why ?

1. You only have the one executable, rather than spawning off multiple executables to do work.
2. You can make use of a wide range of Perl modules to do most anything (see CPAN for the modules available)

In fact I would recommend any scripting language over the shell/awk/sed combination, for the same reasons. I don't have a problem with sed/awk per se, but as your required solutions become more complex/lengthy, I find the more powerful scripting languages more scalable, and (to some degree) refactorable for re-use.