Menu
I have a ksh script that returns a long list of values, newline separated, and I want to see only the unique/distinct values. It is possible to do this?
For example, say my output is file suffixes in a directory:
tar gz java gz java tar class class
I want to see a list like:
tar gz java class
862
To find unique occurrences where the lines are not adjacent a file needs to be sorted before passing to uniq . uniq will operate as expected on the following file that is named authors. txt . As duplicates are adjacent uniq will return unique occurrences and send the result to standard output.
The uniq command can count and print the number of repeated lines. Just like duplicate lines, we can filter unique lines (non-duplicate lines) as well and can also ignore case sensitivity.
bash [filename] runs the commands saved in a file. $@ refers to all of a shell script's command-line arguments. $1 , $2 , etc., refer to the first command-line argument, the second command-line argument, etc. Place variables in quotes if the values might have spaces in them.
You might want to look at the uniq and sort applications.
./yourscript.ksh | sort | uniq
(FYI, yes, the sort is necessary in this command line, uniq only strips duplicate lines that are immediately after each other)
EDIT:
Contrary to what has been posted by Aaron Digulla in relation to uniq's commandline options:
Given the following input:
class jar jar jar bin bin java
uniq will output all lines exactly once:
class jar bin java
uniq -d will output all lines that appear more than once, and it will print them once:
jar bin
uniq -u will output all lines that appear exactly once, and it will print them once:
class java
61
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
