Select unique or distinct values from a list in UNIX shell script

Question

I have a ksh script that returns a long list of values, newline separated, and I want to see only the unique/distinct values. It is possible to do this?

For example, say my output is file suffixes in a directory:

tar gz java gz java tar class class 

I want to see a list like:

tar gz java class 

Answer 1

You might want to look at the uniq and sort applications.

 ./yourscript.ksh | sort | uniq 

(FYI, yes, the sort is necessary in this command line, uniq only strips duplicate lines that are immediately after each other)

EDIT:

Contrary to what has been posted by Aaron Digulla in relation to uniq's commandline options:

Given the following input:

 class jar jar jar bin bin java 

uniq will output all lines exactly once:

 class jar bin java 

uniq -d will output all lines that appear more than once, and it will print them once:

 jar bin 

uniq -u will output all lines that appear exactly once, and it will print them once:

 class java