Menu
I would like to know how to extract a list of unique lines from a text file. Preferably through Cygwin.
500
The uniq command finds the unique lines in a given input ( stdin or a filename command line argument) and either reports or removes the duplicated lines. This command only works with sorted data. Hence, uniq is often used with the sort command. To count how many times each of the lines appears in the file, ...
One of the easiest way to get the number of unique words in your file: tr ' ' '\n' < file_name | sort | uniq -c | wc -l.
To only show lines that are not repeated pass the -u option to uniq . This will output only lines that are not repeated and write the result to standard output.
Your question is somewhat unclear. If you want to eliminate all duplicate lines from a text file you can do something like this:
cat file.txt | sort | uniq
sort -u file > new_file
25
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
