Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I shuffle the lines of a text file on the Unix command line or in a shell script?

I want to shuffle the lines of a text file randomly and create a new file. The file may have several thousands of lines.

How can I do that with cat, awk, cut, etc?

like image 592
Ruggiero Spearman Avatar asked Jan 28 '10 10:01

Ruggiero Spearman


People also ask

How do you shuffle a line in a text file in Linux?

Using the shuf Command The shuf utility is a member of the GNU Coreutils package. It outputs a random permutation of the input lines. The shuf command will load all input data into memory during the shuffling, and it won't work if the input file is larger than the free memory.

How do you scroll through a file in Unix?

On modern Linux systems you can use the [UpArrow] and [DownArrow] keys to scroll through the display. You can also use these keys to move through the output: [Space] - scrolls the display, one screenful of data at a time.

How do you write a line to a file in shell script?

Using '>>' with 'echo' command appends a line to a file. Another way is to use 'echo,' pipe(|), and 'tee' commands to add content to a file.


2 Answers

You can use shuf. On some systems at least (doesn't appear to be in POSIX).

As jleedev pointed out: sort -R might also be an option. On some systems at least; well, you get the picture. It has been pointed out that sort -R doesn't really shuffle but instead sort items according to their hash value.

[Editor's note: sort -R almost shuffles, except that duplicate lines / sort keys always end up next to each other. In other words: only with unique input lines / keys is it a true shuffle. While it's true that the output order is determined by hash values, the randomness comes from choosing a random hash function - see manual.]

like image 98
Joey Avatar answered Sep 22 '22 11:09

Joey


Perl one-liner would be a simple version of Maxim's solution

perl -MList::Util=shuffle -e 'print shuffle(<STDIN>);' < myfile 
like image 42
Moonyoung Kang Avatar answered Sep 21 '22 11:09

Moonyoung Kang