Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Shuffle the records of a list of text files in one single file

I have a list of text files file1.txt, file2.txt, file3.txt .. filen.txt that I need to shuffle creating one single big file as result *.

Requirements:
1. The records of a given file need to be reversed before being shuffled
2. The records of a given file should keep the reversed order in the destination file
3. I don't know how many files I need to shuffle so the code should be generic as possible (allowing to declare the file names in a list for example)
4. Files could have different sizes

Example:

File1.txt
---------
File1Record1
File1Record2
File1Record3
File1Record4

File2.txt
---------
File2Record1
File2Record2


File3.txt
---------
File3Record1
File3Record2
File3Record3
File3Record4
File3Record5

the output should be something like this:

ResultFile.txt
--------------
File3Record5   -|
File2Record2    |
File1Record4    |
File3Record4   -|
File2Record1    |
File1Record3    |-->File3 records are shuffled with the other records and 
File3Record3   -|   are correctly "reversed" and they kept the correct 
File1Record2    |   ordering
File3Record2   -|
File1Record1    |
File3Record1   -|

* I'm not crazy; I have to import these files (blog posts) using the resultfile.txt as input

EDIT:
the result could have any sort you want, completely or partially shuffled, uniformly interleaved, it does not matter. What it does matter is that points 1. and 2. are both honoured.

like image 220
systempuntoout Avatar asked Jun 26 '11 21:06

systempuntoout


1 Answers

What about this:

>>> l = [["1a","1b","1c","1d"], ["2a","2b"], ["3a","3b","3c","3d","3e"]]
>>> while l:
...     x = random.choice(l)
...     print x.pop(-1) 
...     if not x:
...         l.remove(x)

1d
1c
2b
3e
2a
3d
1b
3c
3b
3a
1a

You could optimize it in various ways, but that's the general idea. This also works if you cannot read the files at once but need to iterate them because of memory restrictions. In that case

  • read a line from the file instead of popping from a list
  • check for EOF instead of empty lists
like image 177
Oben Sonne Avatar answered Sep 29 '22 01:09

Oben Sonne