Split a text file into multiple files by percentage for test and train

Question

I have a large txt file with 50000+ line, how can I randomly split them into 70% for training, 20% for test, and 10% for dev.

result expecting : train.txt, test.txt, dev.txt

Answer 1

I found this code much simpler.

## allocating train, test and validate datasets
import random 

fin = open('unique.txt', 'rb') 
f75out = open("train.txt", 'wb') 
f125aout = open("test.txt", 'wb')
f125bout = open("validate.txt", 'wb')

for line in fin: 
  r = random.random() 
  if (0.0 <=  r <= 0.75): 
    f75out.write(line) 
  elif (0.75 < r <= 0.875): 
    f125aout.write(line) 
  else:
    f125bout.write(line)
fin.close() 
f75out.close() 
f125aout.close() 
f125bout.close()