I wish to split a large file (with ~ 17M Lines of Strings) into multiple files with varying number of lines in each chunk. Would it be possible to send in an array to the 'split -l' command like this:
[
1=>1000000,
2=>1000537,
...
]
so as to send those many number of lines to each chunk
To split a file into pieces, you simply use the split command. By default, the split command uses a very simple naming scheme. The file chunks will be named xaa, xab, xac, etc., and, presumably, if you break up a file that is sufficiently large, you might even get chunks named xza and xzz.
First up, right-click the file you want to split into smaller pieces, then select 7-Zip > Add to Archive. Give your archive a name. Under Split to Volumes, bytes, input the size of split files you want. There are several options in the dropdown menu, although they may not correspond to your large file.
To split a file equally into two files, we use the '-n' option. By specifying '-n 2' the file is split equally into two files.
Use a compound command:
{
head -n 10000 > output1
head -n 200 > output2
head -n 1234 > output3
cat > remainder
} < yourbigfile
This also works with loops:
{
i=1
for n in 10000 200 1234
do
head -n $n > output$i
let i++
done
cat > remainder
} < yourbigfile
This does not work on OS X, where head
reads and discards additional output.
The split
command does not have that capability, so you'll have to use a different tool,
or write one of your own.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With