I've seen the answer elsewhere for text files, but I need to do this for a compressed file.
I've got a 6G binary file which needs to be split into 100M chunks. Am I missing the analog for unix's "head" somewhere?
PowerShell can split large files in multiple smaller parts, for example to transfer them as email attachments. Today, we focus on splitting files. In our next tip, we show how you can join the parts back together. To split large files into smaller parts, we created the function Split-File. It works like this:
The Split-File function accepts the file path as an input parameter which will be split into multiple parts and PartSizeBytes to split the file into provided size. The output of the above PowerShell script will split a large log file into multiple files of size 1MB. Use the Get-ChildItem command to get the list of files in the directory.
There's no built-in DOS command for that. Use the dos port of the unix split command: There are 3rd party alternatives, but this is the simplest. Windows has a few commands that can be written in batch file to split a binary file; no need to download external or port other OS utilities.
few ways to split a file with [batch script without external tools] [1]. [1]: stackoverflow.com/questions/28244063/… makecab can split a binary file into smaller encoded chunks in it's own format, but they can't be treated as just raw bytes, similar to a flat binary file eg. via copy. The chunks, however can then be joined by extrac32.
Never mind. Here you go:
function split($inFile, $outPrefix, [Int32] $bufSize){
$stream = [System.IO.File]::OpenRead($inFile)
$chunkNum = 1
$barr = New-Object byte[] $bufSize
while( $bytesRead = $stream.Read($barr,0,$bufsize)){
$outFile = "$outPrefix$chunkNum"
$ostream = [System.IO.File]::OpenWrite($outFile)
$ostream.Write($barr,0,$bytesRead);
$ostream.close();
echo "wrote $outFile"
$chunkNum += 1
}
}
Assumption: bufSize fits in memory.
The answer to the corollary question: How do you put them back together?
function stitch($infilePrefix, $outFile) {
$ostream = [System.Io.File]::OpenWrite($outFile)
$chunkNum = 1
$infileName = "$infilePrefix$chunkNum"
$offset = 0
while(Test-Path $infileName) {
$bytes = [System.IO.File]::ReadAllBytes($infileName)
$ostream.Write($bytes, 0, $bytes.Count)
Write-Host "read $infileName"
$chunkNum += 1
$infileName = "$infilePrefix$chunkNum"
}
$ostream.close();
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With