Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

In Powershell, How do I split a large binary file?

Tags:

powershell

I've seen the answer elsewhere for text files, but I need to do this for a compressed file.

I've got a 6G binary file which needs to be split into 100M chunks. Am I missing the analog for unix's "head" somewhere?

like image 523
djsadinoff Avatar asked Dec 26 '10 10:12

djsadinoff


People also ask

How to split large files into smaller parts in PowerShell?

PowerShell can split large files in multiple smaller parts, for example to transfer them as email attachments. Today, we focus on splitting files. In our next tip, we show how you can join the parts back together. To split large files into smaller parts, we created the function Split-File. It works like this:

How to split a log file into multiple files using PowerShell?

The Split-File function accepts the file path as an input parameter which will be split into multiple parts and PartSizeBytes to split the file into provided size. The output of the above PowerShell script will split a large log file into multiple files of size 1MB. Use the Get-ChildItem command to get the list of files in the directory.

How do I split a binary file into two files?

There's no built-in DOS command for that. Use the dos port of the unix split command: There are 3rd party alternatives, but this is the simplest. Windows has a few commands that can be written in batch file to split a binary file; no need to download external or port other OS utilities.

Is there a way to split a file into smaller files?

few ways to split a file with [batch script without external tools] [1]. [1]: stackoverflow.com/questions/28244063/… makecab can split a binary file into smaller encoded chunks in it's own format, but they can't be treated as just raw bytes, similar to a flat binary file eg. via copy. The chunks, however can then be joined by extrac32.


2 Answers

Never mind. Here you go:

function split($inFile,  $outPrefix, [Int32] $bufSize){

  $stream = [System.IO.File]::OpenRead($inFile)
  $chunkNum = 1
  $barr = New-Object byte[] $bufSize

  while( $bytesRead = $stream.Read($barr,0,$bufsize)){
    $outFile = "$outPrefix$chunkNum"
    $ostream = [System.IO.File]::OpenWrite($outFile)
    $ostream.Write($barr,0,$bytesRead);
    $ostream.close();
    echo "wrote $outFile"
    $chunkNum += 1
  }
}

Assumption: bufSize fits in memory.

like image 72
djsadinoff Avatar answered Oct 12 '22 00:10

djsadinoff


The answer to the corollary question: How do you put them back together?

function stitch($infilePrefix, $outFile) {

    $ostream = [System.Io.File]::OpenWrite($outFile)
    $chunkNum = 1
    $infileName = "$infilePrefix$chunkNum"

    $offset = 0

    while(Test-Path $infileName) {
        $bytes = [System.IO.File]::ReadAllBytes($infileName)
        $ostream.Write($bytes, 0, $bytes.Count)
        Write-Host "read $infileName"
        $chunkNum += 1
        $infileName = "$infilePrefix$chunkNum"
    }

    $ostream.close();
}
like image 40
DrewDouglas Avatar answered Oct 12 '22 00:10

DrewDouglas