Powershell解压缩流

f3temu5u  于 2023-04-12  发布在  Shell
关注(0)|答案(2)|浏览(241)

是否有一个内置的cmdlet或一些组合,可以让我在下载每个块时开始解压缩文件流?我有一个PowerShell脚本,需要下载一个大(10 GB)文件,我必须等待,直到它现在完成,然后才开始扩展。

$wc = New-Object net.webclient
$wc.Downloadfile($appDataSnapshotUri, "%DataSnapshotFileName%.zip") # this can take some time

Expand-Archive -Path "%DataSnapshotFileName%.zip" -DestinationPath Run # so can this
py49o6xq

py49o6xq1#

好的,事实证明zip文件不需要完全下载才能解压缩,你可以压缩/解压缩流。.Net中有一些内置的流压缩功能,但它不适用于zip存档。你可以使用SharpZipLib库:
https://www.nuget.org/packages/SharpZipLib/下载.nupckg将文件提取到任何文件夹。您将需要lib/net 45中的ICSharpCode.SharpZipLib.dll
下面是我对他们的例子的简化翻译:https://github.com/icsharpcode/SharpZipLib/wiki/Zip-Samples#unpack-a-zip-using-zipinputstream-eg-for-unseekable-input-streams

Add-Type -Path ".\ICSharpCode.SharpZipLib.dll"

$outFolder = ".\unzip"

$wc = [System.Net.WebClient]::new()

$zipStream = $wc.OpenRead("http://gitlab/test/test1/raw/master/sample.zip")

$zipInputStream = [ICSharpCode.SharpZipLib.Zip.ZipInputStream]::New($zipStream)

$zipEntry = $zipInputStream.GetNextEntry()

$fileName = $zipEntry.Name

$buffer = New-Object byte[] 4096

$sw = [System.IO.File]::Create("$outFolder\$fileName")

[ICSharpCode.SharpZipLib.Core.StreamUtils]::Copy($zipInputStream, $sw, $buffer)

$sw.Close()

它只会提取第一个条目,你可以添加一个while循环,这个例子可以工作。
下面是一个使用while循环提取多个文件的代码片段(将其放在上面示例中的$zipEntry = $zipInputStream.GetNextEntry()之后):

While($zipEntry) {

$fileName = $zipEntry.Name

Write-Host $fileName

$buffer = New-Object byte[] 4096

$sw = [System.IO.File]::Create("$outFolder\$fileName")

[ICSharpCode.SharpZipLib.Core.StreamUtils]::Copy($zipInputStream, $sw, $buffer)

$sw.Close()

$zipEntry = $zipInputStream.GetNextEntry()

}

编辑

这是我发现的工作…

Add-Type -Path ".\ICSharpCode.SharpZipLib.dll"

$outFolder = "unzip"

$wc = [System.Net.WebClient]::new()

$zipStream = $wc.OpenRead("https://github.com/Esri/file-geodatabase-api/raw/master/FileGDB_API_1.5/FileGDB_API_1_5_VS2015.zip")

$zipInputStream = [ICSharpCode.SharpZipLib.Zip.ZipInputStream]::New($zipStream)

$zipEntry = $zipInputStream.GetNextEntry()

while($zipEntry) {

if (-Not($zipEntry.IsDirectory)) { 
  $fileName = $zipEntry.Name

  $buffer = New-Object byte[] 4096

  $filePath = "$pwd\$outFolder\$fileName"
  $parentPath = "$filePath\.."
  Write-Host $parentPath

  if (-Not (Test-Path $parentPath)) {
      New-Item -ItemType Directory $parentPath
  }

  $sw = [System.IO.File]::Create("$pwd\$outFolder\$fileName")

  [ICSharpCode.SharpZipLib.Core.StreamUtils]::Copy($zipInputStream, $sw, $buffer)
  $sw.Close()

}

$zipEntry = $zipInputStream.GetNextEntry()

}
jk9hmnmh

jk9hmnmh2#

为了扩展Mike Twc的答案,一个脚本可以在有和没有流的情况下完成它,并比较它需要多长时间:

$url = "yoururlhere"

function UnzipStream () {
    Write-Host "unzipping via stream"

    $stopwatch1 =  [system.diagnostics.stopwatch]::StartNew()

    Add-Type -Path ".\ICSharpCode.SharpZipLib.dll"

    $outFolder = "unzip-stream"

    $wc = [System.Net.WebClient]::new()

    $zipStream = $wc.OpenRead($url)

    $zipInputStream = [ICSharpCode.SharpZipLib.Zip.ZipInputStream]::New($zipStream)

    $zipEntry = $zipInputStream.GetNextEntry()

    while($zipEntry) {

    if (-Not($zipEntry.IsDirectory)) { 
    $fileName = $zipEntry.Name

    $buffer = New-Object byte[] 4096

    $filePath = "$pwd\$outFolder\$fileName"
    $parentPath = "$filePath\.."
    Write-Host $parentPath

    if (-Not (Test-Path $parentPath)) {
        New-Item -ItemType Directory $parentPath
    }

    $sw = [System.IO.File]::Create("$pwd\$outFolder\$fileName")

    [ICSharpCode.SharpZipLib.Core.StreamUtils]::Copy($zipInputStream, $sw, $buffer)

    $sw.Close()
    }

    $zipEntry = $zipInputStream.GetNextEntry()

    }

    $stopwatch1.Stop()

    Write-Host "extraction took $($stopWatch1.ElapsedMilliseconds) millis with stream"
}

function UnzipWithoutStream() {

    Write-Host "Extracting without stream"

    $stopwatch2 = [system.diagnostics.stopwatch]::StartNew()
    $outFolder2 = "unzip-normal"

    $wc2 = New-Object System.Net.WebClient
    $wc2.DownloadFile($url, "$pwd\download.zip")

    $of2 = New-Item -ItemType Directory $outFolder2

    Expand-Archive -Path "download.zip" -DestinationPath $of2.FullName

    $stopwatch2.Stop()

    Write-Host "extraction took $($stopWatch2.ElapsedMilliseconds) millis without stream"
}

UnzipStream
UnzipWithoutStream

相关问题