Powershell:在上次运行后从文件中复制新条目

zc0qhyus  于 12个月前  发布在  Shell
关注(0)|答案(2)|浏览(120)

我正在尝试创建一个powershell脚本,每次运行该脚本时,只从一个文件中复制与模式(可以是一行或多行)匹配的新条目到另一个文件中。源文件由应用程序随机更新,但请求是每小时复制最后一个条目。
我正在研究的解决方案是将上次运行的最后一个条目存储在文件中,并与文件中的最后一个条目进行比较,如果这些不匹配,则开始复制该条目之后的新行。这是我卡住的部分,我不知道如何表示,而是每次都复制整个内容。
这是我目前得到的:

Write-Host "Declaring the output log file ..... " -ForegroundColor Yellow
$destinationfile = 'C:\file\out\output.log'

Write-Host "Getting the last line of the source file ..... " -ForegroundColor Yellow
$sourcefile = 'C:\app\blade\inputlogs.log'
$sourcefilelastline = Get-Content $originfile | Select-Object -last 1
$sourcefilelastline

Write-Host "Getting the last line of the destination file ..... " -ForegroundColor Yellow
$destinationfilelastline = Get-Content $destinationfile | Select-Object -last 1
$destinationfilelastline

if ($sourcefilelastline -eq $destinationfilelastline){
    Write-Host "Skipping the process ..... " -ForegroundColor Yellow
}
else{
    Write-Host "Reading the source log file and updating destination file  ..... " -ForegroundColor Yellow
    $sourcefilecontent = Get-Content -Path $sourcefile | Where-Object { $_ -ne '' } | Select-String -Pattern 'error' -CaseSensitive -SimpleMatch
    $sourcefilecontent | Add-Content $destinationfile
}

字符串
有什么办法可以帮我完成吗?谢谢。

1bqhqjot

1bqhqjot1#

Get-content有一个开关“tail”,它可以让你从文件中读取最后一行。按照微软自己的话:
Tail参数获取文件的最后一行。此方法比检索变量中的所有行并使用[-1]索引表示法更快。
你可以在你的情况下使用它,从底线开始,直到它们匹配。

<# The Function.
.SYNOPSIS
 Recive X last number of lines from a file

.DESCRIPTION
Using the "Tail" parameter of get-content we can get the X number of lines from a file.
This should dramatically improve performance instead of reading the entire file.

.PARAMETER FilePath
Mandatory parameter.
Path to the file to get the X number of lines from.

.PARAMETER LastLines
Optional parameter.
How many lines to read from the end of the file.
Default value = 1

.EXAMPLE
Get-FileLastContent -FilePath "C:\Windows\System32\drivers\etc\hosts" -LastLines 1

.NOTES
General notes
#>

function Get-FileLastContent {
    param (
        [Parameter(
            Position = 0,
            Mandatory = $true,
            HelpMessage = 'Path to File')]
        [ValidateScript({ Test-Path -Path $_ })]
        [string]
        $FilePath,

        [Parameter(
            Position = 1,
            HelpMessage = 'Number of lines lines to check')]
        [ValidateNotNullOrEmpty()]
        [int]
        $LastLines = 1
    )


    $FileContent = Get-Content -Path $FilePath -Tail $LastLines
    Return $FileContent 
}

## Inisialization of Variables, as sometimes re-running the script is using values from last run.
$LastLines = 1
$destinationfilelastline = ''
$sourcefilelastline = ''


##Declaring the Destnation File
Write-Host -Object  "Declaring the output log file ..... " -ForegroundColor Yellow
$destinationfile = 'C:\file\out\output.log'

## Using the function to recive it's last line
Write-Host -Object "Getting the last line of the destination file ..... " -ForegroundColor Yellow
$destinationfilelastline = Get-FileLastContent -FilePath $destinationfile
$destinationfilelastline

## Declaring the source file and using the function to get it's last line
Write-Host -Object "Getting the last line of the source file ..... " -ForegroundColor Yellow
$sourcefile = 'C:\app\blade\inputlogs.log'
$sourcefilelastline = Get-FileLastContent -FilePath $sourcefile -LastLines 1
Write-Host -Object $sourcefilelastline

## if source file is empty, empty destination file as well
if ($sourcefilelastline.Length -eq 0) {
    Write-Host -Object "Soruce file is empty, clearing destination" -ForegroundColor Yellow
    Set-Content -Path $destinationfile -Value "" -Force -Encoding UTF8 -NoNewline
}

## If source file is not empty, but not matching the destination file
elseif (($sourcefilelastline -ne $destinationfilelastline)) {
    Write-Host -Object "Reading the source log file and updating destination file  ..... " -ForegroundColor Yellow
    ## if destination file is not empty, loop in the source file bottom-up until it is finding a line matching the destinaion file
    if ($destinationfilelastline.Length -gt 0) {
        while (($sourcefilelastline[0] -ne $destinationfilelastline) -and ($LastLines -le $sourcefile.Length)) {
            $LastLines = $LastLines + 1
            $sourcefilelastline = Get-FileLastContent -FilePath $sourcefile -LastLines $LastLines
        }
        ## If found a match in source file compared to the destination file
        if (($sourcefilelastline[0] -eq $destinationfilelastline)) {
            ## Prepare sourcefilelastline variable for export, skip the first result as it is already in the destination file (they match).
            $sourcefilelastline = $sourcefilelastline | Select-Object -Skip 1 | Where-Object { $_ -ne '' } | Select-String -Pattern 'error' -CaseSensitive -SimpleMatch
            # adding new line at the start
            $sourcefilelastline[0] = "`n" + $sourcefilelastline[0]
            ## Export the sourcefilelastline to the destination file
            $sourcefilelastline | Out-File -FilePath $destinationfile -Force -Encoding UTF8 -Append
        }
        else {
            ## it means that the sourcefile was overwriteen since last check with complete new data
            ## Overwrite destination file with the last line of source file
            $sourcefilelastline[($sourcefilelastline.Length-1)] | Out-File -FilePath $destinationfile -Force -Encoding UTF8
        }
    }
    #If no match found in sourcefile compare to the destination file
    
    # if destination file is empty, copy the last line of the source file to the destination file.
    else {
        $sourcefilelastline | Out-File -FilePath $destinationfile -Force -Encoding UTF8
    }
}
## Skip if source and destination are equal
else {
    Write-Host -Object "Skipping the process ..... " -ForegroundColor Yellow
}

字符串

yizd12fk

yizd12fk2#

这是一个小实验,但似乎效果很好。
函数Read-LastLinesOfTextFile:

  • 接受一个$AppName参数,该参数用于在注册表中定义一个路径来存储值-在本例中,是文件的最后读取结束的位置。
  • 这里的想法是,你可以有多个脚本调用这个函数,但是因为你为每个脚本使用一个唯一的名称,所以它们之间不会有任何冲突。
  • 接受$TextFile参数,该参数是需要读取的文件的路径。
  • 返回作为字符串数组读取的行。
  • 使用seek设置用于读取文件的文件流的读取位置。
  • 从文件流中阅读后,检索新位置并将其保存到注册表中,下次调用函数时将使用该注册表。
function Read-LastLinesOfTextFile {
    param(
        [Parameter(Mandatory = $true, Position = 0)]
        [string]$AppName,
        [Parameter(Mandatory = $true, Position = 1)]
        [string]$TextFile
    )
    $RegistryPath = "HKCU:\Software\$AppName"               # Registry key path for storing registry value
    if (-not (Test-Path "$RegistryPath")) {                 # If key path does not exists
        $null = New-Item -Path $RegistryPath -Force         #   Then create registry key
    }
    # Check if the registry value for LastPosition does NOT exists - https://stackoverflow.com/a/43921551/4190564
    if ( (Get-ItemProperty "$RegistryPath").PSObject.Properties.Name -notcontains "LastPosition" ) {
        # Create LastPosition value
        $null = New-ItemProperty -Path $RegistryPath -Name "LastPosition" -Value 0 -PropertyType DWORD -Force
    }
    # Save registry value LastPosition to $LastPosition
    $LastPosition = Get-ItemPropertyValue -Path $RegistryPath -Name "LastPosition"
    $CurrentFileSize = (Get-Item $TextFile).Length          # Get current file size
    if ($CurrentFileSize -lt $LastPosition) {               # If the file is smaller than it used to be
        $LastPosition = 0                                   #   Then assume it was deleted and now has all new data.
    } elseif ($CurrentFileSize -lt $LastPosition) {
        return @()
    }
    # Open file stream
    try { $FileStream = New-Object System.IO.FileStream($TextFile, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read) }
    catch { return @() }
    # Open stream reader
    try { $StreamReader = New-Object System.IO.StreamReader($FileStream) }
    catch {
        $FileStream.Close()                                 # Close FileStream
        return @()
    }
    $Return = @()                                           # Define default return value of empty array
    try {
        if ($LastPosition) {                                # If LastPosition anything other than 0
            # Seek last position
            $null = $FileStream.Seek($LastPosition, [System.IO.SeekOrigin]::Begin)
        }
        $Text = $StreamReader.ReadToEnd()                   # Read to the end of the file
        # Update the registry with the new last position
        Set-ItemProperty -Path $RegistryPath -Name "LastPosition" -Value $FileStream.Position
        $Return = $Text -split "\r?\n"                      # Split string into lines. https://stackoverflow.com/a/76831908/4190564
    }
    finally {
        $StreamReader.Close()                               # Close StreamReader
        $FileStream.Close()                                 # Close FileStream
    }
    return $Return
}

字符串
要调用这个函数,使用类似于下面的一行,只需要确保将MyLogReader替换为您的脚本唯一的名称,这样您就不会与使用相同函数的其他脚本发生冲突。看起来您正在对“error”进行区分大小写的比较,所以我在这个例子中使用了-cmatch。如果您想要区分大小写,请将其更改为-match

Read-LastLinesOfTextFile 'MyLogReader' "$PSScriptRoot\MyLogFile.LOG" | Where-Object { $_ -cmatch 'error' } | Add-Content $destinationfile

相关问题