powershellemailloggingalerttail

Tail log file with Powershell and email but only on first occurrence until next time

I've found the following script to send an email when a string is found in a log file

Get-Content C:\temp\server.log -Wait -Last 0 |
Where-Object { $_ -match "ERROR \[STDERR\]" } |
ForEach-Object {
Send-MailMessage -SmtpServer x.x.x.x -From ALERT@com -To alert.com -Subject 'Error' -Body $_
}

This works well but I'd like to take it a couple of steps further. When exceptions are thrown in the log file you can have multiple lines with the matched string thus triggering multiple email alerts for each line, what would be better is a single email to catch the whole block of the error message ( or x number of lines before and after)

Is this possible with Powershell?

Thanks in advance

Donald

I tried the block of code above and got multiple emails when I only want a single email

Sample log:

09:14:01,407 ERROR         [org.apache.struts.action.RequestProcessor]
09:21:34,649 ERROR [stderr] (default task-    554java.io.IOException:
09:21:34,650 ERROR [stderr] (default task-554)  at java.base/
09:21:34,650 ERROR [stderr] (default task-554)  at java.base/
12:18:53,286 ERROR     [org.apache.struts.action.RequestProcessor] (default task-    949) Invalid path 
12:24:06,441 ERROR     [org.apache.struts.action.RequestProcessor] (default task-    957) Invalid path
14:23:57,661 ERROR [stderr] (default task-1114at         org.jboss.threads@2.4.0.Final//org.
org.jboss.threads@2.4.0.
14:23:57,661 ERROR [stderr] (default task-1114)     at             org.jboss.xnio@3.8.7.F
Solution

  • The following solution:

    # Create a concurrent (thread-safe) queue.
$msgQueue = [System.Collections.Concurrent.ConcurrentQueue[string]]::new()

# Start a thread job that will monitor the queue and send
# blocks of error lines per email.
$job = Start-ThreadJob {
  
  # Get a reference to the concurrent queue from the caller's scope.
  $msgQueue = $using:msgQueue 
  # Helper variables.
  $msg = $null
  $msgs = [System.Collections.Generic.List[string]] @()
  $sw = [System.Diagnostics.Stopwatch]::new()

  while ($true) {

    # Poll for new queue items every N milliseconds; adjust as needed.
    # Note that Get-Content -Wait itself polls once every second.
    Start-Sleep -Milliseconds 500

    # Collect queued messages, up to 10 at a time, if received within
    # a 1-second window.
    if ($msgQueue.TryDequeue([ref] $msg)) {
      $msgs.Add($msg)
      # Wait for up to 9 more messages for at most (roughly) 1 second, 
      # whichever comes first.
      $sw.Restart()
      while ($msgs.Count -lt 10 -and $sw.Elapsed.TotalSeconds -le 1) {
        if ($msgQueue.TryDequeue([ref] $msg)) { $msgs.Add($msg) }        
        Start-Sleep -Milliseconds 50 # Sleep a little, to avoid a tight loop.
      }
      # Simulate sending the email with the collected lines.
      # Place your Send-MailMessage call here.
      # Note: 
      #  [Console]::WriteLine() must be used in order to print to the screen
      #  from a thread job.
      [Console]::WriteLine("--- Emailing the following lines:`n$($msgs -join "`n")")
      $msgs.Clear()
    } 

  }

}

# Start the tailing operation (Get-Content -Wait).
# Stopping it requires Ctrl-C.
try {
  $logFile = 'C:\temp\server.log'
  Get-Content $logFile -Wait -Last 1 |
    Where-Object { $_ -match 'ERROR \[STDERR\]' } |
    ForEach-Object {
      # Enqueue lines of interest, to be processed by the thread job.
      $msgQueue.Enqueue($_)
    }
}
finally {
  # Ctrl-C was pressed: clean up the thread job before exiting.
  if ($job) { $job | Remove-Job -Force }
}

    Self-contained example with debugging:

    Note:

    Instructions:

    # To work around a PSReadLine bug, unload it for the duration of this script.
if ($psrlWasLoaded = [bool] (Get-Module PSReadLine)) {
  Remove-Module PSReadLine
}

# Initialize the file to be tailed.
# For simplicity, the path is hard-coded to 'sample_log.txt' in the current dir.
1 | Set-Content sample_log.txt

# To trigger activity, repeat the following on demand from a separate session:
$sb = {
$ExecutionContext.InvokeCommand.ExpandString(@'
$([datetime]::now.Tostring('HH:mm:ss.ffff')) 1 ERROR [stderr](default task-    554java.io.IOException: 
$([datetime]::now.Tostring('HH:mm:ss.ffff')) 2 ERROR [stderr] (default task-    555java.io.IOException:
$([datetime]::now.Tostring('HH:mm:ss.ffff')) 3 ERROR [stderr](default task-    556java.io.IOException:
'@)  | Add-Content sample_log.txt
}

Write-Verbose -Verbose @"
To trigger activity, run the following from a separate session:
$sb
"@

# Create a concurrent (thread-safe) queue.
$msgQueue = [System.Collections.Concurrent.ConcurrentQueue[string]]::new()

# Start a thread job that will monitor the queue and send
# blocks of error lines per email.
$job = Start-ThreadJob {
  
  # Get a reference to the concurrent queue from the caller's scope.
  $msgQueue = $using:msgQueue 
  # Helper variables.
  $msg = $null
  $msgs = [System.Collections.Generic.List[string]] @()
  $sw = [System.Diagnostics.Stopwatch]::new()
  
  # !! This is necesary in order to reliably break into the debugger
  # !! on repeat invocations with Debug-Job.
  # !! Breaks whenever $breakHere is assigned to, set in the loop below.
  Set-PSBreakpoint -Variable breakHere
  # !! Wait-Debugger SHOULD work, but as of v7.4.1 doesn't in THREAD jobs.

  while ($true) {

    $breakHere = 1 # Trigger breaking into the debugger when run with Debug-Job.

    # Poll for new queue items every N milliseconds; adjust as needed.
    # Note that Get-Content -Wait itself polls once every second.
    Start-Sleep -Milliseconds 500

    # Collect queued messages, up to 10 at a time, if received within
    # a 1-second window.
    if ($msgQueue.TryDequeue([ref] $msg)) {
      $msgs.Add($msg)
      # Wait for up to 9 more messages for at most (roughly) 1 second, 
      # whichever comes first.
      $sw.Restart()
      while ($msgs.Count -lt 10 -and $sw.Elapsed.TotalSeconds -le 1) {
        if ($msgQueue.TryDequeue([ref] $msg)) { $msgs.Add($msg) }        
        Start-Sleep -Milliseconds 50 # Sleep a little, to avoid a tight loop.
      }
      # Simulate sending the email with the collected lines.
      # Place your Send-MailMessage call here.
      # Note: 
      #  [Console]::WriteLine() must be used in order to print to the screen
      #  from a thread job.
      [Console]::WriteLine("--- Emailing the following lines:`n$($msgs -join "`n")")
      $msgs.Clear()
    } 

  }

}

# Start the tailing operation (Get-Content -Wait).
# Stopping it requires Ctrl-C.
try {

  # Set this to $true to turn on debugging.
  $launchDebugger = $true
  # If debugging, wait until the job is ready to be debugged.
  if ($launchDebugger) { while ($job.State -notin 'Running', 'AtBreakPoint') { Start-Sleep -Milliseconds 100 } }
  
  Write-Verbose -Verbose 'Starting to tail sample_log.txt...'

  # Main loop: Tail a file and add lines of interest to a queue that is
  #            processed by the thread job.
  Get-Content sample_log.txt -Wait -Last 1 |
    Where-Object { $_ -match 'ERROR \[STDERR\]' } |
    ForEach-Object {

      # Enqueue lines of interest, to be processed by the thread job.
      "Received locally: [$_]"
      Write-Verbose -Verbose "Enqueuing..."
      $msgQueue.Enqueue($_)

      # If requested, break into the debugger in the thread job.
      if ($launchDebugger) { 
        Write-Verbose -Verbose 'Invoking the debugger on the thread job...'
        $job | Debug-Job # This is a blocking call.
        Write-Verbose -Verbose 'Resuming tailing...'
      }
    }
}
finally {
  # Ctrl-C was pressed: clean up the thread job before exiting.
  if ($job) { $job | Remove-Job -Force }
  if ($psrlWasLoaded) { Import-Module PSReadLine }
  Remove-Item -LiteralPath sample_log.txt
}