James O'Neill's Blog

July 27, 2011

A “Tail” command in PowerShell

Filed under: Powershell — jamesone111 @ 3:46 pm

I mentioned that I have been working for a small Software company and in this new role I’m having to work with Linux servers and MySQL. MySQL has proved rather better than I expected and Linux itself worse –at least from the perspective of being place for me to get work done (I’ll leave the reasons why for another time).  Our app generates quite a lot of log files, and so do some of the services it uses , and so most of time I’ll have at least one Window open running the Unix tail –f command (tail outputs the last few lines from a file, and –f tells it to follow the file –that is, to keep watching it and output anything added to it).  There should be one for Windows but a quick search didn’t turn one up – so I put one together with PowerShell which pulls some interesting techniques from more than one place.

Watching a file

The first thing that’s needed to give the follow functionality is the ability to know when the log file has changed. I found that James Brundage’s PowerShell Pack gave me ALMOST what I needed. Here’s my modified version

Function Register-FileSystemWatcher {
param(  [Parameter(ValueFromPipelineByPropertyName=$true,Position=0,Mandatory=$true)]
      
 [Alias('FullName')] [string]$Path,
                            [string]$Filter = "*",
                            [switch]$Recurse, 
        [Alias('Do')][ScriptBlock[]]$Process,
                                    $MessageData, 
                         [string[]]$On = @("Created", "Deleted", "Changed", "Renamed"))
begin   { $ValidEvents = [IO.FileSystemWatcher].GetEvents() |
                            select-object -ExpandProperty name
        }
process {
    $realItem = Get-Item -path $path -ErrorAction SilentlyContinue
   
if (-not $realItem) { return }
    if ($realItem -is [system.io.fileinfo]) {
           $Path = Split-Path $realItem.Fullname
         $Filter = Split-Path $realItem.Fullname -leaf }
   
else { $Path = $realItem.Fullname}
    $watcher = New-Object IO.FileSystemWatcher -Property @{
                        
Path=$path;
                      Filter=$filter; 
       IncludeSubdirectories=$Recurse}
    foreach ($o in $on) {
    
 if ($validEvents -Ccontains $o) { #Note CASE SENSITIVE CONTAINS...
       
foreach ($p in $process) {
           
if ($p){Register-ObjectEvent $watcher $o -Action $d -MessageData $MessageData}
        }
     
} 
      Else {Write-warning ("$o is an invalid event name and was ignored." +
           
     [Environment]::NewLine + "Names are case sensitive and valid ones are"
+
                 [Environment]::NewLine + ($ValidEvents -join ", ") + ".")
      }
    }
}
}

The Function takes a path (usually a folder but possibly a file which can be passed via the pipeline) a filter, and a –recurse switch which determine what to watch. If the path isn’t valid the function drops out, and if the path is a file it is split into name and folder parts – the name becomes the filter and the folder becomes the path.
The path, filter and recurse are used to create a FileWatcher object. A FileWatcher raises events , and Register-ObjectEvent hooks PowerShell up to these events: the cmdlet says “when this event happens on that object, run this code block”. Usefully – as I learnt from a post on Ravikanth Chaganti’s blog you can pass something to the registration for use later on – which I’ll come back to in when we see the function in use. I do a quick check to see the event passed is valid before trying to hook up the code to it, generating a warning.

The tail command I created is named “Get-Tail” for two reasons,
(a) What gets returned is the “tail of the file” so GET- is the right verb to use out of the standard ones
(b) If PowerShell can’t find a command name it tries Get-Name as an alias, in other words:
 Get-Tail can be invoked simply as tail. It looks like this

function Get-tail {
param ( $path,
      [int]$Last = 20,
      [int]$CharsPerLine = 500,
      [Switch]$follow
)
$item = (Get-item $path)
if (-not $item) {return}
$Stream = $item.Open([System.IO.FileMode]::Open,
                   [System.IO.FileAccess]::Read, 
                    [System.IO.FileShare]::ReadWrite)
$reader = New-Object System.IO.StreamReader($Stream)
if ($charsPerLine * $last -lt $item.length) {
       $reader.BaseStream.seek((-1 * $last * $charsPerLine) ,[system.io.seekorigin]::End)
}
$reader.readtoend() -split "`n" -replace "\s+$","" | Select-Object -last $Last | write-host
if ($follow) {
          $Global:watcher = Register-FileSystemWatcher -Path $path -MessageData $reader`
        
 -On "Changed" -Process {
               $event.MessageData.readtoend() -split "`n" -replace "\s+$","" | write-host }
      $oldConsoleSetting = [console]::TreatControlCAsInput 
     
[console]::TreatControlCAsInput = $true
      while ($true) {
         
if ([console]::KeyAvailable) {
                       $key = [system.console]::readkey($true) 
                       if (($key.modifiers -band [consolemodifiers]"control")and
                           ($key.key -eq "C")) {
                           
 write-host -ForegroundColor red "Terminating..." ; break } 
                       else { if ([int]$key.keyCHAR -eq 13) { [console]::WriteLine() }
                     
       else { [console]::Write($key.keyCHAR) }} }
                 else {Start-Sleep -Milliseconds 250} } 
     
[console]::TreatControlCAsInput = $oldConsoleSetting
      Unregister-Event $watcher.name
   } 
$Stream.Close() 
$reader.Close()
}

Opening a sharable file in PowerShell

The function takes 4 parameters, –path , -last a -follow switch and a –CharsPerLine which was a bit of an after thought. : the .Open() method is used to open the file as a Read-Only FileStream allowing writes by others; and a StreamReader object is created to read from this Stream.

By using the Reader’s .ReadToEnd() method I could be ready to read anything which is added to the end of the file, and  output the result splitting it on new lines and removing end-of-line spaces – all of which is not much than I could have done with Get-Content.
I added a refinement after realizing that I occasionally deal with log files which are over a Gigabyte in length. Not only will they take ages to read, but using .ReadToEnd() will try to read the whole file into memory which is just horrible. So I added a –CharsPerLine parameter – I multiply this by the number of lines I want to read and if the file is bigger than that I seek forward to that many bytes from the end the file, before calling .ReadToEnd(). The default is a generous 500, so if I request 2000 lines I’ll read 1MB of data which isn’t too terrible even if the average line length is only a few characters. If I’m reading 20,000 lines I might set the parameter lower, or if I know the lines are very long I might set it higher. Then everything is set up for the optional Follow part.

Who reads for the Watchers ?

I want to call the .ReadToEnd() method when the file changes and output everything up to the end of the file. The question is, how to have access to the StreamReader inside the script which runs when FileSystemWatcher fires its changed event ? This is where Ravikanth Chaganti’s tip comes in; by making the Reader the “Message Object” for the event, it can be referenced in the script block.  By the way because I don’t know what else might end up happening I force the output to the console throughout – though my normal custom is to avoid using write-host
{$event.MessageData.readtoend() -split "\s*`n" | write-host }

Taking Control of Control+C

Finally – to mimic the behaviour of tail on unix I trap keyboard input and pass it through to the console until the user presses [Ctrl][c].  This works much better in the “Shell” form of PowerShell than the ISE. When [Ctrl][c]. is pressed the function cleans up and exits.

Job done. 

Update: I’ve put the script here for download

Advertisements

6 Comments

  1. Althhough it is bad form to comment on ones own blog, I wanted to add two things.
    (a) I should have named the “Chars per line” parameter WIDTH in keeping with what is used by other PowerShell functions.
    (b) Comparing this to get the last 20 lines of a 70MB / 800,000 line file with get-content | select -last 20 was 0.46 seconds against 61 seconds.

    Comment by James — July 28, 2011 @ 10:39 am

  2. cool scripts, can you post downloads of the scripts in txt format or ps1 format.
    displaying on the web page has corrupted some of the coding.
    i think i have it mostly debugged but still getting an error with Ctrl+C trying to unregister the watcher object.

    Comment by chris — July 28, 2011 @ 7:18 pm

    • Chris I’ve put the script on skydrive – link in the main body.

      Cheers
      James

      Comment by jamesone111 — July 31, 2011 @ 8:47 am

  3. Nice lateral thinking but you might also want to look at either cygwin (good if you want to use bash scripts) or unxutils (http://unxutils.sourceforge.net/) which are windows compiles of common linux utilities if you just need to run a quick command directly.

    Comment by Pionir — August 3, 2011 @ 10:26 am

  4. There’s tail command for windows too.. check the below..
    http://www.windows-commandline.com/2010/08/tail-command-for-windows.html

    Comment by Kakatiya — August 4, 2011 @ 9:29 am

    • I can’t help feeling I should have had a better knowledge of the resource kit !

      I’ve got more bits to write up on this, though because having re-invented the wheel I’ve made my wheel better 😉

      Comment by jamesone111 — August 8, 2011 @ 10:42 am


RSS feed for comments on this post.

Blog at WordPress.com.

%d bloggers like this: