Friday, October 28, 2011

Merge and sort log4net files

Sometimes it is useful to combine multiple log files into one to get a better picture of what is going on. To do this the following power shell scripts may be useful. These scripts are not particularly elegant, but they do perform the job required…

First step… combine the log files into one. To keep track of where a particular entry originates from then the command below will concatenate all files in the current directory into res.txt, inserting the filename and line number after the timestamp. This command assumes all files start with a timestamp of the form yyyy-mm-dd hh:mm:ss,SSS. If a different format is used then update the regex to match the new beginning of line marker.

foreach($item in Get-ChildItem ".") {$line_number = 0; [RegEx]::Matches([System.IO.File]::ReadAllText($item.FullName), "(?<LogEntry>\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d.*)(?:(?:\r\n|[\r\n])(?!\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d).*)*(?:\r\n|[\r\n])?") | %{$_.Value} | %{$_.Insert(23, " " + $item.Name + " " + ($line_number++).ToString("0000000"))} | %{[RegEx]::Replace($_, "[\r\n]+$", "")} | Add-Content "res.txt" }

Second step… having merged all the log files into one it is necessary to sort them based on the timestamp by issuing the following command:

[RegEx]::Matches([System.IO.File]::ReadAllText("C:\Users\Ross\Desktop\logs\tmp\res.txt"), "(?<LogEntry>\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d.*)(?:(?:\r\n|[\r\n])(?!\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d).*)*(?:\r\n|[\r\n])?") | %{$_.Value} | %{[RegEx]::Replace($_, "[\r\n]+$", "")} | Sort | Add-Content "res2.txt"

Remember to modify the path provided to ReadAllText to point to the file generated in step 1. As with step 2, if a different timestamp format is used to mark the start of log lines then change the regex to match it.



  1. hcg shots

    Tuesday, January 24, 2012 - 04:27:38

    Good article! I’ve been looking thru yahoo for hours looking for appropriate info on this, they really need to position your site on the first page!

  2. Replica Watches

    Wednesday, January 23, 2013 - 08:08:15

    Remember to modify the path provided to ReadAllText to point to the file generated in step 1. As with step 2, if a different timestamp format is used to

  3. _____ SEO

    Thursday, May 9, 2013 - 01:44:23

    It’s a shame you don’t have a donate button!
    I’d definitely donate to this outstanding blog! I guess for now i’ll settle for book-marking and
    adding your RSS feed to my Google account. I look forward
    to fresh updates and will share this website with my Facebook group.
    Chat soon!

Add comment

Fill out the form below to add your own comments


Bookmark Site

Bookmark and Share

Tags

  Availability     Contracting     Perl     PHP     Technology     General     FlatPress     For Sale     C#  

Search

Admin area