Windows Event Logs

Windows Event Logs

Plain text files are most commonly used to log software application events such as exceptions, warnings, information or verbose data to try identify a specific problem. Although using text files can have a performance impact there are tools optimized to handle this kind of logging such as log4net or serilog. These work very well and many developers are fans of them.

I’m a fan of windows event logs as they are optimized by the operating system and I can view logs in the same place. That said, tools like logstash can parse and transform a variety of logs such as windows event logs or text logs to be viewed in once place, so it just depends on your organizations setup and maturity in terms of logging infrastructure.

KIBANA is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster.

Developers generally should not have access to production servers but the operations team can extract and provide the exported windows event viewer logs (.evtx files)

It is possible to open these logs with the built in windows “Event Viewer” but the ‘Find…’ function is in my opinion not very helpful when we are trying to identify a pattern and/or display records based on custom criteria.

I like to import the logs into a database and then run SQL querys to identify patterns or specific logs that are of importance to my problem domain. My ‘EventViewer’ solution was written in C# .NET and is made up of the following projects: EvtxImporter, LogViewWebApplication, Common, Repository, Sharedmodels and Tests.

EvtxImporter

This is a console application that simply reads the .evtx file to memory and bulk inserts the data to our database. The data is not normalized and really should be however for simplicity I’ve imported it ‘as is’ so the SQL columns matches the columns in the .evtx file and is mapped to SharedModels.EventLogModel

The high level process flow is:

  1. Check file location ‘LogPath’ for ‘*.evtx’ files
  2. Check if the files have been processed by looking at a watermark file hamster.json, this is stored in the location WaterMarkFile’
  3. Read the file and map it to SharedModels.EventLogModel
  4. There was no count property so I manually counted the file records by looping twice using a delegate to either call my ‘Count’ or ‘Append’ methods
  5. The data is then bulk inserted
  6. The watermark is then updated

LogViewWebApplication

This is a simple MVC Web Application to read the log files, this could be used by your help desk as they will either not know how to construct SQL querys or care about them.

Common

This is a class library for common classes.

CategoryModel can be used to identify software events using the built in .NET ‘EventLog.WriteEntry’ method.

Short range -32,768 to 32,767 can be used to identify logs from parts of the application. This can be anything from the users Id to values representing classes withing the application.

In my class I have a ‘Categorys’ enum with values UserLogin=1, DebtorsCapture, CreditorsCapture which I use in my ‘Tests’ project. They can be anything really.

The ‘EventLogger’ class is used to write to the windows event log.

Repository

This is a normal repository pattern class library with a PostgreSQL implementation, for more information on a repository pattern see this article.

SharedModels

Im on the fence with sharing data models as this can be messy but I guess as long as your data models (repository) and view models (mvc models) are seprated you will be ok.

The big advanage for my application was to share ‘EventLogModel’ between ‘EvtxImporter’ and ‘Repository’ as this model exactly represents the .evtx file which is read and persisted to the database.

Tests

The general rule is every class you write needs a unit test - this is often not done for several reasons including time, business requirements/policys and simply overkill. My import tool as 2 test groups.

‘EventLoggerTests’ to create ‘dummy’ event logs to test with. This is not a true unit test but the ability to quickly run specifc classes and insert some test data was very useful.

‘ReadEvtxFileTests’ to test reading .evtx files.

Tests for the ‘Repository’ and ‘EvtxImporter.CheckFileLocation’ are still to come! \ :D /

References