CheckEventLog

CheckEventLog is part of the wiki:CheckEventLog module. This page describes the new syntax, for the old syntax refer to the old page: CheckEventLogOld The new syntax is a bit sketchy in the docs as of yet... I shall try to fix some better examples.. but the best idea would be for someone that uses this to help me with that :)

Before you start using CheckEventLog use this command (it is long but a good place to start):

CheckEventLog file=application file=system filter=new filter=out
      MaxWarn=1 MaxCrit=1 
      filter-generated=>2d filter-severity==success filter-severity==informational 
      truncate=1023 unique descriptions "syntax=%severity%: %source%: %message% (%count%)"

This check enumerates all event in the event log and filters out (or in) events and then the resulting list is used to determine state.

Option	Values	Description
file	An event log file name(application, security, system, etc.)	The name of an eventlog file the default ones are Application, Security and System. If the specified eventlog was not found due to some idiotic reason windows opens the "application" log instead.
filter	in, out, any, all	Specify the way you want to filter things. (See section below)
filter	new	Has to be set to use this syntax
descriptions	None	Flag to specify if you want to include string representation of the error messages.
truncate	length of the returned set	This will truncate the output after the specified length. As NRPE can only handle 1024 chars you need to truncate the output.
MaxWarn	number of records	The maximum records to allow before reporting a warning state.
MaxCrit	number of records	The maximum records to allow before reporting a critical state.
syntax	String	A string to use to represent each matched eventlog entry the following keywords will be replaced with corresponding values: %source%, %generated%, %written%, %type%, %severity%, %strings%, %id% and %message% (%message% requires you to set the description flag.) %count% (requires the unique flag) can be used to display a count of the records returned.
filter<mode><type>	<filter value>	A number of strings to use for filtering the event log
unique		Flag to indicate unique filtering is used.

The CheckEventLog uses filters to define the "interesting" records from the eventlog.

Syntax

A filter is made up of three things:

• Filter mode Determines what happens when the filter is matched.
• Filter type What the filter will match (ie. which field).
• An Expression What to check for.

The syntax of a filter is: filter<mode><type>=<expression>

Order

Order is important, as soon as a positive (+) or negative (-) rule is matched it is either discarded or included and the entry is "finished" and it will continue with the next entry. The best way here is to have an "idea" either remove all entries first or include all required ones first (depending on what you want to do). You can mix and such but this will probably complicate things for you unless you actually need to.

Filter modes

Capturing eventlog entries (or discarding them) are done with filters. There are three kinds of filters.

<filter mode>	title	description
+	positive requirements	All these filters must match or the row is discarded.
.	potential matches	If this matches the line is included (unless another lines overrides).
-	negative requirements	None of these filters can match (if any do the row is discarded).

Thus if you want to have: all errors and entries from the last month but not the ones from the cdrom, but if the source is MyModule? get everything. I would break this down as such: (notice there are other options). + type=error - date=older than 2 months . source=MyModule? This would pick up all errors, and drop all old records and then pickup all remaining "MyModule?" records (in this case you could have used + on the source filter since there are no more rules).

other example to simplify it: if for example you want to monitor all errors and to ignore warning and success in the eventlog you can write the following: filter+severity==error filter-severity==success filter-severity==informational

and the command with those parameters with others can be like the following: CheckEventLog file=application file=system filter=new filter=out MaxWarn=1 MaxCrit=1 filter-generated=>2d filter+severity==error filter-severity==success filter-severity==informational truncate=1023 unique descriptions "syntax=%severity%: %source%: %message% (%count%)"

Filter Types

<filter type>	Values	Description
eventType	event type expression	An event type to filter out: error, warning, info, auditSuccess or auditFailure. Note that unlike other commands, this requires '==', for example filter-eventType==info. The info,error, etc are all case sensitive.
eventSource	string-expression	The name of the source of the event. Can be a substring or regularexpression
generated	time-expression	Time ago the message was generated
written	time-expression	Time ago the message was written to the log
message	string-expression	Filter strings in the message. Can be a substring or regularexpression
eventID	numeric-expression	Filter based on the event id of the log message.
severity	event severity expression	Filter based on event severity. (filter-severity==warning)

event type expression

An event type expression is similar to a numeric-expression but instead of a number a "keyword" is taken: error, warning, info, auditSuccess, auditFailure. So filter.eventType==warning or filter.eventType=<>warning are examples of event type expressions. Yes this is correct the syntax is: filter<mode><type>=<expression> in this case <mode> is ".", <type> is "eventType" and <expression> is "<>warning". This IS confusing but it is "simpler to parse" some day maybe I shall improve this.

filter<key><

event severity expression

An event severity expression is similar to a numeric-expression but instead of a number a "keyword" is taken: success, informational, warning or error

time expression

A time expression is a date/time interval as a number prefixed by a filter prefix (<, >, =, <>) and followed by a unit postfix (m, s, h, d, w). A few examples of time expression are: filter+generated=>2d means filter will match any records older than 2 days, filter+generated=<2h means match any records newer then 2 hours. Warning, the bash interprets the "<,>,!". Use the "\" to avoid this. e.g. filter+generated=\>2d . On the Client activate the "Nasty Metachars" Option, to allow the \.

string expression

A string expression is a key followed by a string that specifies a string expression. Currently substr and regexp are supported. Thus you enter filter.message=regexp:(foo|bar) to enter a regular expression and filter-message=substr:foo to enter a substring patter match.

Filter in/out

There are two basic ways to filter:

• in When you filter in it means all records matching your filter will be returned (the "simplest way")
• out When you filter out it means all records matching your filter will be discarded.

So:

filter=in filter+eventType==warning
...
filter=out filter-eventType==warning

Will both have the same effect as the first one filters "in" and matches all warnings and the second one filters out and discards all warnings. There is one very fundamental difference though the first one will only return the warnings where as the second one will return all entries and all warnings.

Unique

When unique is present any duplicate entries matching the filter will be discarded so you will only get back one of each "kind" of error. Uniqueness is determined by log-file, event-id, event-type and event-category.

Examples

Sample Eventlog Command

Check by EventID for target errors that may have transpired over the past 2 hours.
$ARG1$ = file to check ie. Application, Security, System
$ARG2$ = Max Warn amount
$ARG3$ = Max Critical amount
$ARG4$ = eventID Number

Sample Command:

CheckEventLog filter=new file="application" MaxWarn=10 MaxCrit=20 filter-generated=\<2h filter-eventID=="12345" filter-eventType==error filter=in filter=all

OK: ...

Nagios Configuration:

define command {
  command_name <<CheckEventLog>>
  command_line check_nrpe -H $HOSTADDRESS$ -p 5666 -c CheckEventLog -a filter=new file="$ARG1$" MaxWarn=$ARG2$ MaxCrit=$ARG3$ filter-generated=\<2h filter-eventID=="$ARG4$" filter-eventType==error filter=in filter=all
}
<<CheckEventLog>> application!10!20!12345

From Commandline (with NRPE):

check_nrpe -H IP -p 5666 -c CheckEventLog -a filter=new file="application" MaxWarn=10 MaxCrit=20 filter-generated=\<2h filter-eventID=="12345" filter-eventType==error filter=in filter=all

Another sample

Check the Application event log for errors over the past 48 hours. Filter out any Cdrom and NSClient Errors as well as all Warnings. Allow 3 target Errors before firing a Warning, and 7 Errors before firing a Critical State.

This is the corresponding command: Sample Command:

CheckEventLog filter=new file=system file=application MaxWarn=1 MaxCrit=1 filter-generated=>2d filter-eventSource=substr:Service filter-eventSource=substr:Tcpip

CRITICAL: 27 > critical: ESENT

Nagios Configuration:

define command {
  command_name <<CheckEventLog>>
  command_line check_nrpe -H $HOSTADDRESS$ -p 5666 -c CheckEventLog -a filter=new file=system file=application MaxWarn=1 MaxCrit=1 filter-generated=>2d filter-eventSource=substr:Service filter-eventSource=substr:Tcpip
}
<<CheckEventLog>> 

From Commandline (with NRPE):

check_nrpe -H IP -p 5666 -c CheckEventLog -a filter=new file=system file=application MaxWarn=1 MaxCrit=1 filter-generated=>2d filter-eventSource=substr:Service filter-eventSource=substr:Tcpip

Please note: You need to allow_nasty_meta_chars=1 in the NSC.ini to use time filters like "<2d" (last 48 hours).

Check if a script is running as it should

Just to show a 'hidden' parameter ... I check that a script has successfully finished by writing into the eventlog. If after 1 day there is no new log entry, I get the message from Nagios.

Sample Command:

CheckEventLog filter=new file=application MinWarn=0 MinCrit=0 filter-generated=\>1d filter+eventSource="pdf_creation" filter+eventType==auditSuccess filter=in filter=all

OK: ...

Nagios Configuration:

define command {
  command_name <<CheckEventLog>>
  command_line check_nrpe -H $HOSTADDRESS$ -p 5666 -c CheckEventLog -a filter=new file=application MinWarn=0 MinCrit=0 filter-generated=\>1d filter+eventSource="pdf_creation" filter+eventType==auditSuccess filter=in filter=all
}
<<CheckEventLog>> 

From Commandline (with NRPE):

check_nrpe -H IP -p 5666 -c CheckEventLog -a filter=new file=application MinWarn=0 MinCrit=0 filter-generated=\>1d filter+eventSource="pdf_creation" filter+eventType==auditSuccess filter=in filter=all

Don't understand filtering ?

Yes I know, there are a lot of options regarding filtering, and they are a bit hard understand. This section tries to give a more formal definition of what the various options do (form a programming perspective).

Options

There are three different option pairs all used with the same key filter

1. filter=new / filter=old This one decides the code which is used there are two completely different concepts and the "Old" one is preferably not to be used as it is slightly less sane.
2. filter=all / filter=any This is NOT used with filter=new so this option is deprecated and should not be used unless you are using filter=old.
3. filter=in / filter=out This option decides what happens if "nothing matches" if you have filter=in that means if nothing matches you will still get the option where as if you have filter=out you wont. So this is the last thing that happens in the code.

So what you end up with is:

• filter=new this is default in newer versions, so you don't really need this anymore.
• filter=old Should not be used any more.
• filter=any is not used any more.
• filter=all is not used any more.
• filter=in Means you want everything except "something"
• filter=out Means you only want "something"

Rules

There are three different filter rule types all used in the same way except swapping the - for + for .

1. filter+severity==error This means that any entries matching this rule is automatically included in the result. After this we instantly stop matching more rules for this entry.
2. filter-severity==error This means that any entries matching this rule is automatically excluded in the result. After this we instantly stop matching more rules for this entry.
3. filter.severity==error This means that any entries matching this rule is neither automatically excluded nor automatically included in the result. But after this rule we continue matching more more rules for this entry. Since this is the default behavior with filter=in there is no reason to use filter=. in this mode.

How it works

Pseudo code (filter=new)

This is how the filtering is decided:

• bFilterIn=true for filter=in and false for filter=out

bool bMatch = !bFilterIn;
for(<each rule>) {
	bTmpMatched=<result of rule evaluation>
	if ((mode == filter_minus)&&(bTmpMatched)) {
		// a -<filter> hit so thrash item and bail out!
		bMatch = false;
		break;
	} else if ((mode == filter_plus)&&(!bTmpMatched)) {
		// a +<filter> missed hit so thrash item and bail out!
		bMatch = false;
		break;
	} else if (bTmpMatched) {
		bMatch = true;
	}
}
if (bMatch) {
	<deciding factor> = true;
}