Feb 162014
 

Edit: This post is pretty old and Elasticsearch/Logstash/Kibana have evolved a lot since it was written.

Part 3 of 4 – Part 1Part 2Part 4
This is a continuation of http://www.ragingcomputer.com/2014/02/logstash-elasticsearch-kibana-for-windows-event-logs

Again, I took a lot of inspiration from http://sysxfit.com/blog/2013/07/18/logging-with-logstash-part-3/

The nxlog reference manual is surprisingly well written with excellent examples.
http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.pdf

Loggly has some examples I found useful, even if I’m not using their service.
http://community.loggly.com/customer/portal/articles/1266344-nxlog-windows-configuration
https://www.loggly.com/docs/logging-from-windows/

There are other options.
http://www.canopsis.org/2013/05/windows-eventlog-snare-logstash/
http://docs.fluentd.org/articles/windows
http://cookbook.logstash.net/recipes/log-shippers/
http://cookbook.logstash.net/recipes/windows-service/

INSTALL NXLOG
Download and run the windows installer. This is a very fast install.
http://sourceforge.net/projects/nxlog-ce/files/

Edit your nxlog.conf. Its location will depend on your OS.

32bit OS

C:\Program Files\nxlog\conf\nxlog.conf

64bit OS

Advertisement:
C:\Program Files (x86)\nxlog\conf\nxlog.conf

Note: You will need to modify the Define ROOT depending on 32bit or 64bit install.
Note: You will need to modify the input eventlog section depending on windows version.
Note: You will need to modify Host in the Output section to the IP address or hostname of you logstash computer.

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
 
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
 
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
 
<Extension json>
    Module      xm_json
</Extension>
 
# Nxlog internal logs
<Input internal>
   Module im_internal
   Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Input>
 

# Windows Event Log
<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later 
   Module im_msvistalog
 
# Uncomment im_mseventlog for Windows XP/2000/2003
#   Module im_mseventlog

   Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Input>

<Output out>
   Module om_tcp
   Host 192.168.1.126
   Port 3515
</Output>

<Route 1>
   Path internal, eventlog => out
</Route>

START NXLOG SERVICE

Finally, start the service. Either open computer management, open services, find nxlog in the list and start or from an administrator command prompt

net start nxlog

I’m kinda lazy and doing repetitive tasks by hand isn’t my cup of tea, so since I had 20 identical machines to install this on, I whipped up this bat file for installing it while I remotely connected for other maintenance.

install-nxlog.bat

@echo off
echo installing nxlog
msiexec /passive /i "\\shareserver\sharename\path\to\nxlog\nxlog-ce-2.6.1131.msi"
echo copying configuration
move "C:\Program Files\nxlog\conf\nxlog.conf" "C:\Program Files\nxlog\conf\nxlog.conf.default"
copy "\\shareserver\sharename\path\to\nxlog\nxlog.conf" "C:\Program Files\nxlog\conf\nxlog.conf"
echo starting service
net start nxlog
echo done

  7 Responses to “Sending Windows Event Logs to Logstash / Elasticsearch / Kibana with nxlog”

  1. […] In here, you can find very good example about how to configure properly the tool and how to connect the NxLog service to Logstash for feeding the ElasticSearch engine properly. Find also here the NxLog reference manual, This document is very well written and contains excellent examples that will help you to configure your NxLog instance. […]

  2. Hello, I deploy the same enviroment base on your instruction, but whe I started nxlog I see the following in the log file:
    2014-07-02 18:03:39 INFO connecting to 10.240.48.53:3515
    2014-07-02 18:03:39 INFO nxlog-ce-2.7.1191 started
    But any records haven’t copied to logstash.
    How to debug nxlog or maybe receive more detail logs.

  3. i am seeing this

    ERROR couldn’t connect to tcp socket on 192.168.1.22:3515; No connection could be made because the target machine actively refused it.

    any thoughts ?

  4. Hi,
    I have the same issue as Mike, Any thoughts?

  5. Das, Mike

    If the machine is actively refusing it… I’d check that your logstash service is listening on that port on that host and there is not a firewall blocking communication.

    Things to check:
    Logstash service is running
    Logstash listening port is correct
    Logstash listening interface is correct
    Logstash is allowed through machine firewall
    nxlog config is pointed to the correct host
    nxlog config is pointed to the correct port

  6. Nice document any chance you can do a revision to to get more current with some of the products

    it seems some of the links in your post have expired or are not available

    Thanks. Alex

  7. Wonderful document. A unique and indispensable guide. I followed it to the “t”.
    But I cannot get the hostnames of the windows machines in the logs, I am testing this at home on windows 7/8, at work I need to implement this for our PDC’s.

    These are the messages I am getting(not very explanatory):

    September 30th 2015, 06:03:49.685 message: @version:1 @timestamp:September 30th 2015, 06:03:49.685 host:%{host2} type:WindowsEventLog tags:[“_grokparsefailure”,”_jsonparsefailure”] FileName: source_host: eventlog_severity: AccountName: eventlog_channel: EventType: Hostname: Severity: _source:{“message”:”\u0000\b\u0000\u0006\u0000\u0017\u0000\u0018\u0000\u0019\u0000\u000b\u0000\u0002\u0001\u0000\u0000\r\u0000″,”@version”:”1″,”@timestamp”:”2015-09-30T11:03:49.685Z”,”host”:”%{host2}”,”type”:”WindowsEventLog”,”tags”:[“_grokparsefailure”,”_jsonparsefailure”],”FileName”:null,”source_host”:null,”eventlog_severity”:null,”AccountName”:null,”eventlog_channel”:null,”EventType”:null,”Hostname”:null,”Severity”:null} _id:AVAd7DtTaXmKBthRAE0K _type:WindowsEventLog _index:logstash-2015.09.30

     September 30th 2015, 06:03:49.685 message: @version:1 @timestamp:September 30th 2015, 06:03:49.685 host:%{host2} type:WindowsEventLog tags:[“_grokparsefailure”,”_jsonparsefailure”] FileName: source_host: eventlog_severity: AccountName: eventlog_channel: EventType: Hostname: Severity: _source:{“message”:”\u0000\b\u0000\u0006\u0000\u0017\u0000\u0018\u0000\u0019\u0000\u000b\u0000\u0002\u0001\u0000\u0000\r\u0000″,”@version”:”1″,”@timestamp”:”2015-09-30T11:03:49.685Z”,”host”:”%{host2}”,”type”:”WindowsEventLog”,”tags”:[“_grokparsefailure”,”_jsonparsefailure”],”FileName”:null,”source_host”:null,”eventlog_severity”:null,”AccountName”:null,”eventlog_channel”:null,”EventType”:null,”Hostname”:null,”Severity”:null} _id:AVAd7EdbaXmKBthRAF6e _type:WindowsEventLog _index:logstash-2015.09.30

     September 30th 2015, 06:03:49.685 message: @version:1 @timestamp:September 30th 2015, 06:03:49.685 host:%{host2} type:WindowsEventLog tags:[“_grokparsefailure”,”_jsonparsefailure”] FileName: source_host: eventlog_severity: AccountName: eventlog_channel: EventType: Hostname: Severity: _source:{“message”:”\u0000\b\u0000\u0006\u0000\u0017\u0000\u0018\u0000\u0019\u0000\u000b\u0000\u0002\u0001\u0000\u0000\r\u0000″,”@version”:”1″,”@timestamp”:”2015-09-30T11:03:49.685Z”,”host”:”%{host2}”,”type”:”WindowsEventLog”,”tags”:[“_grokparsefailure”,”_jsonparsefailure”],”FileName”:null,”source_host”:null,”eventlog_severity”:null,”AccountName”:null,”eventlog_channel”:null,”EventType”:null,”Hostname”:null,”Severity”:null} _id:AVAd7FePaXmKBthRAHj8 _type:WindowsEventLog _index:logstash-2015.09.30

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)