CentOS下安装Logstash（附带示例）

本文记录一下在CentOS 6.7上，安装Logstash，版本为logstash-2.4.0.tar.gz。

Logstash是一个开源的日志管理工具。

下载安装包

使用wget命令下载logstash安装包，如

[root@dev18 srv]# wget https://download.elastic.co/logstash/logstash/logstash-2.4.0.tar.gz--2017-03-17 16:37:14--  https://download.elastic.co/logstash/logstash/logstash-2.4.0.tar.gzResolving download.elastic.co... 107.22.208.105, 54.243.211.74, 107.21.249.70, ...Connecting to download.elastic.co|107.22.208.105|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 83882952 (80M) [application/x-gzip]Saving to: “logstash-2.4.0.tar.gz”100%[====================================================================================================================================================================================================================================>] 83,882,952  7.90M/s   in 1m 54s  2017-03-17 16:39:10 (721 KB/s) - “logstash-2.4.0.tar.gz” saved [83882952/83882952][root@dev18 srv]#

解压

使用tar -zvxf解压缩Logstash，如：

[root@dev18 srv]# tar -zvxf logstash-2.4.0... ...logstash-2.4.0/vendor/jruby/lib/ruby/shared/securerandom.rblogstash-2.4.0/vendor/jruby/lib/ruby/shared/syslog.rblogstash-2.4.0/vendor/jruby/lib/ruby/shared/tempfile.rblogstash-2.4.0/vendor/jruby/lib/ruby/shared/tmpdir.rblogstash-2.4.0/vendor/jruby/lib/ruby/shared/ubygems.rblogstash-2.4.0/vendor/jruby/toollogstash-2.4.0/vendor/jruby/tool/nailgunlogstash-2.4.0/vendor/jruby/tool/nailgun/Makefile.inlogstash-2.4.0/vendor/jruby/tool/nailgun/README.txtlogstash-2.4.0/vendor/jruby/tool/nailgun/configurelogstash-2.4.0/vendor/jruby/tool/nailgun/ng.exelogstash-2.4.0/vendor/jruby/tool/nailgun/srclogstash-2.4.0/vendor/jruby/tool/nailgun/src/clogstash-2.4.0/vendor/jruby/tool/nailgun/src/c/ng.clogstash-2.4.0/vendor/bundle/jruby/1.9/gems/jrjackson-0.3.9-java/.mvn/extensions.xmllogstash-2.4.0/vendor/bundle/jruby/1.9/gems/ruby-maven-3.3.12/.mvn/extensions.xmllogstash-2.4.0/Gemfilelogstash-2.4.0/Gemfile.jruby-1.9.lock

将解压后的logstash-2.4.0目录名改成logstash

[root@dev18 srv]# mv logstash-2.4.0 logstash

进入logstash目录，查看logstash目录下有哪些目录和文件~

[root@dev18 srv]# cd logstash[root@dev18 logstash]# lltotal 160drwxr-xr-x 2 root root   4096 Mar 17 16:39 bin-rw-rw-r-- 1 root root 102879 Aug 30  2016 CHANGELOG.md-rw-rw-r-- 1 root root   2249 Aug 30  2016 CONTRIBUTORS-rw-rw-r-- 1 root root   4976 Aug 30  2016 Gemfile-rw-rw-r-- 1 root root  22850 Aug 30  2016 Gemfile.jruby-1.9.lockdrwxr-xr-x 4 root root   4096 Mar 17 16:39 lib-rw-rw-r-- 1 root root    589 Aug 30  2016 LICENSE-rw-rw-r-- 1 root root    149 Aug 30  2016 NOTICE.TXTdrwxr-xr-x 4 root root   4096 Mar 17 16:39 vendor[root@dev18 logstash]#

启动Logstash

进入<Logstash_HOME>/bin目录，

[root@dev18 logstash]# cd bin[root@dev18 bin]# lltotal 44-rwxrwxr-x 1 root root 1854 Aug 30  2016 logstash-rw-rw-r-- 1 root root  689 Aug 30  2016 logstash.bat-rwxrwxr-x 1 root root 5330 Aug 30  2016 logstash.lib.sh-rwxrwxr-x 1 root root  439 Aug 30  2016 logstash-plugin-rw-rw-r-- 1 root root  251 Aug 30  2016 logstash-plugin.bat-rwxrwxr-x 1 root root  199 Aug 30  2016 plugin-rw-rw-r-- 1 root root  203 Aug 30  2016 plugin.bat-rwxrwxr-x 1 root root  322 Aug 30  2016 rspec-rw-rw-r-- 1 root root  245 Aug 30  2016 rspec.bat-rw-rw-r-- 1 root root 2947 Aug 30  2016 setup.bat

使用logstash工具即可启动Logstash~~如：

[root@dev18 bin]# ./logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'

示例

Logstash具有较为丰富的输入（input），过滤（filter）以及输出（output）插件。

本文给出两个示例，分别为

1. 标准输入输出
2. 标准输入Redis输出

下面就逐个给出示例~ Here we go~

标准输入输出

在这个示例中，使用最简单的控制台输入（stdin）和控制台输出（stdout），启动命令如下：

[root@dev18 bin]# ./logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'

查看logstash是否正常启动

[root@dev18 ~]# ps -ef|grep logstashroot      1352  1168 14 16:51 pts/0    00:00:39 /usr/java/jdk1.7.0_71/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Xmx1g -Xss2048k -Djffi.boot.library.path=/srv/logstash/vendor/jruby/lib/jni -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/srv/logstash/heapdump.hprof -Xbootclasspath/a:/srv/logstash/vendor/jruby/lib/jruby.jar -classpath :.:/usr/java/jdk1.7.0_71/jre/lib/rt.jar:/usr/java/jdk1.7.0_71/lib/dt.jar:/usr/java/jdk1.7.0_71/lib/tools.jar -Djruby.home=/srv/logstash/vendor/jruby -Djruby.lib=/srv/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /srv/logstash/lib/bootstrap/environment.rb logstash/runner.rb agent -e input{stdin{}}output{stdout{codec=>rubydebug}}root      1448  1423  0 16:56 pts/2    00:00:00 grep logstash[root@dev18 ~]#

控制台输入hello logstash，然后看一下输出~~ :）

[root@dev18 bin]# ./logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'hello logstash{       "message" => "hello logstash",      "@version" => "1",    "@timestamp" => "2017-03-17T08:53:11.975Z",          "host" => "dev18.xxxx.xxxx"}

标准输入Redis输出

在这个示例中，使用控制台输入（stdin），使用Redis订阅作为输出(stdout)~~

因为需要指定Redis的属性，内容较多，所以不像第一个示例那样使用./logstash -e 来启动，本示例采用指定配置文件的方式来启动~

首先，在Logstash安装目录下，创建myconf目录，该目录用于存放配置文件~如：

[root@dev18 logstash]# mkdir myconf[root@dev18 logstash]# lltotal 164drwxr-xr-x 2 root root   4096 Mar 17 16:39 bin-rw-rw-r-- 1 root root 102879 Aug 30  2016 CHANGELOG.md-rw-rw-r-- 1 root root   2249 Aug 30  2016 CONTRIBUTORS-rw-rw-r-- 1 root root   4976 Aug 30  2016 Gemfile-rw-rw-r-- 1 root root  22850 Aug 30  2016 Gemfile.jruby-1.9.lockdrwxr-xr-x 4 root root   4096 Mar 17 16:39 lib-rw-rw-r-- 1 root root    589 Aug 30  2016 LICENSEdrwxr-xr-x 2 root root   4096 Mar 17 19:20 myconf-rw-rw-r-- 1 root root    149 Aug 30  2016 NOTICE.TXTdrwxr-xr-x 4 root root   4096 Mar 17 16:39 vendor

然后，创建一个配置文件，名字为 stdin2redis.conf

[root@dev18 logstash]# cd myconf/[root@dev18 myconf]# vim stdin2redis.conf

stdin2redis.conf文件内容如下：

input {    stdin { }}output {    # 输出到控制台    # stdout { }    # 输出到redis    redis {        host => "172.xx.xx.xxx"   # redis主机地址        port => 6379              # redis端口号        db => 0                   # redis数据库编号        data_type => "channel"    # 使用发布/订阅模式        key => "logstash_channel"  # 发布通道名称    }}

指定配置文件，启动Logstash

[root@dev18 bin]# ./logstash -f ../myconf/stdin2redis.conf Settings: Default pipeline workers: 2Pipeline main started

打开Redis客户端，订阅logstash_channel

​[root@dev18 src]# ./redis-cli 127.0.0.1:6379> SUBSCRIBE logstash_channelReading messages... (press Ctrl-C to quit)1) "subscribe"2) "logstash_channel"3) (integer) 1​

测试，在控制台分别输出三组字符串，分别为“hello logstash”, "hello java" 以及"hello china"

[root@dev18 bin]# ./logstash -f ../myconf/stdin2redis.conf Settings: Default pipeline workers: 2Pipeline main startedhello logstashhello javahello china

我们可以看到，Redis客户端显示订阅的内容~

127.0.0.1:6379> SUBSCRIBE logstash_channelReading messages... (press Ctrl-C to quit)1) "subscribe"2) "logstash_channel"3) (integer) 11) "message"2) "logstash_channel"3) "{"message":"hello logstash","@version":"1","@timestamp":"2017-03-17T11:39:28.884Z","host":"dev18.gzhl.zhhl"}"1) "message"2) "logstash_channel"3) "{"message":"hello java","@version":"1","@timestamp":"2017-03-17T11:39:49.131Z","host":"dev18.gzhl.zhhl"}"1) "message"2) "logstash_channel"3) "{"message":"hello china","@version":"1","@timestamp":"2017-03-17T11:39:53.042Z","host":"dev18.gzhl.zhhl"}"

本文就写到这边，鉴于Logstash具有丰富的输入和输出，后续慢慢玩~~