centos+rsyslog+loganalyzer

Rsyslog+Analyzer搭建

一个项目上需求，需要内部网络上设备的日志存储满足180天，且支持日志报表导出。于是想到搭建一个日志应用。再加上日志报表分析，想到一个组合，rsyslog+loganalyzer。记录备忘。

操作如下：

系统环境

CentOS Linux release 7.6.1810 (Core)

安装步骤

设置时间同步

1
2
3
yum install ntpdate epel-release vim curl wget
ntpdate cn.ntp.org.cn
hwclock -w

配置crontab任务

1
2
3
vim /etc/crontab
00 23 * * * ntpdate cn.ntp.org.cn
30 00 * * * hwclock -w

安装apache、php、mariadb-server

1
yum install php php-mysql php-gd php-mbstring php-mcrypt php-devel php-xml mariadb-server httpd

设置开机启动

1
2
3
4
systemctl enable httpd
systemctl enable mariadb
systemctl start httpd
systemctl start mariadb

配置mariadb-server

1
mysql_secure_installation

安装rsyslog-mysql

1
yum install rsyslog rsyslog-mysql -y

创建rsyslog数据库和用户

导入rsyslog数据库

1
mysql -u root -p < /usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql

创建rsyslog用户，并授权

1
2
3
4
mysql -u root -p
mysql> GRANT ALL ON  Syslog.* TO 'rsyslog'@'localhost' IDENTIFIED BY 'MYSQLPASSOWRD';
mysql> FLUSH PRIVILEGES;
mysql> exit;

设置rsyslog-mysql

修改rsyslog配置文件，启用rsyslog mysql 模块，配置mysql 连接信息

1
2
3
4
vim /etc/rsyslog.conf
# 添加下面的信息
$ModLoad ommysql
*.* :ommysql:127.0.0.1,Syslog,rsyslog,MYSQLPASSWORD

最终如下所示：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
$ModLoad imklog # reads kernel messages (the same are read from journald)
$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
# 新启用的rsyslog mysql模块
$Modload ommysql
*.* :ommysql:localhost,Syslog,rsyslog,MYSQLPASSWORD


#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on

# File to store the position in the journal
$IMJournalStateFile imjournal.state


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

重新启用rsyslog服务

1
systemctl restart rsyslog

至此，rsyslog的日志存储已经可以在mysql中查看，接下来我们还需要对存储的日志可视化操作。即安装loganalyzer。

安装loganalyzer

下载

1
2
3
cd ~/download
wget http://download.adiscon.com/loganalyzer/loganalyzer-4.1.7.tar.gz
tar -xzf loganalyzer-4.1.7.tar.gz

把安装源文件复制到本地站点目录

1
cp -r /loganalyzer-4.1.7/src /var/www/html/loganalyzer

创建loganalyzer的配置文件

创建配置文件的时候，先把selinux关闭，不然后面写入配置文件的时候会有问题。

1
2
3
4
cd /var/www/html/loganalyzer
touch config.php
chown apache:apache ./*
chmod 666 config.php

安装引导

打开安装引导页面

1
http://server-ip/loganalyzer/install.php

根据引导页面开始安装，安装过程中，如果遇到config.php的问题，参考selinux配置，即需要关闭，重启后操作。

连接数据库后，有一个配置创建第一个rsyslog信息源。注意表名SystemEvents这个表名，不确定的，可以登录mysql查看Syslog表名确认。不然这里的创建后面登录后，查不到日志信息。

引导页面安装结束，即可以看到日志。

如果查询不到日志，提示以下错误：

1
“Could not find the configured table, maybe misspelled or the table names are case sensitive”

解决方法如下：

更改loganalyzer 的config.php文件，更改如下行内容：

1
$CFG[‘Sources’][‘Source1’][‘DBTableName’] = ‘systemevents’; => ‘SystemEvents’

更新数据库

1
2
3
4
mysql -u rsyslog -p
mysql> update logcon_sources set DBTableName =’SystemEvents’ where ID=1;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

配置其它主机的日志

添加一台主机，把日志存储到laganalyzer主机上，只需要配置主机的rsyslog配置文件，配置文件如下例：

1
2
3
4
5
6
7
8
9
10
11
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
*.* @loganalyzer-IP:514
# ### end of the forwarding rule ###

开放loganalyzer服务器的514端口

1
2
3
firewall-cmd --permanent --add-port=514/tcp
firewall-cmd --permanent --add-port=514/udp
firewall-cmd --complete-reload

loganzlyzer

linux
#centos #rsyslog
centos+rsyslog+loganalyzer
https://ywmy.xyz/2019/08/07/centos-rsyslog-loganalyzer/
作者
ian
发布于
2019年8月7日
许可协议