从事件日志
问题描述:
解析出PowerShell中的CommandLine数据发送Windows事件日志与WinLogBeat到Logstash - 主要集中在日志中的PowerShell事件。从事件日志
例子:
<'Data'>NewCommandState=Stopped SequenceNumber=1463 HostName=ConsoleHost HostVersion=5.1.14409.1005 HostId=b99970c6-0f5f-4c76-9fb0-d5f7a8427a2a HostApplication=C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe EngineVersion=5.1.14409.1005 RunspaceId=bd4224a9-ce42-43e3-b8bb-53a302c342c9 PipelineId=167 CommandName=Import-Module CommandType=Cmdlet ScriptName= CommandPath= CommandLine=Import-Module -Verbose.\nishang.psm1<'/Data'>
如何使用神交得到以下我提取CommandLine=
场?
Import-Module -Verbose.\nishang.psm1
答
Grok是一个正则表达式的包装。如果你可以用正则表达式解析数据,你可以用grok来实现它。
尽管您的范围特定于CommandLine
字段,但解析大多数键=值日志中的每个字段都非常简单,并且可以为每个字段使用一个正则表达式,并使用一些grok过滤器。如果您打算存储,查询和可视化日志 - 数据越多越好。
正则表达式:
首先,我们先从以下内容:
(.*?(?=\s\w+=|\<|$))
-
.*?
- 匹配任何字符,除了行结束 -
(?=\s\w+=|\<|$))
- 正预测先行断言必须格局匹配以下-
\s\w+=
- 与在它之前的空间中,随后是=
-
|\<|$
任何单词字符 - 可替换地可以匹配<
或线路的端部,以便不包括它们的匹配组中使用。
-
这意味着,每场可以分析类似以下内容:
CommandLine=(.*?(?=\s\w+=|\<|$))
神交:
现在这意味着我们可以开始创建神交过滤器。它的强大之处在于可重用组件可能会将语义语言应用于它们。
/etc/logstash/patterns/powershell.grok:
# Patterns
PS_KEYVALUE (.*?(?=\s\w+=|\<|$))
# Fields
PS_NEWCOMMANDSTATE NewCommandState=%{PS_KEYVALUE:NewCommandState}
PS_SEQUENCENUMBER SequenceNumber=%{PS_KEYVALUE:SequenceNumber}
PS_HOSTNAME HostName=%{PS_KEYVALUE:HostName}
PS_HOSTVERSION HostVersion=%{PS_KEYVALUE:HostVersion}
PS_HOSTID HostId=%{PS_KEYVALUE:HostId}
PS_HOSTAPPLICATION HostApplication=%{PS_KEYVALUE:HostApplication}
PS_ENGINEVERSION EngineVersion=%{PS_KEYVALUE:EngineVersion}
PS_RUNSPACEID RunspaceId=%{PS_KEYVALUE:RunspaceId}
PS_PIPELINEID PipelineId=%{PS_KEYVALUE:PipelineId}
PS_COMMANDNAME CommandName=%{PS_KEYVALUE:CommandName}
PS_COMMANDTYPE CommandType=%{PS_KEYVALUE:CommandType}
PS_SCRIPTNAME ScriptName=%{PS_KEYVALUE:ScriptName}
PS_COMMANDPATH CommandPath=%{PS_KEYVALUE:CommandPath}
PS_COMMANDLINE CommandLine=%{PS_KEYVALUE:CommandLine}
凡%{PATTERN:label}
将利用PS_KEYVALUE
正则表达式,所述匹配组将与JSON该值进行标记。这是你可以灵活地命名你知道的领域。
/etc/logstash/conf.d/powershell。CONF:
input {
...
}
filter {
grok {
patterns_dir => "/etc/logstash/patterns"
break_on_match => false
match => [
"message", "%{PS_NEWCOMMANDSTATE}",
"message", "%{PS_SEQUENCENUMBER}",
"message", "%{PS_HOSTNAME}",
"message", "%{PS_HOSTVERSION}",
"message", "%{PS_HOSTID}",
"message", "%{PS_HOSTAPPLICATION}",
"message", "%{PS_ENGINEVERSION}",
"message", "%{PS_RUNSPACEID}",
"message", "%{PS_PIPELINEID}",
"message", "%{PS_COMMANDNAME}",
"message", "%{PS_COMMANDTYPE}",
"message", "%{PS_SCRIPTNAME}",
"message", "%{PS_COMMANDPATH}",
"message", "%{PS_COMMANDLINE}"
]
}
}
output {
stdout { codec => "rubydebug" }
}
结果:
{
"HostApplication" => "C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\powershell.exe",
"EngineVersion" => "5.1.14409.1005",
"RunspaceId" => "bd4224a9-ce42-43e3-b8bb-53a302c342c9",
"message" => "<'Data'>NewCommandState=Stopped SequenceNumber=1463 HostName=ConsoleHost HostVersion=5.1.14409.1005 HostId=b99970c6-0f5f-4c76-9fb0-d5f7a8427a2a HostApplication=C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\powershell.exe EngineVersion=5.1.14409.1005 RunspaceId=bd4224a9-ce42-43e3-b8bb-53a302c342c9 PipelineId=167 CommandName=Import-Module CommandType=Cmdlet ScriptName= CommandPath= CommandLine=Import-Module -Verbose.\\nishang.psm1<'/Data'>",
"HostId" => "b99970c6-0f5f-4c76-9fb0-d5f7a8427a2a",
"HostVersion" => "5.1.14409.1005",
"CommandLine" => "Import-Module -Verbose.\\nishang.psm1",
"@timestamp" => 2017-05-12T23:49:24.130Z,
"port" => 65134,
"CommandType" => "Cmdlet",
"@version" => "1",
"host" => "10.0.2.2",
"SequenceNumber" => "1463",
"NewCommandState" => "Stopped",
"PipelineId" => "167",
"CommandName" => "Import-Module",
"HostName" => "ConsoleHost"
}
非常感谢你不仅对答案,但您的解释! –