Here Grok, a filter plugin used in ELK stack. It converts unstructured logs to structured ones.
Grok is built upon regular expression. The syntax is %{SYNTAX:SEMANTIC}. For example:
A log line
10.0.1.13 GET /home HTTP/2.0 200 13969
A filter defined in logstash configure file as below
filter {
grok {
match => {
"message" => '%{IPORHOST:ip} \"%{WORD:method} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int})'
}
}
}
The log line passes through the filter, it produces the result
ip: 10.0.1.13
method: GET
request: /home
response: 200
bytes: 13969
With structured data, you can search by specific field while using Elasticsearch.
Grok Debugger
If you are not so sure the pattern would work on your log or not, here are some grok debugger to help you verify the patterns.
Make log processing easier. :D
See also
5 Logstash Filter Plugins
Oniguruma
The Jargon File, which describes itself as a "Hacker's Dictionary" and has been published under that name three times, puts grok in a programming context:
When you claim to "grok" some knowledge or technique, you are asserting that you have not merely learned it in a detached instrumental way but that it has become part of you, part of your identity. For example, to say that you "know" Lisp is simply to assert that you can code in it if necessary — but to say you "grok" LISP is to claim that you have deeply entered the world-view and spirit of the language, with the implication that it has transformed your view of programming. Contrast zen, which is a similar supernatural understanding experienced as a single brief flash.
--Wikipedia
Top comments (0)