Heka Filter afd example: различия между версиями
Материал из noname.com.ua
Перейти к навигацииПерейти к поискуSirmax (обсуждение | вклад) |
Sirmax (обсуждение | вклад) |
||
Строка 15: | Строка 15: | ||
===Cirular buffer=== |
===Cirular buffer=== |
||
Heka's LUA SandBox has bilt-in Circular Buffer Library ehich can be used for data aggregation. |
Heka's LUA SandBox has bilt-in Circular Buffer Library ehich can be used for data aggregation. |
||
+ | <BR>More details: |
||
+ | * https://github.com/mozilla-services/lua_circular_buffer/blob/master/README.md |
||
+ | * https://github.com/mozilla-services/lua_sandbox/ |
||
+ | * http://hekad.readthedocs.org/en/latest/sandbox/ |
||
<BR> |
<BR> |
||
+ | Circular buffer works like RRDTools (RRD is round-robin archive) but store data in RAM. |
||
+ | Simplest buffer created with |
||
+ | looks like: |
||
+ | <PRE> |
||
+ | +-----+-------+ |
||
+ | |Time | Data1 | |
||
+ | +-----+-------+ |
||
+ | |-Ns | aN | |
||
+ | |... | ... | |
||
+ | |-2s | a3 | |
||
+ | |-1s | a2 | |
||
+ | |Now()| a1 | |
||
+ | +-----+-------+ |
||
+ | |||
+ | |||
+ | </PRE> |
||
==Loopback error== |
==Loopback error== |
Версия 15:17, 5 февраля 2016
Heka Filter AFD example
Here is example of AFD (AnomalyFaultDetection) plugin.
The main idea plugin is:
- count number of incoming messages per second
- calculate average rate
- put this rate into InfluxDB and create chart in Grafana
Collecting data
To collect message rate we are using filter which called on each incoming message.
Each process_message() call just add 1 to circular buffer.
Cirular buffer
Heka's LUA SandBox has bilt-in Circular Buffer Library ehich can be used for data aggregation.
More details:
- https://github.com/mozilla-services/lua_circular_buffer/blob/master/README.md
- https://github.com/mozilla-services/lua_sandbox/
- http://hekad.readthedocs.org/en/latest/sandbox/
Circular buffer works like RRDTools (RRD is round-robin archive) but store data in RAM.
Simplest buffer created with looks like:
+-----+-------+ |Time | Data1 | +-----+-------+ |-Ns | aN | |... | ... | |-2s | a3 | |-1s | a2 | |Now()| a1 | +-----+-------+
Loopback error
[heartbeat_filter_lua] type = "SandboxFilter" filename = "/usr/share/lma_collector/filters/afd_test2.lua" message_matcher = "TRUE" ticker_interval = 1 [heartbeat_filter_lua.config] critical_delta = 1
2016/02/03 18:39:40 Plugin 'aggregator_tcpoutput' error: writing to 192.168.0.7:5565: write tcp 192.168.0.7:5565: broken pipe 2016/02/03 18:39:41 Plugin 'aggregator_tcpoutput' error: writing to 192.168.0.7:5565: write tcp 192.168.0.7:5565: broken pipe 2016/02/03 18:39:41 Plugin 'heartbeat_filter_lua' error: attempted to Inject a message to itself 2016/02/03 18:39:41 Plugin 'heartbeat_filter_lua' error: attempted to Inject a message to itself 2016/02/03 18:39:41 Plugin 'heartbeat_filter_lua' error: Terminated. Reason: timer_event() /usr/share/lma_collector/filters/afd_test2.lua:57: inject_payload() creates a circular reference (matches this plugin's message_matcher) 2016/02/03 18:39:41 Plugin 'heartbeat_filter_lua': stopped 2016/02/03 18:39:41 Plugin 'heartbeat_filter_lua': has stopped, exiting plugin without shutting down.