Catch and log exceptions during code call execution about logstash-filter-aggregate

[fbaligand]

Hi @alkuzad,

It's odd to see that I had exactly the same problem (and idea) than you, nearly in the same time :)

To fix the problem, I suggest to do exactly the same thing that this code :
https://github.com/logstash-plugins/logstash-filter-ruby/blob/master/lib/logstash/filters/ruby.rb#L40

It allows to keep logstash up (and not crash logstash), log error when it occurs with @code section, and tag the event that caused the problem.

Makes sense to you ?

I consider to code it and make the pull request for it this thurday, in the evening.

[alkuzad]

Hi @fbaligand

Yes this is even better :) I personally extended the printout to include current map and event @data to find out errors quickly but I do not know if the @logger.error should add event automatically.

[fbaligand]

@alkuzad Can you put your code here ?

[alkuzad]

Hi @fbaligand

It's something like this:

begin
  @codeblock.call(event, map)
rescue => exception
  raise RuntimeError.new("There is an error with \n#{@code}\n, Error:\n#{exception}, Map: #{map}, EventData: #{event.instance_variable_get('@data')}")
end

input{
  stdin {}
}

filter{
  grok{
    match => { "[message]" => "%{NUMBER:id};(?<foo>.*)" }
  }

  if [id] {
    aggregate {
        task_id => "test_%{id}"
        code => "fail 'Asd';"
        map_action => "create"
        end_of_task => true
      }
  }
}

output {
  stdout { codec => rubydebug }
}

[fbaligand]

Thank you @alkuzad.

I will draw inspiration from your code to log error.

[fbaligand]

@alkuzad
The issue is now fixed.

To get it using logstash 1.5, use aggregate plugin version 0.1.4.
To get it using logstash 2.0, use aggregate plugin version 2.0.2.

Feel free to give feedback :)

[alkuzad]

@fbaligand thanks for fixing that :) I will check that when I'm done with fixing my ELK ansible setup :)

[fbaligand]

@alkuzad
OK. Let me know when you will check :)

[alkuzad]

Just playing with ELK again, works like a charm :)

[fbaligand]

Nice to hear that !