implement multi-line-timeout about syslog-ng

[justcallmegreg]

It's a missing feature from syslog-ng OSE that has been never implemented. The docs says different though. This Note will be removed from the docs till the feature will be implemented. Try to use pre and suffix to ensure multi-line-messages to end.

An option could be implemented like multi-line-timeout(<N>) that closes the message if no new lines came in the last N secs.

[jasantiago-stratio]

It also misses the last message in case of a file rotation with multi-line-mode(indented), I think that a multi-line-timeout parameter, as @ngergo wrote, would be a very nice feature.

[phemmer]

This needs to be reclassified as a bug. Using multi-line-mode (with prefix or indent) will result in message loss.
If syslog-ng shuts down, or receives a SIGHUP, the last message received by a source with multi-line-mode is dropped.

Syslog-ng needs to flush the buffer when it:

• goes idle (configurable time period)
• shuts down
• reloads

[faxm0dem]

just stumbled upon this, and I agree with @pzoleex it is a bug as the last message never makes it.
That being said, 10 seconds would be kind of arbitrary. If this is fixed, please make it configurable

[talien]

Hi!

This is a docs bug, syslog-ng OSE never knew timeout with multi-line messages. It is an issue though, if there is no garbage arrives, syslog-ng would stop processing data from this source. But if you think it through, a missing garbage from the input means either the configuration is bad, or the input stream is bad. Garbage is mandatory to be able to process a whole message.

[pzoleex]

I don't agree, garbage is not required. In most cases, there is no garbage at all, for example tomcat log starts with date and that's all. A newer date indicates that a new message started.

Not speaking about possible message loss in a normal situation.

[talien]

You are right, I wasn't correct. What I wanted to mean is when no garbage or prefix is present for a long time, then it can be a misconfiguration. And what I wanted to demonstrate is that the lack of timeout in the OSE is not a bug, instead it is a missing feature. It can be implemented after the logproto refactor is merged into the OSE.

[faxm0dem]

I stumbled again on this issue when trying to send some legacy multi-line logs from files with very low traffic. Many of those have one entry per file, and thus the log never makes it downstream.
It would at least make sense to have a SIGHUP "unstall" the event, but even that doesn't happen.

I'd love to have some idea as to if/when this will be implemented. thanks!

[faxm0dem]

👍

[faxm0dem]

I'm bumping this issue, as one of our users complained again ;-)