Comments (9)
I think you ran through the following within one sec:
Rcurrent.log
——- rotate:
Ts1.log
——- compress:
Ts1.log.gz
——- next log line is written:
Rcurrent.log
Ts1.log.gz
——- rotate again:
Ts1.log
Ts1.log.gz
——- compress - boom
Ts1.log.gz
This wasn‘t possible without the explicitly triggered rotation. Fix should not be difficult.
from flexi_logger.
I am not sure I follow. So during the first rotate Ts1
is timestamp of rotation and for the second one timestamp of the first log line? Why such discrepancy?
It does not look like the issue gets triggered for further rotation as long as the app does not get restarted.
I think I will try the TimestampsDirect
naming to work around this issue.
from flexi_logger.
Ts1 is meant to be in all cases the timestamp of the infix in the filename. It has precision of second, that‘s why the collision can happen if all actions are done in a single second.
Collisions during rotate are detected and escaped with adding restartXXX to the infix.
The same is missing in the compress step.
from flexi_logger.
Yes, lack of collision detection during compression is an issue, you probably should use something like NamedTempFile::persist_noclobber
. But there is a different issue with timestamps used to name rotatef files. Note that between manual rotations passes more than one second.
Let's say I have app_rCURRENT.log
created at 18:25 (i.e. it's creation/birth time, not access or modification time) and nothing more. On 18:30 I launch my app and pass "rotate" command after a bunch of logs were written. Now I have two log files: current
and 18-25-xx.log.gz
. The current file has creation time 18:30.
After 5 minutes during which a bunch of logs were written into the current file I execute rotation command again. I would expect to get 3 files: current
, 18-30-yy.log.gz
, and 18-25-xx.log.gz
, but I get only two: current
and 18-25-xx.log.gz
, with data which should've been saved into 18-30-y.log.gz
overwriting 18-25-xx.log.gz
.
The third manual rotation at 18:40 which contains logs from 18:35 to 18:40 gets rotated with name 18:30-zz.log.gz
.
So it looks like the code chooses timestamp for rotated files incorrectly. The first rotation after cold start uses "current" file creation timestamp, while the subsequent ones use incorrectly cached values.
The issue may not be even specific to manual rotation, it's just the easiest way to trigger it.
from flexi_logger.
I think the issue is caused by this line. You use creation date of rotated file for naming next rotated file. It's wrong by itself as can be seen in my previous comment (you should use current time instead), but it causes name collision when appending is enabled. During initialization you use creation date of current file, which inevitable becomes creation date of next rotation file since you use fs::rename
which IIUC preserves creation date.
from flexi_logger.
Issues are now hopefully all fixed with 0.27.1.
Thanks for reporting and the discussions!
from flexi_logger.
As per my two previous comments, I don't think this issue is fully fixed. Yes, v0.27.1 no longer overwrites previously rotated log files on compression, but it still uses arguably incorrect timestamps for naming rotated files.
After rotating twice today with existing current file created two days ago I got the following files:
app_r2023-09-25_15-20-38.log.gz 2.46 KiB
app_r2023-09-25_15-20-38.restart-0000.log.gz 344 B
The second file contains records only from 2023-09-27 16:17:42 to 2023-09-27 16:17:48.
from flexi_logger.
Sorry for not replying to this part directly. I was distracted by testing and resolving the naming conflicts that also existed.
I Indeed I did not understand your point with the second rotation, your last comment changed that.
It should be solved now with 0.27.2.
from flexi_logger.
Thank you! I confirm that 0.27.2 has fixed the issue.
from flexi_logger.
Related Issues (20)
- Get all log files include rolling
- There is no line number in the log? HOT 2
- Option to disable stderr/stdout output? HOT 4
- How to reopen the output file of a FileLogWriter after Logger::add_writer took it over? HOT 2
- Dynamic logging for parent/daemon HOT 1
- bufsize in std_writer::AsyncHandle HOT 2
- Missing `regex` and `log` dependencies - fail to compile HOT 1
- Method on LoggerHandle for manually forcing rotation?
- Remove remnants of the external_rotation feature
- Different logger instances per thread? HOT 1
- GlobError unwrap happens HOT 3
- [trc] Improve usability by getting values from `Logger` or integrating with it HOT 1
- `kv` support from log crate. Json formatter. HOT 2
- Does this library support multiple log files and/or multiple named/filtered outputs? HOT 1
- Add new file logged on the fly HOT 8
- creating vs modification time HOT 1
- TimestampsCustomFormat with empty current_infix HOT 1
- FileLogWriter does not follow its max_level HOT 1
- Custom logging level HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flexi_logger.