Comments (27)
We ended up migrating away from lograge and writing a replacement that was based on middleware. We have been considering open sourcing it, but before we did, I'd prefer to work with you all to keep this project going and help existing users who need this.
That said, I haven't looked too deeply at it lately to see what it would take to PR and merge with the existing code. The crux of it would be completely retooling the log_subscriber.rb
, which is the majority of the code in this project.
However, if the project contributors are open to a largish PR for discussion, I can spend some time figuring out how to make that happen. My main reservation is that big PRs don't typically go over well without some discussion about the expectations beforehand.
from lograge.
That's cool. We will probably start out with a separate codebase.
I have mainly been waiting to gauge the contributor's interest in this, but totally understand if they want lograge wants to continue down its existing design. It also saves an awkward huge PR and my time to put that together if there's no interest in merging that functionality.
So we'll start out by releasing our stuff as a separate gem. I'll pass this on to our team.
from lograge.
Any thoughts or updates on this? I agree with @thoughtpunch, we want simple KVP logging in production and this issue is the only reason I'm not using lograge.
from lograge.
This is also the showstopper for me. We need all logs to be formatted the lograge way, or none.
from lograge.
@zmoazeni @Silex I'm also up to collaborate.
We have a Json formatter that makes working with ELK and/or cloud services (logentries, etc) excellent. On our list is to clean up the plumbing (middleware/railties/sidekiq support) and extract into an open source gem, so our interests are probably aligned.
from lograge.
How about having a middleware that triggers these errors and transmits them as notifications to the LogSubscriber?
from lograge.
How about having a middleware that triggers these errors and transmits them as notifications to the LogSubscriber?
Not quite following, do you mean "captures these errors"? A middleware like that would probably be able to handle the errors raised by the routing stack, but unless there's a way to reliably differentiate between requests that are handled by rails and requests handled by middleware we might end up with duplicate entries for requests.
I take it you'd prefer to move as little stuff as possible out of the log subscriber?
from lograge.
Hi @roidrage @BRMatt, I'm actually running into a very similar issue. It has to do with the timing of the Notification subscribers and middleware.
Another issue which I think might be tied to #72 is that logging from middleware on outbound responses are not available to lograge.
To try and put the timing in perspective:
- Middleware runs until we get to the Rails dispatch code
- Rails dispatch code runs. Might be successful or raise an error
- Part of the dispatch cycle is to kick off ActiveSupport::Notifications.
- Lograge sends information to the log
- All the middleware runs on the way out to serve the response
After step number 4, anything logged from the middleware is lost to lograge.
from lograge.
Thanks for laying out the issue. I'm generally open to suggestions here on how to improve this or code contributions.
from lograge.
@roidrage Right on. I might take a crack at a pull request where lograge does the final commit in a middleware just to see what you think of the design.
One other problem I'm coming up against is that the idea of overriding append_info_to_payload
breaks down for uncaught exceptions.
# from the readme
class ApplicationController < ActionController::Base
def append_info_to_payload(payload)
super
payload[:host] = request.host
end
end
append_info_to_payload
won't be called if an exception bubbles up. (At least in Rails 3.2) So an uncaught exceptional request would lose the host=
key in this case.
from lograge.
@zmoazeni did you make any headway with this? Not logging Rack requests is the single thing holding me back from using Lograge for every Rails app
@roidrage can we make Rack integration a dev priority?
from lograge.
We wrote something custom for our app that uses the Middleware design and
replaced lograge. I haven't spent any time thinking about it in terms of
lograge though.
On Jan 1, 2015 2:37 PM, "Dan Barrett" [email protected] wrote:
@zmoazeni https://github.com/zmoazeni did you make any headway with
this? Not logging Rack requests is the single thing holding me back from
using Lograge for every Rails app@roidrage https://github.com/roidrage can we make Rack integration a
dev priority?—
Reply to this email directly or view it on GitHub
#48 (comment).
from lograge.
I am investigating this issue. I've build a quick and dirty middleware that is actually working.
This middleware can be installed in parallel with the default Rails logger middleware, or it can substitute that. The middleware is also responsible for registering the Subscriber. This approach should replace the make the Lograge#remove_existing_log_subscriptions
method redundant and the config.lograge.enabled
deprecated (not sure about this).
Data is passed from one space to the other using a Thread variable.
I have some considerations and ideas
- when something goes wrong before the Rails notifications are triggered, the formatting and writing of the logs are lost: this two phases must be moved into the Rack space
- we need two type of
custom_options
callbacks: one that runs in the Rails space and one that runs in the Rack space; in the Rails space we can have some important information that is lost when we return back to the Rack space, while, more common information (such as request timestamp, user agent, host) are accessible in the Rack space, also when the Rails action fails - the current Subscriber should be refactored so that it does only the data collection, while formatting and writing passes to the middleware
- the
Lograge
configuration should be accessible both from the Subscriber (Rails space) and from the middleware (Rack space), so it should be extracted to its own class - the new middleware logs everything, so only logs the "not found"s, but also all the assets managed by the asset pipeline, but this could be managed by a new
ignore_custom
that runs in the Rack space - a new
ignore_custom
that is called in the Rack space should be considered (see above)
That's all.
You can take a look at the (awful) code.
from lograge.
I'm happy to help out with trying to get something landed but I'm not in a position where I can see this through myself. All contributions welcome! 👍
from lograge.
@zmoazeni: big +1, wether it ends up improving lograge or creating a new gem. I'll try to help either ways.
Not having a decent straightforward solution to good logs in Rails in 2017 is really annoying.
from lograge.
@roidrage, @zmoazeni, @eprothro: ping
from lograge.
Not sure this is the right place to do this, but here it goes.
Here's what I would like to see:
- per logger key/value tagging
- per request key/value tagging
- JSON and KVP formatting option
- Default to something like
{msg: "some plain message"}
for nonmerge
able log arg.
- Default to something like
- Transparent Rails 4 and 5 support (e.g. integrate if available, not a dependency -- if it has to be one, split out into separate rails-* gem)
- Transparent Sidekiq support (sidekiq process's logger)
- Rack implementation
- Ability to change logger level without process restart
Potential contributors:
- Are we roughly aligned?
- Is revamping lograge the right move, or spin off?
from lograge.
@zmoazeni: ping me when you have something, I'll test/debug/implement some features :-)
from lograge.
Any news?
from lograge.
Before we spin-off, does anyone have any experience with http://rocketjob.github.io/semantic_logger ?
from lograge.
Sorry, I haven't prioritized splitting out the code yet.
@eprothro thanks for sharing that project. I did a quick dive into the docs and the code. It certainly looks interesting, but it looks like it's fundamentally different than lograge and what my team needs on our projects which is to bundle all log entries for the entire request/unit-of-work. Meaning each line of output would be formatted individually with that project.
Though perhaps someone could write a custom file appender with SemanticLogger to collect, detect, and then write to the log as a single unit. It looks like elasticsearch appender is doing a bit of that with a Concurrent::TimerTask
but it's still just flushing individual bits of logs in bulk.
from lograge.
Ha! In http://rocketjob.github.io/semantic_logger/rails.html the logs looks good, very similar to the lograge ones. I'll give it a try.
from lograge.
Ok FYI I switched to http://rocketjob.github.io/semantic_logger and I find it fantastic! I just had to tweak it a little to have the behavior I wanted and the maintainer is really helpful.
from lograge.
Hey, do we have any updates on this?
I'm trying to configure better logging for Sidekiq Web UI and it doesn't play well with Lograge since it's a simple rack app.
I wasn't aware of http://rocketjob.github.io/semantic_logger and will check it out.
Best,
from lograge.
Has anyone who's moved to semantic_logger found a formatter that is similar to lograge? I've tried setting it to one_line
but that just seems to strip linebreaks out of the already-verbose logs that semantic_logger generates.
from lograge.
I suggest sticking with lograge and try to fork it, improve the maintenance status of it. Semantic logger does some sneaky monkey patches under the hood so setting up and running it does open the door for some really unexpected problem
from lograge.
@swrobel For a single line json output in semantic_logger create an appender using the json formatter, it would be a single line output then. See Appenders , docs are pretty good around all the output possibilities
from lograge.
Related Issues (20)
- Why are we striping params for redirect url?
- [Question] Project status HOT 2
- Monkey patching of Rails::Rack::Logger#call_app loses request.action_dispatch instrumentation HOT 1
- [Question] Logstash-event loaded from gh
- How to add extra log payload in action cable?
- Rails support HOT 1
- Should we demote some of our formatters to "contrib"?
- Unable to Filter ActionCable Params HOT 2
- exception cannot show on rescue_from custom handler exception HOT 1
- custom_payload is "persisted' (or per se kept) between 2 requests HOT 4
- Request: new gem release HOT 5
- Any thoughts on supporting `Turbo::StreamsChannel`? HOT 1
- Sinatra support? HOT 1
- Simpler way to remove all log subscribers. HOT 3
- Redirect in routes not handled by lograge
- Confused about what Rails logging is eliminated HOT 1
- ActionView logs not completely suppressed with Rails 7.1+
- TypeError: no implicit conversion of String into Integer when adding lograge to Rails project with Rack 3.0.9 and Puma 6.4.2 HOT 3
- Suggestion from Rails regarding better patching mechanism
- All logs show as INFO
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lograge.