nreco / logging

Generic file logger for .NET Core (FileLoggerProvider) with minimal dependencies

NReco MDD Framework

NReco is an ASP.NET application framework that enables lightweight model-driven development and domain-specific modelling for real-life applications.

NReco Framework includes:

• reusable components for building loosely-coupled web applications and suitable for generative programming
• special XML models processor that transforms abstract domain-specific XML models into ASP.NET components and IoC-container configuration
• ready-to-use set of domain-specific models that cover different aspects of typical ASP.NET application (UI elements like forms, lists dialogs; business-logic layer; data access and data permissions, data indexing etc)

NReco is a right choice for:

• rapid prototyping and development of custom web-applications
• custom enterprise/business web applications
• software product-lines development and maintenance (projects "family")
• SaaS-products and other scalable web-applications hosted in the cloud

## More information

• Official NReco Framework website
• Online NReco App Generator

## Standalone NReco Components

• NReco.PdfGenerator (WkHtmlToPdf wrapper for .NET for HTML-to-PDF conversion)
• NReco.Recommender (Collaborative Filtering framework; port of Apache Mahout Taste CF)
• NReco.VideoConverter (FFMpeg wrapper for .NET)
• NReco.PivotData (C# Aggregation Library for bulding pivot reports)

Copyright & License

NReco Version 1.0 (nreco1 branch) © 2008-2013 Vitalii Fedorchenko + other contributors (published under LGPL) NReco Version 2.x (master) © 2013-2015 Vitalii Fedorchenko + other contributors (LGPL)

logging's Issues

Log files not being deleted automatically

Hi, I've noticed old log files do not get deleted automatically in my project, all other configurations work though: filename, filesize limit. Here's my configuration

  "Logging": {
    "LogLevel": {
      "Default": "Debug",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Warning"
    },
    "File": {
      "Path": "Log-{0:yyyy}{0:MM}{0:dd}.log",
      "Append": "True",
      "FileSizeLimitBytes": 10485760,
      "MaxRollingFiles": 5
    }
  },

public static IHostBuilder CreateHostBuilder(string[] args)
{
    return Host.CreateDefaultBuilder(args)
    .ConfigureWebHostDefaults(webBuilder => webBuilder.UseStartup<Startup>())
    .ConfigureLogging((config, logging) =>
        logging
            .AddFile(config.Configuration.GetSection("Logging"), fileLoggerOpts =>
            {
                fileLoggerOpts.FormatLogFileName = fName => string.Format(fName, DateTime.UtcNow);
                fileLoggerOpts.MaxRollingFiles = 5; //try set rollfiles manually, no effect
            }
            )
            .AddConsole()
            .AddEventSourceLogger()
            .AddDebug()
     ) .....

As seen in the code I tried adding the MaxRollingFiles option manually, but it had no effect. I checked the nreco source files and saw no File.Delete at all. Is the deletion supposed to be provided automatically by the .NET framework maybe? I'm using netcore 3.1 and the deletion does not work neither when local developing nor in an azure service.

Rolling file with Date variable from appsettings.json

¿Is posible configure name of file with Date?

"File": {
  "Path": "logs\\app_{0:yyyy}-{0:MM}-{0:dd}.log",
  "Append": "True",
  "FileSizeLimitBytes": 1048576, // use to activate rolling file behaviour 1MB
  "MaxRollingFiles": 5 // use to specify max number of log files
}

Missing logs in .NET

Hi.
When logging with a .NET application at the end of the program, the last logging is not committed to the file.
Could it be that the dispose method is not waiting for the writing to complete?
Have also noticed that when creating a new file for logging, creating a log event, and then closing the console, only a empty file is created, with no logging included.

Try Catch does not log

Hi,

I am new to logging and currently exploring logging frameworks, while testing out this framework, when I have a try catch condition, it seems like the Exception log is not logged. Perhaps anyone could advise regarding this matter?

Here is my test code (using MVC architecture and this section of code is in the controller):
public async Task TestError()
{
int statCode = 0;
string msg = "";
try
{
Guid g = new Guid("ABC"); //Here it will definitely create an error since "ABC" is not a proper Guid format
statCode = 200;
msg = g.ToString();
}
catch (Exception ex)
{
statCode = 500;
msg = ex.ToString();
}
return StatusCode(statCode, msg);
}

Thanks in advance.

ImplementationType null injecting with .AddLogging() method

Not yet sure why, but ImplementationType null injecting with .AddLogging() method.
Advice would be appreciated. Also is there a simpler way of configuring this logging library with DI?

Screenshot:

Minimal working example (uncomment the line that works):

using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Linq;
using CsvHelper;
using CsvHelper.Configuration;
using Newtonsoft.Json;
using Microsoft.Extensions.DependencyInjection;
using NReco.Logging.File;
using Microsoft.Extensions.Logging;

class Program
{
    static void Main()
    {
        var services = new ServiceCollection();

        services
            //.AddSingleton<ILogger>(new FileLogger("app.log", new FileLoggerProvider("app.log")))
            .AddLogging(loggingBuilder => loggingBuilder.AddFile("app.log", append: true))
            .AddTransient<Appy>();

        var serviceProvider = services.BuildServiceProvider();

        var app = serviceProvider.GetService<Appy>();
        app.logSomething();
    }

}

class Appy
{
    private readonly ILogger _logger;

    public Appy(ILogger logger)
    {
        _logger = logger;
    }

    public void logSomething()
    {
        _logger.LogInformation($"MyApplication Started!");
    }
}

Windows service log to executable path

Is there a way of specifying that the relative path given by Path in the settings should be relative to the executable instead of c:\windows\system32 when running a console app as a Windows service?

Ability to stack error file handlers

Essentially, I would think that if in the HandleFileError I add a string to the file name, then if the initial log file is being used by another process and the alternative file name is also being used by a different process, it would just do something like "FileName_alt_alt.log" instead of just crashing on "FileName_alt".

Thoughts?

Logging through singleton

Hello, firstly thanks for your work.

Is it possible to pass this code below to a singleton ?
From:

builder.Services.AddLogging(loggingBuilder => {
	loggingBuilder.AddFile("app.log", append:true);
});

To:

builder.Services.AddSingleton(new ServiceA(configuration, logFactory.CreateLogger<ServiceA>())); 

I'm trying to migrate logging from EventLog to File. For now it keeps logging into EventLog obvly. I'm new to C# and I'm struggling with this.. My Service look like this:

public ServiceA(IConfiguration configuration, ILogger<ServiceA> logger){}

.SetMinimumLevel() has no effect

I use this implementation in a Core Console app without settings file. This is how I create a logger:

var lf = LoggerFactory.Create(builder =>
	{
		builder.SetMinimumLevel(LogLevel.Trace);
		builder.AddDebug();
		builder.AddConsole(c => c.DisableColors = false);
		builder.AddFile($"{logPath}\\SomeName_{DateTime.Now:yyyy-MM-dd_HH-mm-ss-ffff}.log")
			.SetMinimumLevel(LogLevel.Trace);
	});
_logger = lf.CreateLogger<Program>();

Is there a way to set the default log level to LogLevel.Trace programmatically?

Implement rolling log file behaviour

Feature goals:

• prevent 'one huge file' when append=true is used: 2 possible modes - when new log file is created on app restart, or new file log is used when file name pattern is changed (typical case - use separate log file for each day or month)
• limit total size of the log file(s)

Would be possible to use LogLevel structure ?

It looks like appsettings.json parameter
"MinLevel": "Warning", // min level for the file logger
replaces normal

    "LogLevel": {
      "Default": "Information",
      "System": "Warning",
      "Microsoft": "Warning"
    }

Am I missing something ? Is tehre any plan to use the microsoftish way of configuring levels ?
Thanks for clarification. Jiri

LogDebug does not work

Hi,

unfortunately LogDebug does not work.

My configuration:

// appsettings.json
{
  "Logging": {
    "LogLevel": {
      "Default": "Debug",
      "System": "Debug",
      "Microsoft": "Error"
    },
    "File": {
      "Path": "app.log",
      "Append": "True",
      "FileSizeLimitBytes": 10485760, // 10 * 1024 * 1024 => 10MB
      "MaxRollingFiles": 3 // use to specify max number of log files
    }
  },
  "AllowedHosts": "*"
}

// Startup.cs
services.AddLogging(loggingBuilder => {                
  var loggingSection = Configuration.GetSection("Logging");
  loggingBuilder.AddFile(loggingSection);
});

The calls:

_logger.LogInformation("INFORMATION");
_logger.LogDebug("DEBUG");
_logger.LogWarning("WARN");
_logger.LogCritical("CRIT");
_logger.LogError("ERROR");

The result:

2020-11-27T12:31:36.2114667+01:00	INFO	[My.Controllers.MyController]	[0]	INFORMATION
2020-11-27T12:31:36.2345011+01:00	WARN	[My.Controllers.MyController]	[0]	WARN
2020-11-27T12:31:36.2375962+01:00	CRIT	[My.Controllers.MyController]	[0]	CRIT
2020-11-27T12:31:36.2403344+01:00	FAIL	[My.Controllers.MyController]	[0]	ERROR

Does anyone have an idea what I am doing wrong?

Add option to specify wildcard in DetermineLastFileLogName()

Could an option be added to specify the wildcard used by DetermineLastFileLogName()? Line 166 of FileLoggerProvider.cs has a hard-coded "*". In my use case, I have MaxRollingFiles set to 3. So the log files should be basename, basename1 and basename2. The wildcard on line 166 would best be a "?" in my case. The problem with "*" is that there may be other log files that have been saved off for later investigation in the folder that "*" picks up, but "?" would filter out. As it is now, the code overwrites those log files that we had saved for later investigation. Work-around is to add a prefix to the filename, change the extension, or move them to a different folder.

Get rolling file name index

This is related to #5, but not exactly.
Here is my NReco.Logging registration with .NET DI:

services.AddLogging(loggingBuilder => {
        loggingBuilder.AddFile("Logs\\app_{0:yyyy}-{0:MM}-{0:dd}.log", fileLoggerOpts => {
        fileLoggerOpts.FormatLogFileName = getLogFileName;
        fileLoggerOpts.FileSizeLimitBytes = 1 * 1024 * 1024; // 1MB
        fileLoggerOpts.MaxRollingFiles = 10;
        fileLoggerOpts.UseUtcTimestamp = true;
    });
})

<...>
// callback for FormatLogFileName 
static String getLogFileName(String fName) {
    // cache current file name in static property:
    LogFilePath = String.Format(fName, DateTime.UtcNow);

    return LogFilePath;
}

Everything works ok with one exception: I loose current file name tracking when log file is rolled. By looking in Logs directory, I see that file names are appended with rolling index:

And I have no idea how to access this index so I reference to actual current file in LogFilePath property. Any ideas?

Append: True not working and other questions

Adding the logger config as per documentation

  "Logging": {
    "LogLevel": {
      "Default": "Debug",
      "System": "Information",
      "Microsoft": "Error"
    },
    "File": {
      "Path": "app.log",
      "Append": "True",
      "FileSizeLimitBytes": 3, // use to activate rolling file behaviour
      "MaxRollingFiles": 4     // use to specify max number of log files
    }
  }

And registering in my console app

var logFilePath = configuration.GetSection("Logging");

new ServiceCollection()
            .AddLogging(loggingBuilder => loggingBuilder.AddFile(logFilePath))

Then injecting to my class

public sealed class Application : IApplication
{
    private readonly IRORepository<Grv, int> _grvRORepository;
    private readonly ILogger<Application> _logger;

    public Application(
        IRORepository<Grv, int> grvRORepository,
        ILogger<Application> logger)
    {
        EnsureArg.IsNotNull(grvRORepository, nameof(grvRORepository));
        EnsureArg.IsNotNull(logger, nameof(logger));

        _grvRORepository = grvRORepository;
        _logger = logger;
    }

    public async Task RunAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("hello there!");

        var grv = await _grvRORepository.GetByIdAsync(8, cancellationToken);

        _logger.LogError("YOU WOT M8");
    }
}

Generates 4 separate files:

app.log

2021-05-28T23:25:25.3186362+01:00	INFO	[Microsoft.EntityFrameworkCore.Infrastructure]	[Microsoft.EntityFrameworkCore.Infrastructure.ContextInitialized]	Entity Framework Core 3.1.15 initialized 'DomainContext' using provider 'EntityFrameworkCore.Jet' with options: DataAccessProviderFactory 

app1.log

2021-05-28T23:25:25.3284572+01:00	INFO	[Allways.Application.Application]	[0]	hello there!

app2.log

2021-05-28T23:25:25.7681790+01:00	INFO	[Microsoft.EntityFrameworkCore.Database.Command]	[Microsoft.EntityFrameworkCore.Database.Command.CommandExecuted]	Executed DbCommand (60ms) [Parameters=[@__id_0='?' (DbType = Int32)], CommandType='Text', CommandTimeout='30']
SELECT TOP 1 `t`.`GRVNumber`, `t`.`GrvNumber`
FROM `TGRV` AS `t`
WHERE `t`.`GRVNumber` = @__id_0

app3.log

2021-05-28T23:25:25.8888947+01:00	FAIL	[Allways.Application.Application]	[0]	YOU WOT M8

These files are overwritten each time I run the application

Is that as expected? Shouldn't my "Append": "True" option cause them to not overwrite previous log files?

Also, how does the log level work, and how do I know what log is going to be written to which file?

The docs, while useful in registering the library, don't really explain how to use it properly

Really appreciate any guidance, thanks

How to get an instance of logger without injection?

In ASP.NET Core 3.0 injection is limited for the Startup constructor:

The most significant change from WebHostBuilder to HostBuilder is in dependency injection (DI). When using HostBuilder, you can only inject IConfiguration and IHostingEnvironment into Startup's constructor.

Is it possible to get an instance of logger without injection in cases like this one?

File open errors inside FileLoggerProvider constructor are not handled by HandleFileError

From #57 (comment)_

FileWriter.OpenFile uses "HandleFileError" when it is defined, however at the time of FileLoggerProvider creation (when log file is opened for the first time) this handler is not initialized yet. To fix this it would be enough to postpone log file open and open it only after initialization.

Understanding why this happened.

How do I log to my services.AddLogging file?

So I am trying to put logging into an app of mine. In my Program.cs file I created a service.AddLogging file with a name of app_debug.txt.
Being that I am new at using this package. I used the LoggingTests as my examples and created a new LoggingFactory and AddProvider with the file name app.txt. Created a new logger and then build methods to write to the log file (Start, End, Message and Exception).
When I run the app from a browser I get all the messages from my debug window in VS written to the app_debug.txt file. All the other messages are written to app.txt.
So I am wondering whether creating the factory and logger then disposing of the factory for every message call is going to cause problems if I have 300 people working on the app?
Questions?

1. What would be the best way to only have one LoggingFactory and one ILogger logger per session?
2. How coud I write loggs to the app_debug.tx file. I cannot figure this out.

One suggestion. This would be easier if there was some examples of how to use the package.

Thank you,

Michael

Need support for formatting filename based on category name

When formatting logging filename, I think the category name needs to be added as a parameter, too.

how to inject to controller and log

ijnect to controller and create new file log on a particular exception

can u please elaborate

how do we inject to a controller and log it , request and response as json

How to set filename to reset every midnight

Hi,

I would like to my logs to be writable to a file which represents the current day (Log_yyyy-mm-dd.log). It is possible to achieve this with some configuration? Also, I would need to change the current filename if the application is running and it get past midnight. Therefore at 00:00:00 it would start writing to the next file.

Any help would be appreciated.

Prevent empty log file at startup

The core tool creates a new empty file as soon as the logger is initialized. This works fine if you use a non-granular naming scheme, like "LogFile.log" because all of your logs will then be appended to the same file. However, when using granular file names - like adding a timestamp to the file name - the initial log file remains empty. This adds noise to the log output folder.

It would be very helpful if files were only created when a message is being written

Associate 'File' alias with FileLoggingProvider

In MVC Core 2 logging infrastructure uses https://github.com/aspnet/Logging/blob/dev/src/Microsoft.Extensions.Logging/ProviderAliasAttribute.cs to determine provider alias for filtering rules.
Lets use 'File' alias, this will allow to use "File": { "LogLevel": { } } in configuration section loaded with logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging"));

logger every day

How to use your library to creating new *.log files every day

Log file cleanup

First of all, thanks for this useful provider!

We use logs with names based on timestamps like this:

services.AddLogging(loggingBuilder => {
    loggingBuilder.AddFile("Logs\\app_{0:yyyy}-{0:MM}-{0:dd}.log", fileLoggerOpts => {
        fileLoggerOpts.FormatLogFileName = fName => String.Format(fName, DateTime.UtcNow);
        fileLoggerOpts.FileSizeLimitBytes = 1 * 1024 * 1024; // 1MB
        fileLoggerOpts.MaxRollingFiles = 10;
        fileLoggerOpts.UseUtcTimestamp = true;
        LogFilePath = fileLoggerOpts.FormatLogFileName.Invoke("Logs\\app_{0:yyyy}-{0:MM}-{0:dd}.log");
    });
})

However, we observed that older log files generated in previous app executions are not deleted. Is this by design and we need to write our own cleanup logic or it is because how we generate file names?

When using multiple instances of same console app we are getting error when writing logs

When using multiple instances of same console we are getting error when writing logs.

we are getting file in use error

My app crash randomly with error `Exception Info: System.IO.IOException: The process cannot access the file 'C:\...\Logs.log' because it is being used by another process.`

Hello, I have a problem that I think comes from this extension. My application is hosted on a Windows Server with IIS. It works perfectly for several days, then after a while it crashes with the error Exception Info: System.IO.IOException: The process cannot access the file 'C:\...\Logs.log' because it is being used by another process.

At first I thought it was because the log file was open next to it in Visual Studio, but even after closing the file and restarting my app, the error comes back a few days later. I'm pretty sure the log file isn't used anywhere else. And even if it is opened elsewhere, it shouldn't be a problem in theory, since I can open it in parallel with Visual Studio, and see the changes made to the file by nreco/logging. (so there's no problem with a file being opened and modified by two programs at the same time).

The extension is supposed to handle concurrency problems and use a queue, so I don't understand where the problem comes from, especially as the crashes occur at times that seem random and irregular to me.

Here is the full exception message:

Application: w3wp.exe
CoreCLR Version: 7.0.523.17405
.NET Version: 7.0.5
Description: The process was terminated due to an unhandled exception.
Exception Info: System.IO.IOException: The process cannot access the file 'C:\...\Logs.log' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access)
   at NReco.Logging.File.FileLoggerProvider.FileWriter.<OpenFile>g__createLogFileStream|7_0(<>c__DisplayClass7_0&)
   at NReco.Logging.File.FileLoggerProvider.FileWriter.OpenFile(Boolean append)
   at NReco.Logging.File.FileLoggerProvider.FileWriter..ctor(FileLoggerProvider fileLogPrv)
   at NReco.Logging.File.FileLoggerProvider..ctor(String fileName, FileLoggerOptions options)
   at NReco.Logging.File.FileLoggerProvider..ctor(String fileName, Boolean append)
   at Microsoft.Extensions.Logging.FileLoggerExtensions.<>c__DisplayClass0_0.<AddFile>b__0(IServiceProvider srvPrv)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Extensions.Hosting.HostBuilder.<>c__DisplayClass35_0.<PopulateServiceCollection>b__2(IServiceProvider _)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Extensions.Hosting.HostBuilder.ResolveHost(IServiceProvider serviceProvider, DiagnosticListener diagnosticListener)
   at Microsoft.Extensions.Hosting.HostApplicationBuilder.Build()
   at Microsoft.AspNetCore.Builder.WebApplicationBuilder.Build()
   at Program.<Main>$(String[] args) in D:\...\Program.cs:line 32

As far as I'm concerned, the problem is just on line 32 of my Main, after that the problem comes solely from dotnet and this extension. My Program.cs:

using ...;


var builder = WebApplication.CreateBuilder(args); 


const string corsPolicyName = "CorsPolicySysProd";
var logFolderPath = builder.Environment.IsDevelopment() ? string.Empty : "C:\\inetpub\\logs\\LogFiles";
const string logFileName = "Logs.log";


// Add services to the container.

builder.Services.AddCors(options =>
    options.AddPolicy(corsPolicyName, corsBuilder =>
        corsBuilder.WithOrigins("http://..").AllowAnyMethod().AllowAnyHeader()));
builder.Services.AddControllers();
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen(options =>
{
    // add a custom operation filter which sets default values
    options.OperationFilter<SwaggerDefaultValues>();

    var fileName = typeof(Program).Assembly.GetName().Name + ".xml";
    var filePath = Path.Combine(AppContext.BaseDirectory, fileName);

    // integrate xml comments
    options.IncludeXmlComments(filePath);
});
builder.Services.AddLogging(loggingBuilder => loggingBuilder.AddFile(Path.Combine(logFolderPath, logFileName), append: true));

var app = builder.Build();


// Configure the HTTP request pipeline.

app.UseSwagger();
app.UseSwaggerUI();

//HTTPS redirection is disabled because ... is in HTTP.
//app.UseHttpsRedirection();

app.UseCors(corsPolicyName);

app.UseAuthorization();

app.MapControllers();

app.Run();

The only workaround I have so far is to restart the app in case of a crash. Let me know if there's anything else I can do to help identify the problem. thanks in advance for your help, and thanks for creating this extension :)

APIs Filtering

Firstly I'd like to thank you for your library,

I used it in the following way:
appsettings.json: contains the configuration
middleware.cs: contains the HttpContext and creates a Logger then use it to LogInforamtion
StartUp.cs: two methods:
a. ConfigureServices: services.AddLogging and do the configuration
b. Configure : calls the middleware for the app

I want to know if there is any configuration or any way to check the API method type before I call the middleware?

Thank you.

Trace logging isn't working correctly for in code config

I am fiddling around with trace logging for debugging some of my code and I haven't been able to write the trace level to file no matter what I do. I have this configuration:

            {
                loggingBuilder
                    .SetMinimumLevel(LogLevel.Trace)
                    .AddConsole()
                    .AddFile($"log.txt", append: true);
            });

and my file output looks like this:

2021-07-21T15:13:37.7811744-04:00	INFO	[StaticSiteGenerator.Markdown.Parser.MarkdownFileParser]	[0]	Starting to convert string contents to Markdown

but my console log looks like this:

trce: StaticSiteGenerator.StaticSiteGenerator[0]
      Starting conversion of static site.
info: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Starting to convert string contents to Markdown
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Starting to convert file markdownInput/markdownFile.md
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Reading file: markdownInput/markdownFile.md
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Read file contents: # Test Header
      
      Test body
      
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Succesfully converted markdownInput/markdownFile.md into Block Elements
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Converted file: markdownInput/markdownFile.md
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Starting to convert file markdownInput/headerOnlyFile.md
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Reading file: markdownInput/headerOnlyFile.md
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Read file contents: # This is one header
      # This is another header
      
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Succesfully converted markdownInput/headerOnlyFile.md into Block Elements
trce: StaticSiteGenerator.Markdown.Parser.MarkdownFileParser[0]
      Converted file: markdownInput/headerOnlyFile.md
trce: StaticSiteGenerator.StaticSiteGenerator[0]
      Finished conversion of static site.

which is obviously more detailed. Is this style of configuration simply unsupported?

Windows 11 - Smart App Control

Windows 11 introduced Smart App Control. One of the security checks for apps is ensuring that the app and its binaries are signed. Because NReco.Logging.File does not sign its binaries, software using it will be blocked from running when Smart App Control is enabled.

Please sign DLLs produced for NReco.Logging.File.

https://support.microsoft.com/en-us/topic/what-is-smart-app-control-285ea03d-fa88-4d56-882e-6698afdb7003

NReco.Logging.File (unsigned):

Microsoft dll (signed):

Collisions/concurrency?

I am using the Provider once in a CLI (in rolling file configuration)

I am configuring it in code, like this:

.ConfigureLogging((context, logging) =>
                {
                    logging.ClearProviders();
                    logging.AddFile($"logs-{Process.GetCurrentProcess().Id}.log", options =>
                    {
                        options.Append = true;
                        options.MinLevel = LogLevel.Information;
                        options.MaxRollingFiles = 1;
                        options.FileSizeLimitBytes = 10 * 1000 * 1000;
                    });

At runtime, there are several ILogger.Trace() being recorded in the log file correctly, and then I see this exception:

System.IO.IOException: The process cannot access the file 'C:\......\logs-6136.log' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access)
   at NReco.Logging.File.FileLoggerProvider.FileWriter.<OpenFile>g__createLogFileStream|7_0(<>c__DisplayClass7_0& )
   at NReco.Logging.File.FileLoggerProvider.FileWriter.OpenFile(Boolean append)
   at NReco.Logging.File.FileLoggerProvider.FileWriter..ctor(FileLoggerProvider fileLogPrv)
   at NReco.Logging.File.FileLoggerProvider..ctor(String fileName, FileLoggerOptions options)
   at Microsoft.Extensions.Logging.FileLoggerExtensions.<>c__DisplayClass1_0.<AddFile>b__0(IServiceProvider srvPrv)

the rest of stack trace is my code, all the way back to a line in the code that tries to make another call to ILogger.Trace()

The last ILogger.Trace() call I make is after shutdown my IHost.

Seems like the LoggingProvider may be keeping the file open all the time? or perhaps we have a writing concurrency issue?
Or not correctly closing the file after exiting?

Could that be the case?

Good project! Support for .NET 8? console log was changed after add this lib.

windows 10
.NET 8

var builder = Host.CreateApplicationBuilder();
builder.Logging.AddFile("app.log", append: true); // add this row
builder.Logging.AddSimpleConsole(option =>
{
    option.IncludeScopes = true;
    option.TimestampFormat = "MM-dd HH:mm:ss ";
});

Preview console log:

Later console log:

Logging not working with non-specialized ILogger interface

When I use the non-specialized ILogger interface the file logging doesn't log.
...
var logger = serviceProvider.GetService<ILogger>();
services.AddSingleton(logger);
...
void MyClass(ILogger logger)
{
logger.LogInformation("This doesn't file log but the console and debug loggers do log it");
}

LogMessage Suggestion

Hi,
Thanks for this little library.

I just have two minor suggestions regarding LogMessage.

The first is to consider making it a class. Perhaps there's a reason you can't use a class(?), but as a struct we can't extend it, and structs holding reference types can cause some difficulties to those who are unaware (remember that Exception is mutable).

The second is to make the constructor public (it's currently internal), or simply delete it entirely, so we can create LogMessages in our code. Without this, refactoring sometimes gets more awkward than it needs to be.

internal static string FormatLogMessage(string someargument)
{
    LogMessage lm = new LogMessage()
    {
        Message = someargument // ERROR
    };
    lm = new LogMessage(someargument);//ALSO ERROR
    return FormatLogMessage(lm);
}

internal static string FormatLogMessage(LogMessage arg)
{
...

Neither are at all critical but just something to consider for next time you work on the library.

Thanks again

Following file with rolling logfile

For testing purposes I've set this:

"FileSizeLimitBytes": 2000,
"MaxRollingFiles": 10

and I want to follow the file (with tail -f or a tool like baretail).
For this I need the last log entries in the "main" file but currently the new content is added in MyLog[number].log.

Is it possible to have the newest entries in the main file?
I'm planning to have rolling over 10 files with FileLimitSize of 2 MB. So MyLog9.log should contain the oldest content.

Add scope support

Would it be possible to add Scope support? I'm not quite sure what level of effort it entails, but these pages seem to lay out a basic implementation

https://www.meziantou.net/asp-net-core-json-logger.htm

https://www.codeproject.com/Articles/1556475/How-to-Write-a-Custom-Logging-Provider-in-ASP-NET

No issue, just wanted to say thanks!

I'm amazed that Microsoft don't offer a file logger provider out of the box, but I'm very grateful that you've gone to the effort of creating and sharing one.
Thanks!

App deadlock on FileLoggerProvider.WriteEntry

We use the file logger in ASP.NET Core .net7 app the simple way by adding in Startup
loggingBuilder.AddFile(loggingSection);

We have now face random (hours...week) situation when app gets "stuck". By taking a dump we found that all threads wait in this place - entryQueue.Add(message); in FileLoggerProvider

		internal void WriteEntry(string message)
		{
			if (!entryQueue.IsAddingCompleted)
			{
				try
				{
					entryQueue.Add(message);
				}
				catch (InvalidOperationException)
				{
				}
			}
		}

(the 2and 5 threads wait there as well...)
But there is no clear indication "why" ? Of course the only way to get over this is to kill app and restart.

We noticed that the FileLoggerProvider is using a blocking collection

private readonly BlockingCollection<string> entryQueue = new BlockingCollection<string>(1024); 

And noticed that there is no "defensive measure" so once anything goes south here (or the loop simply terminates...

		private void ProcessQueue()
		{
			foreach (string item in entryQueue.GetConsumingEnumerable())
			{
				fWriter.WriteMessage(item, entryQueue.Count == 0);
			}
		}

We very likely end up with filled blocking queue, nobody taking items, app stuck....

We noticed that in methods
WriteMessage(string message, bool flush)
CheckForNewLogFile()

There are no try-catch so it is a matter of time, when something bad happens...

Do you consider it relevant ?
Would you mind to add some corrections by some PR/Fork from us?
Or you find it so easy to make it by yourself ?

We'd:
a) Allow to configure an optional timeout, and changeentryQueue.Add(message);to entryQueue.TryAdd(message,timeout);
b) Add some try-catch around manipulation with LogFileWriter

For us losing some part of log is much acceptable then having app deadlocked.

BR
Jiri

log file created but not written

Hello,

I tried to use AddFile(file, options => {}) to add a filte logger to my logger initialization, but this did not work

The log file is created, but nothing is written into it.

I pushed a minimalist project here : https://git.geekwu.org/bastien/test_log

I also tried to use the loggerFactory.AddProvider(new FileLoggerProvider(file, true)); approch, but I got the same (non) result

What did I miss ?

Logger can't write log outside the function that logger created

Hi, Thanks for your logging module,
but I have got some issues when using it.

Logger can only work in the function that be created.
Here is my code:
Program.cs

using Microsoft.Extensions.Logging;
using Backend.Common.Log;

namespace Backend
{
    class Program
    {
        static void Main(string[] args)
        {
            LogUilts.LoggerOutput<Program>(LogLevel.Error, "Log that in Main"); // This doesn't work
        }
    }
}

Backend.Common.Log.LogUtils:

using System;
using System.Collections.Generic;
using Microsoft.Extensions.Logging;

namespace Backend.Common.Log
{
    class LogUilts
    {
        private static  ILogger logger;

        static LogUilts()
        {
            loggers = new();
        }

        public static void LoggerOutput<T>(LogLevel logLevel, string message)
        {
            string className = typeof(T).Name;
            if(!loggers.ContainsKey(className))
            {
                var tmpLogger = CreateLogger<T>();
                logger .Log(LogLevel.Information, "Log out of function that logger created");  // This doesn't work
                tmpLogger .Log(LogLevel.Information, "Log out of function that logger craeted"); // This doesn't work
            }
        }

        private static ILogger CreateLogger<T>()
        {
            using var loggerFactory = LoggerFactory.Create(builder =>
            {
                builder
                .AddConsole();
            });

            var timeStamp = DateTime.Now.ToString("yyyyMMddHmmss");
            string name = typeof(T).Name;
            loggerFactory.AddProvider(new NReco.Logging.File.FileLoggerProvider($"./logs/{name}_{timeStamp}.log", true)
            {
                FormatLogEntry = (msg) => {
                    var sb = new System.Text.StringBuilder();
                    System.IO.StringWriter sw = new System.IO.StringWriter(sb);
                    var jsonWriter = new Newtonsoft.Json.JsonTextWriter(sw);
                    jsonWriter.WriteStartArray();
                    jsonWriter.WriteValue(DateTime.Now.ToString("o"));
                    jsonWriter.WriteValue(msg.LogLevel.ToString());
                    jsonWriter.WriteValue(msg.LogName);
                    jsonWriter.WriteValue(msg.EventId.Id);
                    jsonWriter.WriteValue(msg.Message);
                    jsonWriter.WriteValue(msg.Exception?.ToString());
                    jsonWriter.WriteEndArray();
                    string result = sb.ToString();
                    return result;
                }
            });

            logger = loggerFactory.CreateLogger<T>();
            logger.Log(LogLevel.Information, "Log in function that logger crated");    // This is work
            return logger;
        }

    }
}

But all of these can output to console.
Here is the output in console:

info: Backend.Program[0]
      Log in function that logger crated
info: Backend.Program[0]
      Log out of function that logger created
info: Backend.Program[0]
      Log out of function that logger craeted
fail: Backend.Program[0]
      Log that in Main

And this is the output in file:

["2021-08-25T12:28:16.7199510+09:00","Information","Backend.Program",0,"Log in function that logger crated",null]

Multiple file providers with matching prefix

This had me bashing my head in the wall, until I went to look through the code.

TLDR: If you have multiple file providers and at least one of them is rolling, be sure to use different prefixes for the fileNames or you will get a file in use exception upon opening the rolling file provider. Prefix being the part of the fileName before the first occurrence of a dot ..
Example: two providers configured for app.unhandled.log and app.log (rolling) respectively, will cause an error.

I set up two FileLoggerProviders: one for app.unhandled.log and one for a rolling file app.log.
I first initialize the app.unhandled.log provider because that is what I am falling back to for unhandled exceptions:

var loggerFactory = LoggerFactory.Create(logging =>
            {
                logging.AddFile("app.unhandled.log");
            });

Later I initialize the app.log provider:

app.ConfigureLogging((context, logging) =>
                {
                    var loggingSection = context.Configuration.GetSection("Logging");
                    logging.AddFile(loggingSection);
                    logging.AddConsole();
                })

Every time I reach logging.AddFile(loggingSection); I was getting a "file is being used exception for app.unhandled.log".

The problem was resolved by changing the name of app.unhandled.log to unhandled.log.

Turns out in FileWriter class, DetermineLastFileLogName method, a pattern is being constructed using the fileName and extension on the passed in log file when using rolling files. The log fileName was app.log in my case. The pattern constructed was app*.log.
It then retrieves all log files matching the pattern and starts writing to the one that was last written to, which turns out to be app.unhandled.log. That one is already open for writing though, so we generate a file in use error.

I am not sure of a good way to fix this in the library, but thought it might be good to mention, in case anyone stumbles upon this.

New to C# and .NET, really struggling to get this library to work

Completely new to the C#/.NET Core environment. I've looked everywhere in the Microsoft Docs and on Google/Stack Overflow but cannot find a single decent code sample for NReco file logging. Is there any way you guys could add even just a basic implementation of NReco file logging to your README please ? Would help me out immensely.

I saw you guys said it's similar to ConsoleLogger so I looked that up and didn't come across simple code samples for that either.

Thanks.

Alternative handling of IConfiguration in AddFile

Currently AddFile(IConfiguraton) methods expect that passed config section is a parent (usually, "Logging" section) for "File" section and this section name ("File") is hardcoded. It might be a need to use another section name, or have more than one file logging provider that are configured in appsettings.json. It easy to modify configuration processing in this way:

• if passed IConfiguration contains "File" subsection use it as before (which guarantees full backward compatibility)
• otherwise, if passed IConfiguration contain "Path" property (which is only required property to enable file logger), use it as a "File" section to handle "File" configuration section that is passed explicitly, for example loggingBuilder.AddFile(loggingSection.GetSection("FileLogAll"))

Documentation

Some documentation and an actual example would be nice.

Logger not adding anything below debug warning level

So ive done same as others

"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Debug",
"Microsoft.Hosting.Lifetime": "Debug"
},
"File": {
"MinLevel": "Debug",
"Path": "app.log",
"Append": "True",
"FileSizeLimitBytes": 10485760, // 10 * 1024 * 1024 => 10MB
"MaxRollingFiles": 3 // use to specify max number of log files
}
}

and it seems anything belowWarning is not added to log file.
Can you help on this as im about to use another logger.

_logger.Debug - not added to log file

_logger.Warning - added to log file

also ive used to load settings with code below.

services.AddLogging(loggingBuilder =>
{
var loggingSection = hostContext.Configuration.GetSection("Logging");
loggingBuilder.AddFile(loggingSection);
});

AddFile(fileName, append) does not works as expected

AddFile(fileName, false) and AddFile(fileName, true) are works in the same way. When 'append = false' is expected behaviour is:

• First launch logs is saved to file
• Next launch logs is saved to file, but previous logs are removed and does not present in file

Can you please look at this problem?

Add 'UseUtcTimestamp' option to output timestamps in UTC time.

Like Microsoft.Extensions.Logging.Console.ConsoleFormatterOptions has UseUtcTimestamp property would be nice to have similar option for FileLoggerOptions .

ASP .NET Core 5 usage: cannot read log file content - The process cannot access the file '...' because it is being used by another process

Hello.
I use your library in ASP .NET Core 5 Web app. In Startup.cs inside ConfigureServices(IServiceCollection services) I have:

services.AddLogging(loggingBuilder => {
                loggingBuilder.AddFile("Logs/app.log", options => {
                   options.Append = true;
                    options.MinLevel = LogLevel.Warning;
                    options.FileSizeLimitBytes = 5 * 1024 * 1024;
                    options.MaxRollingFiles = 1000;
                    });
            });

In Program.cs I resolve Ilogger via host.Services.GetService<ILogger<TelegramBot>>() and pass it to the constructor of a TelegramBot class. Inside the constructor logger is saved in local variable that is later used to log some events with logger.LogWarning() (specifically - bot start\stop events).
On some page in my application I'm trying to read log file content with the following code (start\stop events are definitely doesn't happend and logger.LogWarning() is definitely not called at that moment):

            try
            {
                using (FileStream stream = File.Open(filePath, FileMode.Open, FileAccess.Read, FileShare.Read))
                ....
            }
            catch (IOException ex)
            {
                //Here I get 'The process cannot access the file '...' because it is being used by another process'
            }

but I get IOException

The process cannot access the file '...' because it is being used by another process

So what's wrong with my library usage? How can I properly read log file content?
Can it be that logger unblocks log file only on Dispose() event? Should it live as short as possible (resolve->write log->dispose)?

Can't Get it to log anything...

I am trying to setup basic logging to a file. I followed the documention on the main page and it makes a log file called app.log. But it only ever logs startup stuff::

2020-04-03T15:56:06.2621145-06:00 INFO [Microsoft.Hosting.Lifetime] [0] Application started. Press Ctrl+C to shut down.
2020-04-03T15:56:06.2656741-06:00 INFO [Microsoft.Hosting.Lifetime] [0] Hosting environment: Development
2020-04-03T15:56:06.2669093-06:00 INFO [Microsoft.Hosting.Lifetime] [0] Content root path: C:\my\project\path\here

I setup my appsettings.json like this:

"Logging": {
  "LogLevel": {
    "Default": "Debug",
    "System": "Debug",
    "Microsoft": "Debug"
  },
  "File": {
    "Path": "app.log",
    "Append": "True",
    "FileSizeLimitBytes": 1000000, 
    "MaxRollingFiles": 5 
  }
}

And I have this in my ConfigureServices method in Startup.cs:

 services.AddLogging(loggingBuilder =>
 {
      var loggingSection = Configuration.GetSection("Logging");
      loggingBuilder.AddFile(loggingSection);
 });

And lastly, in one of my classes I have this:

private readonly Microsoft.Extensions.Logging.ILogger<MyClassNameHere> fileLog;

public MyClassNameHere( Microsoft.Extensions.Logging.ILogger<MyClassNameHere> fileLog)
{
     this.fileLog = fileLog;
}

public async Task<MyResponse> DoSomething()
{
     fileLog.LogDebug("Log Stuff");
}

A few notes:

• I can see that the nroc logger is attached to the ILogger instance and has the min log level set to debug.
• I can also see that the other settings from the config file are present in the logger.

What am I missing to get this to log to a file?

Cannot access logName when implementing FileLoggerOptions.FormatLogEntry

Great project!

I only noticed one minor issue when trying to add some additional information to the log output like the thread ID.

We can set FileLoggerOptions.FormatLogEntry to customize the text that is written to the log file.

The original code in Log<TState> includes logBuilder.Append(logName). When using FormatLogEntry it doesn't seem possible to read logName in order to write it to the text file. Or am I missing anything?