Logging with Serilog

The Suite Framework makes usage of Serilog as the underlying logger implementation.

Serilog features a Structured Logger, compared to the Text Loggers that we previously used like Log4Net, Serilog receives a String Template and a Data Structure.

Serilog's structured logger integrates seamlessly with .NET's ILogger abstractions, so you won't even notice it when coding your features. You will keep using ILogger<T> in your applications and modules.

The Suite Framework strives to provide a seamless experience with the previous logging scaffolding, by keeping away the developer from the burden of configuring additional logging stuff. Some of the actions taken in that direction were:

• Default console sink: A default Console Sink is available out of the box in the Suite Framework to provide the developers the same experience they have had until now.
• Logging configured in templates: The templates used to create new applications from .NET CLI include Elasticsearch Sink, in addition to the Console Sink mentioned before, with the aim to facilitate the log aggregation in an external repository.

Important

Most of the time it will be enough having only the Console sink, which is added by default. If it doesn't fit your needs, feel free to add more Sinks as is explained later on in this section.

By using Elasticsearch in the our services we expect to gather the logs produced by all the applications, and infrastructure components, in the same repository, to facilitate further analysis and troubleshooting. Take this into account when developing applications or components that must be monitored.

Serilog as middle tier

Serilog acts as a middle tier between the ILogger<T> from .NET and the final destinations. The destination in the Serilog's world is called Sinks.

Serilog will take care of the structured logging process, decomposing the elements of the log entires to store them apart from the message itself, leveraging further search and analysis processes.

The Suite Framework depends on Serilog to produce all its logs in a structured way. There is no way to opt-out this feature since it is embedded at the framework level. The reason for that is that we wanted to have logging as soon as possible in our services, that is why you can inject an ILogger<T> in your Suite Modules.

Best Practices for Structured Logging

There are some ground rules to have good structured logs:

1. Never use String Interpolation for the message being logged. This is because Serilog uses the message as a template, and it will store the template apart from the values, aiming to identify the messages of the same type.
2. Use always meaningful placeholders. The placeholders can be used later on during the analysis, so do not use positional placeholders ({0}, {1}, etc).
3. Don't use dots in placeholder property names, for example {Customer.Id}. Serilog doesn't support it.
4. Use @, the destructuring directive, to store the whole object's content serialized instead of its string representation (through ToString() method invocation). See here for a more detailed explanation.

5. When logging exception, take care of placing the exception as the first argument, otherwise the exception could be handled as a format parameter or argument, preventing the logger to dump the stack trace and other exception's valuable information. For example, always do:

       logger.LogError(exception, "An exception occurred");
   

   instead of:

       logger.LogError("An exception occurred: {exception}", exception);
   

Configuring sinks

By default the Suite Framework configures the Console Sink. This sink act as a replacement for the conventional one provided by Microsoft.Extensions.Logging package. The console log cannot be disabled, it will there anytime you run your application because the runtime pick it up, configures using IConfiguration mechanism, at bootstrap time.

There is another Sink pre-defined in the Suite Framework called Elasticsearch, which is responsible to route the log entries and events an external Elasticsearch log repository. Every application must relay in an external durable sink to centralize its logs, this way we will be able to do better troubleshooting with cross-application integrated logs, by using queries that comprises multiple application we can get tremendous benefits out of it.

The Elasticsearch Sink is included in the ITsynch.Suite.Observability library, and it can be enabled by calling the ConfigureSuiteObservability method over the IHostBuilder before calling ConfigureSuiteWebAppForRootModule, as following:

    private static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureSuiteObservability()
            .ConfigureSuiteWebAppForRootModule<BasicSampleModule>();

You can still plug any other Sink of your interest, for example File Sink or Seq Sink using the Serilog standard configuration, but we encourage you to use only Elasticsearch, besides of the Console Sink mentioned before, in your services projects.

The templates provided to create applications for .NET provides a basic configuration to start a new application with Elasticsearch enabled.

Configuring Elasticsearch

The configuration of the default Elasticsearch Sink is handled by the ITsynch.Suite.Observability library, which can be configured using any method supported by IConfiguration. The configuration structure for the appsettings.json is shown below:

{
    "ObservabilityOptions": {
        "ElasticsearchUri": "http://localhost:9200"
    }
}

Due to the needs of configuration externalization, we need to provide a way to configure the Elasticsearch host using Environment Variables. At development time appsettings.json file is enough to configure local or remote Elasticsearch host. But for production environment, there must be a way to provide Elasticsearch URL from the running environment without the needs of human intervention.

You can use the ObservabilityOptions_ElasticsearchUri Environment Variable to supply the URL of the Elasticsearch host to use.

Pay attention to the tokens composing the variable identifier:

• ObservabilityOptions: points to the Observability module configuration section
• ElasticsearchUri: points to the attribute of the target host where the sink must send the logs to

As you may already know, Serilog requires an Index Pattern to be able to register the logs on Elasticsearch. This Index Pattern is automatically set by the ITsynch.Suite.Observability library and cannot be changed. The pattern is "Suite-App-{0:yyyy.MM}".

Add additional sinks

Serilog is initialized reading its configuration from the IConfiguration object of the .NET . As such, if you need to configure additional Sinks you can do as follows:

1. Add the desired Sink to the Serilog's configuration. The easiest way is to modify the appsettings.json file, by adding a new WriteTo entry with the name of the Sink, and its configuration if any is needed for that Sink.
2. Add the Sink package dependency to your runtime project. Find the correct package and add it to the references. At runtime Serilog will scan every project which depends on Serilog's package to find dependencies. Therefor, by letting your runtime project depend on the Sink's package you're telling Serilog to look for dependencies in your project's dependency graph too. In this manner Serilog is able to find, and load, its dependencies at runtime.

Throughout the next example you will see how to add a new Sink to our runtime project by adding the package dependency first, and finally by adding the sink to Serilog configuration.

First step: add the Sink package to the runtime project. In this case we are using the File Sink which allows us to write the log entries to a file.

...
    <ItemGroup>
        <PackageReference Include="Serilog.Sinks.File" />
    </ItemGroup>
...

Second step: add the Sink to Serilog's configuration through appsettings.json application's configuration file.

"Serilog": {
    "MinimumLevel": {
        "Default": "Information"
    },
    "WriteTo": [
        {
            "Name": "File",
            "Args": {
                "path": "MyApp-logs.txt",
                "rollingInterval": "Day"
            }
        }
    ]
}

Warn

Always append additional Sinks to the end of the already defined ones. The order in which they appear is important due to the fact that any configuration supplied from environment variables is pointing to Sink configuration through its relative position in the json array of the appsettings.json file.