fantasticfiasco / serilog-sinks-http-sample-elastic-stack

Sample application of Serilog.Sinks.Http sending log events to Elastic Stack.

Serilog and the Elastic Stack

Table of contents

• Introduction
• What you will end up with
• Requirements
• Usage
  • Bringing up Elastic Stack
  • Publishing log events using Serilog
  • Using Kibana to render the log events
• Credit

---

Introduction

Elastic Stack is fantastic at collecting and visualizing log events. Serilog is fantastic at producing structured log events. This repository provides a sandbox where developers can explore the life of a log event starting with its birth in Serilog, its transport over the network to Logstash, its fields being indexed by Elasticsearch and finally its legacy being recorded as a historical event in Kibana.

What you will end up with

With a running Elastic Stack and Serilog producing log events you are now ready to take it to the next level. If you fancy the producing part you'll dig deeper into Serilog and its configuration of log contexts, enrichers and message formatters. If you enjoy monitoring applications in production you'll explore Kibana with its visualizations and dashboards.

Requirements

• Docker

Usage

Bringing up Elastic Stack

Start the stack using docker:

cd .\elastic-stack\

# This command is only necessary the first time the stack is started
docker compose up setup

docker compose up

Publishing log events using Serilog

Run the following commands to publish log events to Logstash using Serilog:

cd .\serilog\
docker compose up

If you decide to run the application outside of Docker in your terminal, don't forget to change the request URI to http://localhost:31311. More information can be found in .\serilog\Program.cs.

Using Kibana to render the log events

Access the Kibana web UI by hitting http://localhost:5601 with a web browser, and when prompted enter username elastic and password changeme.

Credit

The elastic-stack directory is a clone of docker-elk with minor modifications. Credit to deviantony for publishing the Elastic Stack boilerplate.

serilog-sinks-http-sample-elastic-stack's Issues

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Port 5000 already used on macOS

Describe the bug
Port 5000 on macOS is already in use if you're using Airplay. Let's configure Logstash to use another port.

To Reproduce
Just follow the instructions in README.md

Desktop (please complete the following information):

• OS: macOS

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

docker-compose

elastic-stack/docker-compose.yml

serilog/docker-compose.yml

dockerfile

elastic-stack/elasticsearch/Dockerfile

elastic-stack/kibana/Dockerfile

elastic-stack/logstash/Dockerfile

elastic-stack/setup/Dockerfile

serilog/Dockerfile

• mcr.microsoft.com/dotnet/sdk 8.0-alpine

github-actions

.github/workflows/ci-cd.yml

• actions/checkout v4
• docker/setup-buildx-action v3

.github/workflows/update-copyright-years-in-license-file.yml

• actions/checkout v4
• FantasticFiasco/action-update-license-year v3

nuget

serilog/serilog-example.csproj

• Serilog.Sinks.Http 9.0.0
• Serilog.Sinks.Console 5.0.1
• Serilog.Extensions.Logging 8.0.0
• Bogus 35.5.1

---

• Check this box to trigger a request for Renovate to run again on this repository

http://logstash:31311 -> http://localhost:31311

Describe the bug
The URL http://logstash:31311 is written incorrectly.
https://github.com/FantasticFiasco/serilog-sinks-http-sample-elastic-stack/blob/master/serilog/Program.cs#L16

To Reproduce
In my case, The URL http://localhost:31311 works normally.

LogsStash confing does not handle bundled events

LogStash config in your example:

http {
    codec => json
}

This does not work with JSON format this sink is sending (all events are treated as single message by logstash):

{
  "events": [
    {
        <..>
    }
  ]
}

What I'm missing?