nihiue / loki-enhance-middleware

Add GeoIP & UA-Parser support for Grafana Loki

😃 About Me

• Developer and Geek

• Keep 3 Cats 🐱😽😺

• Tech Blog

• Career

          

⚡ Overview

🚣 Application

🚀 Library

🎮 Fun & Hack

⛏️ Tools

💡 Demo

loki-enhance-middleware's Issues

Need help with expressions.

Hi, I would love to use this, but I dont know how to setup the expression via json.

Here is an example of my json output (json is written to file per line)
{"ct_id":0,"ct_mark":0,"detected_application":303,"detected_application_name":"126.netify.google","detected_protocol":188,"detected_protocol_name":"QUIC","detection_guessed":1,"dhc_hit":true,"digest":"99b10c8d5fa0dd3fdd17abe1bfab5b82bad037f2","first_seen_at":1674703379228,"first_update_at":1674703379228,"host_server_name":"clients2.google.com","ip_nat":false,"ip_protocol":17,"ip_version":4,"last_seen_at":1674703379269,"local_ip":"10.0.5.143","local_mac":"c8:2a:dd:88:de:e3","local_origin":true,"local_port":40180,"other_ip":"142.250.176.14","other_mac":"20:6d:31:11:10:d2","other_port":443,"other_type":"remote","vlan_id":0,"interface":"br-lan","internal":true,"type":"flow"}

Here is my promtail.yml config

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://lokimiddleware:3100/loki/api/v1/push

- job_name: netify-logs
  static_configs:
  - targets:
      - localhost
    labels:
      job: netifylogs
      __path__: /var/log/netify/*log
    pipeline_stages:
      - json:
          expressions:
            detected_protocol_name: detected_protocol_name
            host_server_name: host_server_name
            local_ip: local_ip
            local_mac: local_mac
            local_port: local_port
            other_ip: local_ip
            other_ip: other_ip
            other_mac: other_mac
            other_port: other_port
      - labels:
          host_server_name:
          local_ip:
          local_mac:
          local_port:
          other_ip:
          other_mac:
          other_port:
      - template:
          source: output_msg
          template: 'detected_protocol_name="{{ .detected_protocol_name }}" local_ip="{{ .local_ip }}" local_mac="{{ .local_mac }}"  local_port="{{ .local_port }}"  GeoIP_Source="{{ .other_ip }}" other_mac="{{ .other_mac }}" other_port="{{ .other_port }}"'
      - output:
          source: output_msg

Support Basic auth

I am not entirely sure, but it seems like this middleware will not pass on the basic auth, meaning you cannot use it if you have authentication enabled for pushing logs to loki.

Could it make sense to ensure that if basic auth is in the request, then it will be passed along?