GithubHelp home page GithubHelp logo

logstash-plugins / logstash-filter-http Goto Github PK

View Code? Open in Web Editor NEW

This project forked from lucashenning/logstash-filter-rest

12.0 7.0 29.0 133 KB

HTTP Filter Plugin for Logstash

License: Apache License 2.0

Ruby 100.00%

logstash-filter-http's Introduction

Logstash REST Filter Build Status

This is a filter plugin for Logstash.

It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.

Documentation

This logstash filter provides an easy way to access RESTful Resources within logstash. It can be used to post data to a REST API or to gather data and save it in your log file.

Usage

1. Installation

You can use the built-in plugin tool of Logstash to install the filter:

$LS_HOME/bin/logstash-plugin install logstash-filter-rest

Or you can build it yourself:

git clone https://github.com/lucashenning/logstash-filter-rest.git
bundle install
gem build logstash-filter-rest.gemspec
$LS_HOME/bin/logstash-plugin install logstash-filter-rest-0.1.0.gem

2. Filter Configuration

Add the following inside the filter section of your logstash configuration:

filter {
  rest {
    request => {
      url => "http://example.com"        # string (required, with field reference: "http://example.com?id=%{id}" or params, if defined)
      method => "post"                   # string (optional, default = "get")
      headers => {                       # hash (optional)
        "key1" => "value1"
        "key2" => "value2"
      }
      auth => {
        user => "AzureDiamond"
        password => "hunter2"
      }
      params => {                        # hash (optional, available for method => "get" and "post"; if post it will be transformed into body hash and posted as json)
        "key1" => "value1"
        "key2" => "value2"
        "key3" => "%{somefield}"         # sprintf is used implicitly
      }
    }
    json => true                         # boolean (optional, default = true)
    target => "my_key"                   # string (mandatory, no default)
    fallback => {                        # hash describing a default in case of error
      "key1" => "value1"
      "key2" => "value2"
    }
  }
}

Print plugin version:

bin/logstash-plugin list --verbose | grep rest

Examples for running logstash from cli:

bin/logstash --debug -e 'input { stdin{} } filter { rest { request => { url => "https://jsonplaceholder.typicode.com/posts" method => "post" params => { "userId" => "%{message}" } headers => { "Content-Type" => "application/json" } } target => 'rest' } } output {stdout { codec => rubydebug }}'
bin/logstash --debug -e 'input { stdin{} } filter { rest { request => { url => "https://jsonplaceholder.typicode.com/posts" method => "post" body => { "userId" => "%{message}" } headers => { "Content-Type" => "application/json" } } target => 'rest' } } output {stdout { codec => rubydebug }}'
bin/logstash --debug -e 'input { stdin{} } filter { rest { request => { url => "http://jsonplaceholder.typicode.com/users/%{message}" } target => 'rest' } } output {stdout { codec => rubydebug }}'
bin/logstash --debug -e 'input { stdin{} } filter { rest { request => { url => "https://jsonplaceholder.typicode.com/posts" method => "get" params => { "userId" => "%{message}" } headers => { "Content-Type" => "application/json" } } target => 'rest' } } output {stdout { codec => rubydebug }}'

Contributing

All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.

Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.

It is more important to the community that you are able to contribute.

For more information about contributing, see the CONTRIBUTING file.

logstash-filter-http's People

Contributors

edmocosta avatar fylie avatar gandalfb avatar jsvd avatar kaisecheng avatar karenzone avatar kares avatar lazaroofarrill avatar logstashmachine avatar lucashenning avatar mashhurs avatar mschneider82 avatar yaauie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

logstash-filter-http's Issues

Is it possible to do PUT requests from this plugin

Hiya,

Is it possible to do PUT requests from this plugin, From the documentation it doesn't look like it can however unless I'm mistaken the only thing stopping the plugin from doing it is PUT requests aren't in the VALID_VERBS list?

Thanks

JR

Debug message causes Fatal error

I have a very simple pipeline using the following http filter, but I continue to get errors (see here: https://gist.github.com/defensivedepth/7b8d22aeec6dedac6438d8adeaed4b1d 1)

I have tried different sites, different settings, but always come back to this error. Logstash 6.6.1

Appears to be a bug - https://discuss.elastic.co/t/http-filter-undefined-local-variable-or-method-parameters-sprintfed/169905?


input {
file {
path => "/input*.json"
codec => json
}

}

filter {
http {
url => "https://site.com/%{identifier}"
}

}

output {
stdout {}
}

Plugin crashes on empty response body

If a web service responds with HTTP/1.0 204 No Content and an empty response body the plugin crashes. This is due to the invocation of body.strip on this line.

[2020-10-15T09:39:51,324][DEBUG][logstash.filters.http    ][main][bccba3b61c91111734d8fbaddeef151ae2e9d3ff77dae9376279dd322eb0bac6] success received {:code=>204, :body=>nil}
[2020-10-15T09:39:51,352][ERROR][logstash.javapipeline    ][main] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"main", :error=>"(NoMethodError) undefined method `strip' for nil:NilClass", :exception=>Java::OrgJrubyExceptions::NoMethodError, :backtrace=>["usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_http_minus_1_dot_0_dot_2.lib.logstash.filters.http.process_response(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-http-1.0.2/lib/logstash/filters/http.rb:116)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_http_minus_1_dot_0_dot_2.lib.logstash.filters.http.filter(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-http-1.0.2/lib/logstash/filters/http.rb:76)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.do_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:159)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:178)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1809)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:175)", "org.logstash.config.ir.compiler.AbstractFilterDelegatorExt.multi_filter(org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:134)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:293)"], :thread=>"#<Thread:0x30e761f3 sleep>"}
[2020-10-15T09:39:51,513][DEBUG][logstash.filters.http    ][main] Closing {:plugin=>"LogStash::Filters::Http"}
[2020-10-15T09:39:51,531][DEBUG][logstash.filters.http    ][main] Closing {:plugin=>"LogStash::Filters::Http"}
[2020-10-15T09:39:51,532][DEBUG][logstash.filters.http    ][main] Closing {:plugin=>"LogStash::Filters::Http"}
[2020-10-15T09:39:51,939][INFO ][logstash.runner          ] Logstash shut down.

Handle Empty body in HTTP response

When I request an API with an empty body response, the filter crashes and the pipeline is interrupted

Here the error message :

| [2022-04-20T07:46:02,164][ERROR][logstash.javapipeline ] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"Test", :error=>"(NoMethodError) undefined method `strip' for nil:NilClass", :exception=>Java::OrgJrubyExceptions::NoMethodError, :backtrace=>["usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_http_minus_1_dot_0_dot_2.lib.logstash.filters.http.process_response(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-http-1.0.2/lib/logstash/filters/http.rb:116)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_http_minus_1_dot_0_dot_2.lib.logstash.filters.http.filter(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-http-1.0.2/lib/logstash/filters/http.rb:76)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.do_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:159)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:178)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1809)", "usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:175)", "org.logstash.config.ir.compiler.AbstractFilterDelegatorExt.multi_filter(org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:134)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:293)"], :thread=>"#<Thread:0x610d43eb sleep>"}

To solve this issue, I have modified the code in lib\logstash\filters\http.rb and added theses lines :

if body.nil?
   body = ""
end

After the line 136 in http.rb (after "event.set(@target_headers, headers)" before "if content_type == "application/json"").

Maybe other users have the same issue, so it could be a good idea to add this code to the plugin, or any other method to manage this case

If you what to reproduce, the Kibana API to delete a case doesn't reponse with a body.
Doc:
https://www.elastic.co/guide/en/security/current/cases-api-delete-case.html

Plugin crashes when it receives Array type of content-type header.

Logstash information:

Please include the following information:

  1. Logstash version (e.g. bin/logstash --version)
  2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker)
  3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
  4. How was the Logstash Plugin installed

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

  1. JVM version (java -version)
  2. JVM installation source (e.g. from the Operating System's package manager, from source, etc).
  3. Value of the JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system):

Description of the problem including expected versus actual behavior:

Steps to reproduce:

  • Create and run the server which returns list of content-type headers, example nodejs source
var http = require("http");

http.createServer(function (request, response) {
   response.writeHead(200, {'Content-Type': ['text/plain', 'logstash/custom-type']} );
   response.end('Hello World\n');
}).listen(8081);

console.log('Server running at http://127.0.0.1:8081/');
  • Config Logstash with filter-http which calls and receives list of content-type headers
filter {
    http {
        url => "http://localhost:8081"
        target_body => "body"
        target_headers => "headers"
    }
}
  • Run Logstash and observe the error

Provide logs (if relevant):

[2023-03-15T11:36:25,372][ERROR][logstash.javapipeline    ][main] Pipeline worker error, the pipeline will be stopped 
{:pipeline_id=>"main", :error=>"(NoMethodError) undefined method `split' for [\"text/plain\", \"logstash/custom-type\"]:Array", 
:exception=>Java::OrgJrubyExceptions::NoMethodError, :backtrace=>
["RUBY.process_response(/Users/mashhur/Dev/elastic/logstash/logstash-plugins/logstash-filter-
http/lib/logstash/filters/http.rb:136)", "RUBY.filter(/Users/mashhur/Dev/elastic/logstash/logstash-plugins/logstash-filter-
http/lib/logstash/filters/http.rb:96)", "RUBY.do_filter(/Users/mashhur/Dev/elastic/logstash/logstash-
core/lib/logstash/filters/base.rb:159)", "RUBY.multi_filter(/Users/mashhur/Dev/elastic/logstash/logstash-
core/lib/logstash/filters/base.rb:178)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1865)", 
"RUBY.multi_filter(/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/filters/base.rb:175)", 
"org.logstash.config.ir.compiler.AbstractFilterDelegatorExt.multi_filter(org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.ja
va:133)", "RUBY.start_workers(/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/java_pipeline.rb:304)"], 
:thread=>"#<Thread:0x7fc1e9a8@/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 sleep>"}

Friendlier handling of non 200 response codes

Currently anything that's not a 200 means the plugin won't process headers or the body.
In many cases this isn't ideal as these are still valuable for debugging / observability purposes.

A way to improve this would be to always process the body and headers, but only apply the filter_matched method on successful requests (200 status code)

Doesn't respect quoted characters

Logstash information:

Please include the following information:

  1. Logstash version (e.g. bin/logstash --version) 8.5.1 but also tested with 7.17.8
  2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker)
  3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
  4. How was the Logstash Plugin installed

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

  1. JVM version (java -version)
  2. JVM installation source (e.g. from the Operating System's package manager, from source, etc).
  3. Value of the JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system): any

Description of the problem including expected versus actual behavior:
When an field of the event contains quoted characters, like \" the payload sent to the server should also contains the same quoting.

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

  1. Run an http mock server
wget https://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-jre8-standalone/2.35.0/wiremock-jre8-standalone-2.35.0.jar

java -jar wiremock-jre8-standalone-2.35.0.jar --verbose 
  1. run the following pipeline
input {
generator {
  count => 1
  message => '{"Field1": "22222wwwww\"www\"wwwwww"}'
  codec => "json"
}
}

filter {
http {
  target_body => "response_body"
  verb => "POST"
  url => "http://localhost:8080"
  body => "%{Field1}"
}
}

output {
 stdout{
   codec => rubydebug
 }
}
  1. check the WireMock console
    It receives
22222wwwww"www"wwwwww

while we would have expected

22222wwwww\"www\"wwwwww

Provide logs (if relevant):

Apache http client wire logs

[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> POST / HTTP/1.1
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Connection: Keep-Alive
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> content-type: text/plain
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Content-Length: 21
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Host: localhost:8080
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> User-Agent: Manticore 0.9.1
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> Accept-Encoding: gzip,deflate
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "POST / HTTP/1.1[\r][\n]"
[2023-02-28T16:09:20,469][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "content-type: text/plain[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Content-Length: 21[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Host: localhost:8080[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "User-Agent: Manticore 0.9.1[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "[\r][\n]"
[2023-02-28T16:09:20,470][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 >> "22222wwwww"www"wwwwww"
[2023-02-28T16:09:20,475][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "HTTP/1.1 404 Not Found[\r][\n]"
[2023-02-28T16:09:20,475][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "Content-Type: text/plain[\r][\n]"
[2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "Transfer-Encoding: chunked[\r][\n]"
[2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "[\r][\n]"
[2023-02-28T16:09:20,476][DEBUG][org.apache.http.wire     ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << "54[\r][\n]"
[2023-02-28T16:09:20,477][DEBUG][org.apache.http.headers  ][main][2cc5221ef0bcc203d671c6ad3ff26a8083380f8f89498dbedddea98d891870d2] http-outgoing-0 << HTTP/1.1 404 Not Found

WireMock console

2023-02-28 16:09:20.472 Request received:
127.0.0.1 - POST /

Connection: [keep-alive]
Content-Type: [text/plain]
Content-Length: [21]
Host: [localhost:8080]
User-Agent: [Manticore 0.9.1]
Accept-Encoding: [gzip,deflate]
22222wwwww"www"wwwwww


Matched response definition:
(no response definition configured)

Response:
HTTP/1.1 404
(no headers)

Need clarification on sending data as json

Tell us about the issue

Description:
User has confusion about how to send data in json format as the document misses to point out the requirement of body and body_format.

body_format => "json" works only when body is an array or a hash

URL:
https://www.elastic.co/guide/en/logstash/current/plugins-filters-http.html#plugins-filters-http-body
https://www.elastic.co/guide/en/logstash/current/plugins-filters-http.html#plugins-filters-http-body_format

Anything else?

Implement native caching for higher scale lookups

There have already been some demand for native caching for HTTP lookups with this plugin. This would help enable higher throughput without the need for usage with conjunction with third-party caching systems like Memcached.

Please feel free to +1 if you are interested in this feature.

Unhelpful error message when body is hash and body_format is text

In 7.3.2, if the body option is a hash and the body_format is set to text then the resulting error message is not particularly helpful.

http { body => { "foo" => "bar" } body_format => "text" url => "http://google.com/" verb=> "GET" }

results in

[ERROR][logstash.filters.http    ] error during HTTP request {:url=>"http://google.com/", :body=>{"foo"=>"bar"}, :client_error=>"undefined method `encoding' for {\"foo\"=>\"bar\"}:Hash"}

Implement ECS-Compatibility Mode

As a part of the effort to make plugins able to run in an ECS-Compatible manner
by default in an upcoming release of Logstash, this plugin needs to implement
an ECS-Compatibility mode that does not implicitly use fields that conflict
with ECS.

Absent target_body and target_headers directives, this plugin uses root-level
fields body and headers respectively, which are undefined in ECS and
therefore have the possiblity of conflicting with future versions of ECS. The
plugin may benefit from an ECS-Compatibility mode that requires these directives
to be set explicitly or otherwise reduces the risk of future conflict.

Unable to add event field to http body

I would like to reference a value from an event field into the http body.
It doesn't seem to be possible, but it states it is possible with headers? Is it also possible with the http body?

Cannot send Content-Type header

When assigning a header, for example: "Content-Type" => "application/x-www-form-urlencoded", filter sends header, but also sends an additional "content-type: text/plain".

This breaks the API call I am attempting to make, as the end-point complains about the dual headers.

This is because the source provides a binary choice between JSON or text as content type. There are hundred of other possibilities, and this restriction severely limits the ability to make HTTP calls.

An improvement would allow any string as a value for body_format, and can still default to text/plain. Will submit a PR.

Authentication username and password are not interpolated

Logstash 7.6.2
Plugin version logstash-filter-http (1.0.2)

Summary

The http filter is not interpolating the variables for the parameters username / password.

How to reproduce

input {
    exec {
        command => 'echo "myusername mypassword something else as message"'
        interval => 30
    }
}
filter {
    grok {
        match => {
            "message" => "(?<username>.*?) (?<password>.*?) %{GREEDYDATA:message}"
        }
        overwrite => [ "message" ]
    }
    http {
        url => "https://postman-echo.com/post"
        verb => "POST"
        body_format => text
        body => "%{[message]}"
        headers => {
            "mytest" => "logstash"
        }
        user => "%{[username]}"
        password => "%{[password]}"
    }
}
output {
    stdout {
        codec => rubydebug
    }
}

Expected

The header should contain Basic bXl1c2VybmFtZTpteXBhc3N3b3Jk

Output

The header contains Basic JXtbdXNlcm5hbWVdfTole1twYXNzd29yZF19 (represents %{[username]}:%{[password]}).

Workaround

    ruby {
        init => "require 'base64'"
        code => "event.set('auth_header', 'Basic ' + Base64.strict_encode64(event.get('username')+':'+event.get('password')))"
    }
    http {
        url => "https://postman-echo.com/post"
        verb => "POST"
        body_format => text
        body => "%{[message]}"
        headers => {
            "authorization" => "%{[auth_header]}"
            "mytest" => "logstash"
        }
        remove_field => [ "auth_header"]
    }

From https://discuss.elastic.co/t/http-filter-plugin-authorization-as-parameter-values/232085/6

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.