GithubHelp home page GithubHelp logo

Comments (6)

jeesim2 avatar jeesim2 commented on July 29, 2024

I'm sorry I'm new to mark down.

from fluent-plugin-elasticsearch.

dterror-zz avatar dterror-zz commented on July 29, 2024

Hi, what version of the plugin are you using? and also, can you post what your configuration looks like?

from fluent-plugin-elasticsearch.

jeesim2 avatar jeesim2 commented on July 29, 2024

thanks!

version

[test@testbin]$ fluent-gem list
*** LOCAL GEMS ***
elasticsearch (0.4.11)
elasticsearch-api (0.4.11)
elasticsearch-transport (0.4.11)
fluent-plugin-elasticsearch (0.3.0)
fluentd (0.10.53, 0.10.48)
fluentd-ui (0.2.0)

configuration

<source>
  type http
  port 18888
  bind 0.0.0.0
  body_size_limit 32m
  keepalive_timeout 10s
</source>

<match dodol.summary.real>
  type elasticsearch
  logstash_format true
  logstash_prefix summary
  logstash_dateformat %Y
  utc_index false
  host 10.98.168.207 
  port 9200 
</match>

from fluent-plugin-elasticsearch.

jeesim2 avatar jeesim2 commented on July 29, 2024

I tried on lasted version of plugin.
then @timestamp value of the json indexed to elasticsearch @timestamp field correctly :)

but the type of @timestamp created as string.
should I set a mapping to @timestamp field before index? like this?

{
  "@timestamp": {    "type" : "date",   "format" : "yyyy/MM/dd HH:mm:ss"}
}

when I send a timestamp(unix time) parameter to fluent http source
timestamp indexed as date field

# data to fluentd
'http://localhost:18888/my.tag?time=123908213&json=~~~~

# index mapping created automatically
"@timestamp" : {
"format" : "dateOptionalTime",
"type" : "date"
},

from fluent-plugin-elasticsearch.

jeesim2 avatar jeesim2 commented on July 29, 2024

OK,
It was also my fault.
@timestamp format was wrong.

this is right

2014-04-07T000:00:00-00:00

instead of

2014-04-07T00:00:00-00:00

please check the document timestamp format

By default, when inserting records in logstash format, @timestamp is dynamically created with the time at log ingestion. If you'd like to use a custom time. Include an @timestamp with your record.
{"@timestamp":"2014-04-07T000:00:00-00:00"}

from fluent-plugin-elasticsearch.

dterror-zz avatar dterror-zz commented on July 29, 2024

Thanks @jeesim2

from fluent-plugin-elasticsearch.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.