Comments (6)
I'm sorry I'm new to mark down.
from fluent-plugin-elasticsearch.
Hi, what version of the plugin are you using? and also, can you post what your configuration looks like?
from fluent-plugin-elasticsearch.
thanks!
version
[test@testbin]$ fluent-gem list
*** LOCAL GEMS ***
elasticsearch (0.4.11)
elasticsearch-api (0.4.11)
elasticsearch-transport (0.4.11)
fluent-plugin-elasticsearch (0.3.0)
fluentd (0.10.53, 0.10.48)
fluentd-ui (0.2.0)
configuration
<source>
type http
port 18888
bind 0.0.0.0
body_size_limit 32m
keepalive_timeout 10s
</source>
<match dodol.summary.real>
type elasticsearch
logstash_format true
logstash_prefix summary
logstash_dateformat %Y
utc_index false
host 10.98.168.207
port 9200
</match>
from fluent-plugin-elasticsearch.
I tried on lasted version of plugin.
then @timestamp value of the json indexed to elasticsearch @timestamp field correctly :)
but the type of @timestamp created as string.
should I set a mapping to @timestamp field before index? like this?
{
"@timestamp": { "type" : "date", "format" : "yyyy/MM/dd HH:mm:ss"}
}
when I send a timestamp(unix time) parameter to fluent http source
timestamp indexed as date field
# data to fluentd
'http://localhost:18888/my.tag?time=123908213&json=~~~~
# index mapping created automatically
"@timestamp" : {
"format" : "dateOptionalTime",
"type" : "date"
},
from fluent-plugin-elasticsearch.
OK,
It was also my fault.
@timestamp format was wrong.
this is right
2014-04-07T000:00:00-00:00
instead of
2014-04-07T00:00:00-00:00
please check the document timestamp format
By default, when inserting records in logstash format, @timestamp is dynamically created with the time at log ingestion. If you'd like to use a custom time. Include an @timestamp with your record.
{"@timestamp":"2014-04-07T000:00:00-00:00"}
from fluent-plugin-elasticsearch.
Thanks @jeesim2
from fluent-plugin-elasticsearch.
Related Issues (20)
- ILM policy not created with data_stream_ilm_policy
- Fluentd stops shipping logs to elasticsearch after hitting non UTF-8 character in log HOT 2
- Unable to activate fluent-plugin-elasticsearch-5.3.0, because faraday-1.10.3 conflicts with faraday (>= 2.0.0), faraday-excon-1.1.0 conflicts with faraday-excon (>= 2.0.0) HOT 1
- Not able to send syslogs to Elasticsearch using fluent-plugin-elasticsearch HOT 1
- Could not communicate to OpenSearch, resetting connection and trying again. no address for <OpenSearch host> HOT 2
- installed un-desired version of elasticsearch client HOT 1
- Installed un-desired version of elasticsearch client HOT 3
- How to get rid of debug log "Detected ES 8.x or above: document type will not be used" HOT 5
- Problems routing to dynamically named data streams HOT 2
- uninitialized constant Elasticsearch::Transport (NameError) HOT 1
- `template_file` and `customize_template` ignored for Data Streams HOT 3
- Getting 400 status in fluentd - request body is required HOT 3
- Load Balancer behaviour clarification HOT 1
- How does load balancing work with multiple replicas
- ILM policy creation failed HOT 5
- in_elasticsearch client.clear_scroll does not work when elastic return extremely long scroll_id HOT 1
- Rejected by Elasticsearch [error type]: document_parsing_exception [reason]: '[1:660] failed to parse field [kubernetes.labels.app] of type [text] in document with id HOT 21
- failed to flush the buffer (use https, but elasticsearch received http traffic) HOT 1
- Question: id_key in @type elasticsearch_data_stream doesn't function as expected ... HOT 3
- Pipeline not executed with datastreams
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluent-plugin-elasticsearch.