Comments (7)
Can you share a bit of background on how you plan to query and use this data? Especially interested in the collector and relay in the above.
I definitively see value on knowing through which hops an event went at the same time this can contain more then just 3 hops as you pointed out in the above with relay0, relay1 and don't know yet how we should cover that. Would order matter? What I was thinking so far is that originator = agent.
from ecs.
One use case: currently we collect logs from our switches this way:
- switch sends syslog events to rsyslog server [switch is the originator]
- filebeat forwards logs from rsyslog logged events to Logstash [filebeat is agent, but not originator, it is just relay this time]
- logstash receives logs from filebeat and forwards it to kafka [logstash is relay]
- another logstash reads logs from kafka and stores it in elasticsearch [this logstash is collector]
But lets say, 3+4 is collector. For us it is important to know the originator IP and hostname, the filebeat (here relay) IP and hostname and IP address of peer from which the collector received logs from relay (can be different from relay management IP because of NAT for example).
Would order matter?
Probably.
from ecs.
So in your case you are interested in the switch and filebeat info but not the other hops? Or you would add all of them?
from ecs.
Yes, in this case I am interested about the switch and filebeat info. Btw. this use case also shows I need to distinguish log data from metadata. For switch log it is not clear if something like device.originator is related to the event generator or to some event content (for example 802.1x auth info where for the first time device.originator can look like the device authenticating to the switch).
from ecs.
I'm hesitant on the value of adding too much metadata surrounding how events are gathered or transported to this common schema, at least this early.
If we clarified how to use ECS while also adding other fields that aren't part of ECS (or not yet), would that help? I've seen internal discussions about this, but it hasn't really been communicated in this repo yet.
from ecs.
++ on the comment from @webmat
One thing that we started for such proposal is that it can be provided as a use case (see notes there): https://github.com/elastic/ecs#use-cases
from ecs.
observer.*
has been introduced, which I think addresses some of the original concerns here, in part.
There's a meta issue in #940 tracking past issues/discussions around pipeline support and guidance in ECS. I've noted this issue in the list captured there.
If there are any current thoughts or concerns on the topic, let's take the discussion there.
from ecs.
Related Issues (20)
- Clarify the type of container disk and network metrics HOT 1
- ECS Vulnerability Published field
- Add threat.indicator.tags field
- [Proposal] Make event.kind a list HOT 3
- Incorrect generated/beats/fields.ecs.yml, not accounting on top_level: false HOT 1
- [ECS] Addition - http.request.header.bytes & http.response.header.bytes HOT 2
- Add lowercase normaliser to ECS fields which support security incident response process
- Mark experimental fields as `beta` in generated files
- Elastic-Agent Integrations Use of Legacy Mapping Types Impacts .fleet_globals & prevents agents from being upgraded
- Add `related.url` field
- Add event.zone and event.environment fields
- Addition of additional allowed values for event.type
- Support cloud events in schema HOT 1
- Better abstraction of the type event.kind: alert
- ECS can no longer map all components out of the box HOT 13
- Define a standard way to identify prevention and detection security alerts HOT 5
- Support multi-key fields from SemConv HOT 5
- Allow risk object to be nested under network.
- Add a multi-field user.id.text to the user.id field.
- [Discuss] Add `agent.group` and `host.group` field
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ecs.