Comments (11)
Maybe network.forwarded_ip?
from ecs.
@vbohata we defined the "device fields" to describe the entity that is actually forwarding the events to the Elastic Stack, when it is not the host on which the event happened. For example, an IDS, or network sensor, etc. In your case, the SYSLOG FORWARDER details should be mapped to the device.*
fields.
Also, as you say, the ORIGINATOR DEVICE details should be mapped to the host.*
fields, and the source.*
and dest.*
fields should capture the details of the log message.
Note that in many cases, some details in the host.*
and source.*
fields will be identical. This would allow any analysis content that uses either host fields or source fields to correctly identify the origin of the event.
Does this make sense?
from ecs.
The "device" field does not seem to be correct field, the name of this field is a little bit confusing for this use case. In doc I can see "TLS fields" according to the doc (The tls fields contain the TLS related data about a specific connection.) should describe some connection. So I think source IP of the connection should be under some another field (+ tls fields also). For example:
connection.source_ip
even for tls it make sense to put it in connection like: connection.tls.....
With this field it is more clear than with device.ip. In device.ip I do not know which IP it is. The device can have tens of IPs but I need to know the source ip used in current TCP connection. How can I handle the situation with multiple IPs?
from ecs.
@vbohata we agree that the device.*
name is confusing, and we have been trying to think of a better name (suggestions welcome), but that is the field set that is intended to describe the attributes of the intermediary device that sends events to the Elastic Stack. Commonly, the device.ip
would be populated with the management IP address of the sensor/SYSLOG FORWARDER.
There have been other discussions (e.g., #9) about creating a connection.*
field set as an "anchor" for certain fields such as TLS and other flow-related or connection-related fields. Let's see how this develops.
The source.ip
field should be populated with the actual IP address used as the source of the connection, regardless of how many interfaces a host may have.
from ecs.
The way I understand the above is that here syslog
is the agent: https://github.com/elastic/ecs#agent In most cases I would think the agent ip is not relevant but for the cases where it is, it could be put under agent.host.ip
. The host object is reused here under the agent
object.
from ecs.
While there may be a syslog agent running on the host, the sensor/SYSLOG FORWARDER is a separate entity which should map to the device.*
in the ECS model. This diagram shows what I mean.
If we don't use agent.*
as a top level field set, then I think it would fit better to have host.agent.*
rather than agent.host.*
from ecs.
It seems in the above setup the Syslog NG Server is only forwarding the event. Do we really need data about it?
What if we have 2-3 hops in between instead of only one. Do we want to store all this information?
Note: For the above I think device.*
does not fit as when I think of device I think of hardware.
from ecs.
It seems in the above setup the Syslog NG Server is only forwarding the event. Do we really need data about it?
Yes, we want to know a bit about the entity that is actually sending the data to the Elastic stack. In the case of a syslog server, we at least want to know about its IP address, identity, and location information. If it was a firewall or IDS, then we'd want to know more info, such as the rule set which was used to make a forwarding or detection decision. For example, an alert dashboard showing total event activity (e.g. network traffic, or intrusions detected) can be mapped to show which of my syslog servers is seeing the most activity, to help the analyst know how to take the next step in incident response.
What if we have 2-3 hops in between instead of only one. Do we want to store all this information?
I don't think so, but this is a good question. I was thinking that we capture just the entity that sends the event to the Elastic Stack, but what if we had an IDS, FW, or Bro sensor forwarding to a Syslog NG server? then I think the Syslog server is just an aggregator, and is less important. Better think a bit more about this one. Note, we are not including network device hops here, only logical entities involved in event creation or processing.
Note: For the above I think device.* does not fit as when I think of device I think of hardware.
As mentioned above, agreed that the device.*
name can be confusing, and we have been trying to think of a better name (suggestions welcome). "Device" was chosen since this entity is commonly a firewall, network sensor, or intrusion detection system, all of which fit more nicely into the "device" category, and some existing schemas refer to this entity as a device. For example Splunk CIM uses dvc
to refer to the IDS device that detected an intrusion, the DLP system that detects data leakage, etc. In this diagram below, we see how a network sensor, or IDS maps onto the device.*
field set.
from ecs.
To clarify one of our use cases:
FIREWALL(or another network device like SWITCH or ROUTER or...) -----> LINUX SERVER running as network log concentrator, receives logs via rsyslog and stores them to local log files
on LINUX SERVER there is also filebeat which reads stored log files and sends them to Logstash.
so: FIREWALL -> LOG CONCETRATOR (rsyslog + filebeat) -> LOGSTASH
So in the reality the FIREWALL is hardware "device" (not log concentrator/forwarder). Here even probably the "host" field should be never used as the top level entity because the FIREWALL is some host ... and also LINUX SERVER CONCENTRATOR is the host.
Maybe the device could be used only with another custom second level field which should identify the device or the device type itself like:
device.originator.host.ip
device.relay.host.ip
To be able to use multiple relays, device.relay should be array, nested or ... I dont know :)
I use the same field names (originator, relay) as https://tools.ietf.org/html/rfc5424, why to reinvent the wheel.
from ecs.
And maybe another field device.collector.peer.host.ip to identify the source ip address which is used to send logs from relay device (can be different and sometimes it is usefull to know both or just device.collector.peer.host.ip).
from ecs.
All - do we feel like this issue is still relevant?
from ecs.
Related Issues (20)
- new field set: for log documents themselves
- Clarify the type of container disk and network metrics HOT 1
- ECS Vulnerability Published field
- Add threat.indicator.tags field
- [Proposal] Make event.kind a list HOT 3
- Incorrect generated/beats/fields.ecs.yml, not accounting on top_level: false HOT 1
- [ECS] Addition - http.request.header.bytes & http.response.header.bytes HOT 2
- Add lowercase normaliser to ECS fields which support security incident response process
- Mark experimental fields as `beta` in generated files
- Elastic-Agent Integrations Use of Legacy Mapping Types Impacts .fleet_globals & prevents agents from being upgraded
- Add `related.url` field
- Add event.zone and event.environment fields
- Addition of additional allowed values for event.type
- Support cloud events in schema HOT 1
- Better abstraction of the type event.kind: alert
- ECS can no longer map all components out of the box HOT 13
- Define a standard way to identify prevention and detection security alerts HOT 5
- Support multi-key fields from SemConv HOT 5
- Allow risk object to be nested under network.
- Add a multi-field user.id.text to the user.id field.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ecs.