Comments (6)
I like the idea of adding thread
as well. Good catch.
Out of curiosity, where do you see process and thread IDs in HEX? If you need conversion to DEC, your pipeline get that done for you. But the question is, do you still need the HEX afterwards, or is it simply an idiosyncrasy of this event source?
My preference would be to store only one of them, not both.
On the usage side, if you still need to view the HEX value after ingestion, could you live with a scripted field instead? (e.g. no aggregations on HEX, just on DEC).
from ecs.
For example (not only) MS SharePoint logs process id and thread id in hex. Our users used to use this value. But for correlation with OS processes I need it in DEC. I think I should keep both values. Scripted field is not good as I need to find some value between tens of gigabytes of events.
I did some research and found that linux ps command can show both of these values - pid for process id in DEC and xpid for process id in HEX. So here can be the same naming convention.
So finally following fields seems to be appropriate:
process.pid (or process.id)
process.xpid or (process.xid, but xpid looks better, so for convention pid, xpid, ppid and xppid is probably better)
process.ppid (or process.parent.id)
process.xppid (or process.parent.xid)
thread.tid
thread.xtid
thread.name
I know linux or IBM's ps use sid for "session id" (and xsid) but here thread is more general as usually I need to know info about it from the application side where I know its tid and name. So thats why to use separate thread field.
from ecs.
+1 on adding a thread id. Let's focus here on which fields we should add and in a separate issue about renaming potential existing fields.
Should it be process.thread.tid
, meaning thread should be inside the process object?
from ecs.
Yes, could be inside process. It is highly related/dependent/part of process, currently can not imagine use case for using thread outside of process context.
from ecs.
Ok I was not aware of this use case. I'm in favor of adding the hex representation fields, then.
We're trying to simplify names (process.id, host.name), but in some cases some names are very deeply ingrained all over the culture (pid, hostname). I think PID is one of those cases. I was not really aware of xpid, but I think I would stick with the consistency of the whole bunch of the field names:
process.pid
process.xpid
process.ppid
process.xppid
process.thread.tid
process.thread.xtid
However ECS is just a schema, so it will be the responsibility of the user's pipelines to either fill in the DEC value or the HEX value as needed, at the appropriate time in their pipeline.
from ecs.
We've added process.thread.id
recently.
We have however decided not to add fields for hex IDs. Users of ECS are free to add additional custom fields to their index, and you should feel free to do so. The chances of conflicts are really low, since we don't plan to add these fields.
I'm aware that we're not yet addressing the thread name mentioned above (nor thread priority, which would also make sense). But the creation of object process.thread
leaves space for those eventually. This will likely happen post 1.0, however.
I will close this issue for now.
from ecs.
Related Issues (20)
- Clarify the type of container disk and network metrics HOT 1
- ECS Vulnerability Published field
- Add threat.indicator.tags field
- [Proposal] Make event.kind a list HOT 3
- Incorrect generated/beats/fields.ecs.yml, not accounting on top_level: false HOT 1
- [ECS] Addition - http.request.header.bytes & http.response.header.bytes HOT 2
- Add lowercase normaliser to ECS fields which support security incident response process
- Mark experimental fields as `beta` in generated files
- Elastic-Agent Integrations Use of Legacy Mapping Types Impacts .fleet_globals & prevents agents from being upgraded
- Add `related.url` field
- Add event.zone and event.environment fields
- Addition of additional allowed values for event.type
- Support cloud events in schema HOT 1
- Better abstraction of the type event.kind: alert
- ECS can no longer map all components out of the box HOT 13
- Define a standard way to identify prevention and detection security alerts HOT 5
- Support multi-key fields from SemConv HOT 5
- Allow risk object to be nested under network.
- Add a multi-field user.id.text to the user.id field.
- [Discuss] Add `agent.group` and `host.group` field
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ecs.