Comments (15)
Packetbeat has fairly extensive TLS metadata.
from ecs.
I'm not sure if any of these make sense to include in the schema but they might be useful.
x509.expired
- A simple boolean that indicates if the certificate was expired at the time of use / time of observation.x509.validity_period
- Number of seconds between the not_before and not_after dates. Sometimes it can be useful for filtering on short duration certs like Let's Encrypt or free ones issued by other CAs.
from ecs.
Not sure why, but I also feel uneasy about calculating a boolean based on an expiration date.
On the other hand I think it would be really useful to have. It's a very important thing that's straightforward to alert on, or display in red & so on. Everyone should catch those earlier by working off of the expiration date, but somehow there's always that one cert somewhere, that slips through the cracks :-)
So I'm in favour 👍
from ecs.
The fields from Bro's SSL log might be a good starting point:
https://www.bro.org/sphinx/scripts/base/protocols/ssl/main.bro.html#id2
from ecs.
Thanks for the link @bndabbs . Thinking about it further, I think what we possibly really need is an x509
type to describe the certificate.
These things are commonly conflated, but I think it makes sense to keep them separate. Ideally tls.certificates
would be a list of x509 fields, instead of just the vague keyword blob it currently is.
So, the schema would be more like:
tls: {
// connection stuff
certificates: {
// x509 fields
raw: "---- BEGIN CERTIFICATE----", // PEM encoded x509 cert.
}
}
from ecs.
I agree with them being different, but related.
There is some overlap here, but Bro break the session information down in the SSL log and then puts the actual x509 file information into the x509 log. If we want to eventually have the majority of the Bro fields mapped to ECS, we will need to account for both of these.
Here is an example of the information written to each log for a single connection:
"ssl": {
"established": true,
"cipher": "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
"server_name": "sa.scorecardresearch.com",
"client_cert_chain_fuids": [],
"curve": "secp256r1",
"subject": "CN=*.scorecardresearch.com,OU=COMODO SSL Wildcard,OU=Domain Control Validated",
"cert_chain_fuids": [
"FL0gSF4BtUVC67r6xb",
"FZIS8w319zmNuIacH",
"FMYaSdTqvds74iNC5",
"FDCjVu160RpzNmAaX2"
],
"next_protocol": "http/1.1",
"validation_status": "ok",
"resumed": false,
"version": "TLSv12",
"issuer": "CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB"
}
"x509": {
"certificate_version": 3,
"certificate_subject": "CN=*.scorecardresearch.com,OU=COMODO SSL Wildcard,OU=Domain Control Validated",
"san_dns": [
"*.scorecardresearch.com",
"scorecardresearch.com"
],
"certificate_sig_alg": "sha256WithRSAEncryption",
"certificate_key_type": "rsa",
"certificate_key_length": 2048,
"certificate_not_valid_before": "2017-11-27T00:00:00.000000Z",
"certificate_key_alg": "rsaEncryption",
"certificate_serial": "816E2422058E2390DF7CE5C6FA282C26",
"certificate_exponent": "65537",
"basic_constraints_ca": false,
"fuid": "FL0gSF4BtUVC67r6xb",
"certificate_not_valid_after": "2018-12-20T23:59:59.000000Z",
"certificate_issuer": "CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB"
}
from ecs.
I do think x509
should sit at the top level instead of being nested under tls
, though. There may be some cases where x509 keys are being used for something other than TLS encryption.
from ecs.
tls.certificates
I should have defined it with PEM format in the original proposal, if you look at the keyword it can be an array, what you are proposing is to transform that into an object?
Moving it to a single object would mean the raw field will have a single certificate chain? I think we should probably keep it an array but move it to an object for more flexibility.
from ecs.
I agree with @bndabbs that we should hoist x509
up to the top level.
I propose we add new root level key to ECS under x509
as follows.
{
"x509": {
"properties": {
"raw": "-----BEGIN CERTIFICATE-----...", // Raw PEM encoded cert
"role": {
"type": "keyword" // Typically either 'server' or 'client'. Potentially "peer" as well.
}
"not_before": {
"type": "date"
},
"not_after": {
"type": "date"
},
"public_key_size": {
"type": "integer"
},
"alternative_names": {
"type": "keyword" // Should we use path-hierarchy to tokenize these?
},
"subject": {
"province": {
"type": "keyword"
},
"common_name": {
"type": "keyword" // Should we use path-hierarchy to tokenize these?
},
"country": {
"type": "keyword"
},
"organization": {
"type": "keyword"
},
"locality": {
"type": "keyword"
}
},
"signature_algorithm": {
"type": "keyword"
} ,
"public_key_algorithm": {
"type": "keyword"
},
"version": {
"type": "short"
},
"serial_number": {
"type": "keyword" // hex encoded representation, up to 20 octets
},
"issuer": {
"country": {
"type": "keyword"
},
"organization": {
"type": "keyword"
},
"common_name": {
"type": "keyword"
},
},
"extensions": {
"properties": {
"name": {
"type": "string"
},
"value": {
"type": "keyword"
}
}
}
}
}
}
from ecs.
I think the period is great.
I'm a little hesitant about expired
. That feels weird to compute a value whose result will be different in the future. The validity period cannot change, but expired
can. Do we do that elsewhere with dates?
from ecs.
OTOH, it would be good for users looking back to see if they have a history of expired certs in use, so maybe that's not a great point to make. What do others think?
from ecs.
One other thought, I think we need to make each x509
cert a nested document. It would commonly be the case that one would search for both NotValidBefore
and NotValidAfter
and have both client/server certs OR have a certificate chain.
Are there any objections to making these documents nested.
from ecs.
I don't think we should start to add any nested datatypes to ECS. It makes querying it trickier and can be challenging on the Kibana side. The way we handled this so far in Beats is that each "nested object" is it's own document (which in Lucene it is anyways) but we have an id for example to correlate them.
To move this forward I would suggest to open a PR against the use-cases with this to have it in there quickly to show to users and discuss the fields in more details. This will make it then pretty easy to migrate it to ECS later as it's just copying it over.
from ecs.
I ran into a little bit of x.509 when working on digital code signatures #681 (PR #733).
Some of the fields used for the most common use cases are
- subject name (often called signature_signer)
- valid (bool): checksum match, etc
- trusted (bool): does the chain check out and trace back to a trusted root?
- status (string): logging for all of the other properties we couldn't capture
There's a good chance that I'm significantly oversimplifying things, but I also don't want to make my current use case more complicated than necessary
from ecs.
Closing due to #762 being merged.
from ecs.
Related Issues (20)
- ECS Vulnerability Published field
- Add threat.indicator.tags field
- [Proposal] Make event.kind a list HOT 3
- Incorrect generated/beats/fields.ecs.yml, not accounting on top_level: false HOT 1
- [ECS] Addition - http.request.header.bytes & http.response.header.bytes HOT 2
- Add lowercase normaliser to ECS fields which support security incident response process
- Mark experimental fields as `beta` in generated files
- Elastic-Agent Integrations Use of Legacy Mapping Types Impacts .fleet_globals & prevents agents from being upgraded
- Add `related.url` field
- Add event.zone and event.environment fields
- Addition of additional allowed values for event.type
- Support cloud events in schema HOT 1
- Better abstraction of the type event.kind: alert
- ECS can no longer map all components out of the box HOT 13
- Define a standard way to identify prevention and detection security alerts HOT 5
- Support multi-key fields from SemConv HOT 5
- Allow risk object to be nested under network.
- Add a multi-field user.id.text to the user.id field.
- [Discuss] Add `agent.group` and `host.group` field
- ECS Tooling - Mapping & Template Settings
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ecs.