Comments (20)
I don't think that's feasible. I don't see a lot of of users doing selective filtering without configuring the -service
flag.
from fastly-exporter.
This is meant as an additional improvement on #31. Possibly a replacement, depending on where the bottlenecks are.
CC @neufeldtech
from fastly-exporter.
Super cool.
from fastly-exporter.
So wait, given /discovery
returns
[
{ "targets": ["A", "B", "C"] }
]
there's no absolute or explicit URL mapping to each of A, B, and C — I would just assert in the exporter (or a flag or something) that they map to e.g. /service/{A, B, C}
which the scraping Prometheus would have to be configured to match a priori. Is that right?
from fastly-exporter.
Yes, it needs to be re-mapped by a relabel config in Prometheus no matter what. The discovery injects the target list items into the __address__
meta-label in discovery. Prometheus doesn't treat those as URLs, it treats them as host:port
pairs.
from fastly-exporter.
@SuperQ The bottleneck is parsing the JSON from the Fastly API. By default, the exporter fetches all services available to the provided token on startup, and launches a goroutine per service to poll the API and update the relevant metrics. The -service*
flags are used to control the set of services we poll, so I think that means we need to keep them. Though maybe I could play some trick — could I lazy-launch the polling goroutine for a given service only once the exporter received a request for that service? It would mean the first scrape either had a 1s+ delay, or returned an empty response; would either of those be acceptable?
from fastly-exporter.
This is why readiness checks are a thing. Discovery can just wait for the readiness endpoint to say OK.
from fastly-exporter.
Discovery can just wait for the readiness endpoint to say OK.
Not sure how that would work. Is readiness per-SD-target? I'm suggesting that the exporter not poll a service ID unless/until it receives a scrape request for that service ID from Prometheus.
from fastly-exporter.
The exporter shouldn't expose a newly added service to the discovery output until it has valid API connection.
from fastly-exporter.
Right, but the only meaningful optimization is to try to avoid polling the API and parsing the large JSON responses if we can somehow. For example, if the user starts the exporter with a configuration that makes 10 services "visible" but only has Prometheus scrape 2 of them, it would be ideal if we didn't poll/parse/present metrics for the other 8. The only way I can think of to accomplish that is to "lazy load" the per-service polling goroutines. Is that feasible? Sounds like no. That's fine.
from fastly-exporter.
Is there a canonical example of an exporter that supports this new capability?
from fastly-exporter.
Not that I'm aware of, this would be the first one.
from fastly-exporter.
So this is an interesting design conundrum. I think the right way to think about this is that we're adding /metrics endpoints that are backed by a kind of pseudo-registry: something that takes the main registry and filters based on a label. I guess you could do that by wrapping the Registry with something that takes a service ID and yields a custom Gatherer? Or am I looking at this the wrong way?
from fastly-exporter.
Ah! Huh.
So the other angle would be something like maintaining a separate gen.Metrics/prometheus.Registry for each service. That makes per-service /metrics endpoints easy. Could you then just use a prometheus.Gatherers to abstract over all of them to serve the existing /metrics endpoint? Would that be efficient/effective?
from fastly-exporter.
Yes, setting up a separate registry per-service makes the most sense to me.
Although, typically, an exporter like this would manage the data internally. You implement your own prometheus.Collect(ch)
to send data to the Prometheus via the interface. We do something like this in the node_exporter.
from fastly-exporter.
→ #73
from fastly-exporter.
Ah, @SuperQ, what would the Prometheus config look like for this? Especially whatever relabel config would be necessary.
from fastly-exporter.
The scrape config and relabel is in the issue summary.
from fastly-exporter.
I tested this out locally, looks really good. It even works well with the modulo sharding so you get the best of both worlds, horizontal sharding and automatic discovery.
from fastly-exporter.
the issue summary
Thank you 🤦
from fastly-exporter.
Related Issues (20)
- Cannot pull new Docker image HOT 3
- Support HTTP3 metrics
- Better logging for rt.fastly.com (Client.Timeout exceeded while awaiting headers) HOT 2
- remove tls_version="any" from fastly_rt_tls_total HOT 3
- metric consolidation for http2/http3
- Race condition between processing and scraping
- carnality explosion with fastly_rt_datacenter_info HOT 3
- Consider adding a way to (optionally) track tokens HOT 2
- Reduce output size of metrics endpoint HOT 3
- Expose fastly rt api 'recorded' timestamp in prometheus metrics HOT 3
- Add all those wild new Compute@Edge metrics
- Missing Docker images HOT 9
- RT API meta-metric HOT 3
- Metric Timestamps HOT 15
- Add datcenters API enricment HOT 2
- Add partial docker tags HOT 4
- iterating http_sd support towards generic discovery HOT 21
- Continue starting up if fastly is down HOT 1
- Exporter should set it's own custom user-agent HOT 5
- Pagination is not working properly (not fetching every pages) HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fastly-exporter.