Comments (4)
My first thought regarding HPA.
The biggest issue with scaling down nodes is that we would need to use the Solr Collections APIs to remove and replicas from the Solr nodes that are being removed. Not sure how we would integrate with the stateful set scaling.
It would be awesome to get the operator to work well with autoscaling though, seems like one of the biggest feature requests from almost everyone I've talked to.
from solr-operator.
That makes sense. Perhaps we do a built in scaling solution within the solr-operator then? I don't think we have to feel forced to use HPA. Other projects like CoreDNS use their own built in kube-dns autoscaler and don't rely upon HPA for scaling.
from solr-operator.
Integrating Operator with AutoScaling framework I think is the way forward. We probably don't want some outside process to add and remove PODs at will? There are three levels of scaling at play here (at least):
Level 1: Nodes (One per host/VM)
Level 2: Pods (One per Solr node)
Level 3: Shards/Replicas/Cores (several per Pod)
Now, AutoScaling assumes a stable number of Nodes and Pods, and will create/remove/move cores around in the Solr cluster depending on CPU, disk, load. In the future, try to balance things out.
So what if AutoScaling could provide an API where it publishes its "external" wishes. I.e. if AutoScaling sees too full disks or too high latency, and cannot compute a plan inside the current Solr cluster to fix it, it could publish something like this:
{ "tooFewNodes": true, "tooManyNodes": false, "avgCpuPct": 80, "memoryPressurePct": 75, "diskFillRatePct": 40 }
With such feedback, the Operator could make decisions on adding Pods (or asking external system to add more VMs), or to reconfigure Pod size depending on memory, disk or CPU pressure in the cluster. No need to start more Pods if all you need is some more disk.
from solr-operator.
Thanks for the overview! I think we can close this and focus on the points discussed. The K8s HPA is probably out of scope for this problem
from solr-operator.
Related Issues (20)
- Support replicaPlacementFactory in solr.xml HOT 2
- Liveness probe failing for Prometheus Exporter connected to a large SolrCloud
- Disabling PodDisruptionBudgets for both zk pods and solr pods HOT 3
- adding automountServiceAccountToken HOT 1
- Replica allocation after Node is DisabledScheduling HOT 1
- zkHost and zkServer generated incorrectly - helm templates HOT 2
- Solr 8.11 with SolrMetrics produces duplicate samples with prometheus v2.52 HOT 12
- Scale down operation fails and is never requeued if `getReplicasForPod` fails transiently HOT 2
- Configure Resources for zookeeper operator HOT 1
- Allow resizing (expanding) of persistent data PVCs
- Upgrade from Kubebuilder 3 to 4
- SolrOperator leads to 404 HOT 1
- Facing trouble while restoring solr in 8.11.3 HOT 1
- Job Solr-operator-zookeeper-operator-pre-delete without limits
- Unable to pass shareProcessNamespace to PodOptions
- It is impossible to setup TLS between Solr and Zookeeper HOT 1
- Cannot specify an imagePullSecret for solr-operator Helm chart
- Configuration of Solr MultiAuthplugin with JWT and basic auth gives the error of PKI authentication on creating cores. HOT 9
- [Regression] security.json is not uploaded during the first initialization of SolrCloud HOT 1
- How to keep the configsets directiory in solr pods with deployed with helm chart
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from solr-operator.