opencost / opencost-helm-chart

OpenCost Helm chart

OpenCost Helm Chart

Maintainers

Name
mattray
toscott
brito-rafa

Usage

Helm must be installed to use the charts. Please refer to Helm's documentation to get started.

Once Helm is set up properly, add the repo as follows:

helm repo add opencost https://opencost.github.io/opencost-helm-chart

See the Chart Documentation for chart install instructions.

Testing

Testing your chart (optional)

Presumes you've got Helm unittest installed: (i.e. helm plugin install unittest) and that your in the root directory of your cloned repo:

helm unittest charts/opencost

Should produce a result like this:

### Chart [ opencost ] charts/opencost

 PASS  test deployment	charts/opencost/tests/deployment_test.yaml
 PASS  test deployment snapshot	charts/opencost/tests/opencost_test.yaml

Charts:      1 passed, 1 total
Test Suites: 2 passed, 2 total
Tests:       2 passed, 2 total
Snapshot:    1 passed, 1 total
Time:        31.011089ms

---

OpenCost Links

• OpenCost
• Documentation

opencost-helm-chart's Issues

Athena configuration incomplete

Describe the bug
Error in pod logs after following AWS integration instructions for OpenCost and KubeCost for Athena integration.

To Reproduce
Steps to reproduce the behavior:

1. Configure config map below.

apiVersion: v1
kind: ConfigMap
metadata:
  name: opencost-aws
  namespace: opencost
data:
  aws.json: |
    {
        "provider": "aws",
        "description": "AWS Provider Configuration. Provides default values used if instance type or spot information is not found.",
        "CPU": "0.031611",
        "spotCPU": "0.006655",
        "RAM": "0.004237",
        "GPU": "0.95",
        "spotRAM": "0.000892",
        "storage": "0.000138888889",
        "zoneNetworkEgress": "0.01",
        "regionNetworkEgress": "0.01",
        "internetNetworkEgress": "0.143",
        "spotLabel": "kops.k8s.io/instancegroup",
        "spotLabelValue": "spotinstance-nodes",
        "awsSpotDataRegion":"eu-west-1",
        "awsSpotDataBucket": "iobspotdatafeedsubscription",
        "awsSpotDataPrefix": "spot-datafeed",
        "athenaBucketName": "s3://aws-athena-query-results-REDACTED-eu-west-1",
        "athenaRegion": "eu-west-1",
        "athenaDatabase": "athenacurcfn_opencost_report",
        "athenaTable": "opencost_report",
        "projectID": "REDACTED"
    }

2. Install opencost on AKS using helm chart

Extract of the values.yaml that configures the configmap.

opencost:
  exporter:
    extraEnv:
      CONFIG_PATH: "/tmp/custom-config"
    extraVolumeMounts:
      - mountPath: /tmp/custom-config
        name: custom-configs

extraVolumes:
  - name: custom-configs
    configMap:
      name: opencost-aws

3. View pod logs and see the following errors.

2023-10-13T09:59:20.927916579Z ERR Failed to lookup reserved instance data: no reservation data available in Athena
2023-10-13T09:59:20.927937945Z ERR Failed to lookup savings plan data: Error fetching Savings Plan Data: QueryAthenaPaginated: athena configuration incomplete

Expected behavior
Athena integration should work according to the Kubecost documents

Which version of OpenCost are you using?
Helm chart version 1.20.1
Opencost App Version 1.106.2

suggestion: adding init container for azure billing enrollment

based on this link: https://www.opencost.io/docs/azure-prices#grant-billing-access-to-your-service-principal
a bash script is provided.

Would it be good to run the bash script in an init container?

Issue with mounting ConfigMap to supply spot instance configs

ERR Error downloading default pricing data: Failed to write file: open /custom-config/aws.json: read-only file system

Im mounting the below Configmap as a volume to Opencost so that it will go here.
I tried setting readOnly: false as written here but it didn’t solve the problem.

opencost:
  exporter:
    extraEnv:
      # The default config path is read only, for customizing we have to swap spots.
      CONFIG_PATH: "/custom-config"
    extraVolumeMounts:
      - mountPath: /custom-config
        name: custom-configs
        readOnly: false
extraVolumes:
  - name: custom-configs
    configMap:
      name: opencost-conf

Missing opencost.cloudIntegrationSecret value in the opencost chart README.md and values.yaml

I will looking into how to add the VolumeMount for cloud-integration and found that there is a opencost.cloudIntegrationSecret value not documented in the README.md
or the values.yaml

Only found this value by looking in templates/deployment.yaml

Publish the helm chart to a registry

It would be much easier to install and track if the chart is published to a registry with a tag

ui authentication

it would be great to be able to have authentication of some kind on the webui so that you can safely expose it to the internet. i was thinking about an oauth2-proxy sidecar.

The secret is not created even if I put in the aws settings.

You can create a secret in the "aws" setting by modifying as follows
{{- if or .Values.opencost.prometheus.username .Values.opencost.prometheus.password .Values.opencost.prometheus.bearer_token .Values.opencost.exporter.aws.access_key_id -}}

Expose Prometheus values so we can override the names

https://github.com/opencost/opencost-helm-chart/blob/main/values.yaml#L24

Make these configurable:

    serviceName: my-prometheus-server
    namespaceName: prometheus

Add network policies support

Network policies are required to run OpenCost in a default deny environment. This is quite common in a production environment.
Ref: https://kubernetes.io/docs/concepts/services-networking/network-policies/

If you would like to implement this feature, I would offer you my contribution.
Let me know!

opencost deploys to default namespace

Despite creating the opencost namespace, the helm chart deploys to the default namespace.

Change merge strategy to "squash merging"

Hi everybody

I already asked for this in the slack channel #opencost.

As I am one of the maintainer of the Argo CD helm repository, we follow what most chart repositories do:

Can you change this as well here?

/cc: @sntxrr @Pokom

Expose PROMETHEUS_SERVER_ENDPOINT as a configurable value

There are scenarios where an operator may want to use a prometheus instance that does not run alongside opencost. We should provide the ability to define the PROMETHEUS_SERVER_ENDPOINT when deploying the helm chart.

Add podLabels to opencost's pods

As title suggests. Same idea as podAnnotations.

CSV export on S3 should not replace the whole file each time

Currently the CSV export functionality on S3 replaces the whole file each time. S3, like other object storage services, does not support file appends, only replacement. This means that the CSV export is being exported as a whole each time. It would be better if the exports are split per reference day, with one new CSV file created each day with the corresponding timestamp/date in the filename. This way the CSV export on S3 would scale much better.

Configure duration of data to store

OpenCost currently defaults to 15 days of data retention, we should probably make this configurable and expose it at the Helm chart level.

Add CI check to ensure PR's bump versions

Currently it is possible to merge changes into main that do not bump the appVersion in the Charts.yaml file. There should be some safeguards in place to prevent this to ensure main can always be published.

Image tag is always set to latest (if unset)

Because the default value for tag: is set to the latest , the template will use that if tag is unset, instead of the appversion in Chart.yaml. I discovered this when upgrading from 1.22.2 to 1.26.2. Is this intentional ?

Incorporate the Prometheus OpenCost exporter here

It makes sense to make this the home of the code.

prometheus-community/helm-charts#3872

No data for "App cost by pod" panel in grafana example dashboard

I imported the dashboard @ https://github.com/opencost/opencost-helm-chart/blob/main/examples/dashboard/kube-prometheus-stack-opencost-dashboard.json to Grafana (kube-prometheus stack)

Seems metrics related to pods or containers are not showing

I tried editing some existing panel to use kube_node_labels{job="opencost"} instead of just kube_node_labels as commented in #9 (comment) but does not seem to fix anything

• AKS: v1.25
• quay.io/kubecost1/kubecost-cost-model:prod-1.105.1
• kube-prometheus v0.12.0

opencost ui must run as root user ?

Hello,

I am trying to install opencost-ui as non-root user and i get this error

sed: can't create temp file '/var/www/index.60cea723.jsXXXXXX': Permission denied

is there anyway to install the ui without the root user access?

Missing opencost.ui.enabled value in README of opencost chart

opencost.ui.enabled exists in values.yaml but not in README.md

It seems to be getting deleted automatically for some reason.
https://github.com/opencost/opencost-helm-chart/pull/156/files

[Feature request] namespaceOverride possibility

Hi, we have found some gaps in Opencost chart, unfortunately Opencost chart doesn't support any possibility for changing the target namespace e.g. https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/values.yaml#L11. It would be nice to have such option, thanks in advance.

Conflict with multiple declarations of volumeMounts in deployment.yaml

Description

There are multiple declarations of the volumeMounts field in the deployment.yaml template.

Currently, the volumeMounts field can be specified in three ways in the values.yaml file:

1. By providing extraVolumeMounts values in .Values.opencost.exporter.extraVolumeMounts
2. By enabling the persistence in .Values.opencost.exporter.persistence.enabled
3. By providing extraVolumeMounts values in .Values.opencost.ui.extraVolumeMounts

So, this is preventing mounting of secrets for the service principal to the Azure API.

Expected behavior

Be able to enable all volumeMounts.

How can i disable not required metrics?

Hello, i see that opencost support metrics disabling https://github.com/opencost/opencost/blob/c324846753beda858a1ea18a00fc50b5ca509309/pkg/metrics/metricsconfig.go#L24

But i don't see how this can be done using chart. Is there a way to do it?

Incorrect behaviour after define prometheus bearer token values

When using the following values to define bearer token secret name and key, to connect to the prometheus, the helm template rendered to incorrect output:

• .Values.opencost.prometheus.bearer_token_key
• .Values.opencost.prometheus.secret_name

Reproduce:

Changes to values:

     prometheus:
+      secret_name: spcld-opencost-prometheus-token
+      bearer_token_key: token

Changes in argoCD:

Add Ingress for UI

Just a basic feature.

Instance/Node label duplicated for Node IP and name

Description

I've installed the OpenCost helm chart 1.18.0 with the following values

opencost:
    nodeSelector:
      kubernetes.io/os: linux
    metrics:
      serviceMonitor:
        enabled: true
    exporter: 
      extraEnv: 
        EMIT_KSM_V1_METRICS_ONLY: true  
        EMIT_KSM_V1_METRICS: false
        # The default config path is read only, for customizing we have to swap spots.
        CONFIG_PATH: "/tmp/custom-config"

      extraVolumeMounts:
        - mountPath: /tmp/custom-config
          name: custom-configs
    prometheus: 
      internal:
        enabled: true
        serviceName: svcname
        namespaceName: monitoring
        port: 9090
    ui: 
      enabled: true
      ingress: 
        enabled: true
        ingressClassName: "nginx"
        # -- Annotations for Ingress resource
        annotations: {}
          # kubernetes.io/tls-acme: "true"
        # -- A list of host rules used to configure the Ingress
        # @default -- See [values.yaml](values.yaml)
        hosts:
          - host: _____
            paths:
              - /
  extraVolumes:
  - name: custom-configs
    configMap:
      name: opencost-conf

The extra Configmap is used to disable the v1 kube state metrics.

apiVersion: v1
kind: ConfigMap
metadata:
  name: opencost-conf
data:
    default.json: |
        {
            "provider": "custom",
            "description": "Default prices based on GCP us-central1",
            "CPU": "0.031611",
            "spotCPU": "0.006655",
            "RAM": "0.004237",
            "spotRAM": "0.000892",
            "GPU": "0.95",
            "storage": "0.00005479452",
            "zoneNetworkEgress": "0.01",
            "regionNetworkEgress": "0.01",
            "internetNetworkEgress": "0.12"
        }
    metrics.json: | 
        {
            "disabledMetrics":[
                "kube_pod_owner",
                "kube_pod_labels",
                "kube_namespace_labels",
                "kube_node_labels",
                "kube_node_status_condition"
                ]
        }
        

For metrics, I'm using kube-prometheus-stack (with node-exporter and kube-state-metrics).

Problem

The cost model is exporting metrics for the same node with different instance/node labels (one for the node IP and another with the node name). This is a problem because the node_**_cost metrics have the node label value as the node name and the cost model ends up discarding the metrics with the IP.
This image shows Node Exporter containers, but this happens for almost every pod/container.

OpenCost UI

Prometheus

I'm I doing anything wrong?
Can anyone help me understand which metrics the cost model looks at?

kubecost-cost-model container not starting up in Azure when deployed with OpenCost Helm chart.

Hi,

It appears kubecost-cost-model container is not starting up when deployed in Azure AKS with Opencost Helm chart. There are no errors in logs, even changing LOG_LEVEL to debug didn't have any effect (probably because container doesn't even start). We suspect there is something with the OpenCost Helm chart incorrectly configuring deployment of kubecost-cost-model. Tried many things but no difference, same log Info and container not started:

1. Helm chart 1.14.0, regular quay.io/kubecost1/kubecost-cost-model:prod-1.103.1 and with gcr.io/kubecost1/opencost:kc-eu-2023
2. Helm chart 1.14.3, regular quay.io/kubecost1/kubecost-cost-model:prod-1.103.1 and with gcr.io/kubecost1/opencost:kc-eu-2023
3. With Service Principal secret and without.

Details:
Azure AKS v1.25.6
Deployed with ArgoCD v2.6.7+5bcd846

project: ska
source:
  repoURL: 'https://opencost.github.io/opencost-helm-chart'
  targetRevision: 1.14.3
  helm:
    values: |
      extraVolumes:
      - name: service-key-secret
        secret:
          secretName: azure-service-key
      opencost:
        exporter:
          extraEnv:
            LOG_LEVEL: debug
          extraVolumeMounts:
          - mountPath: /var/secrets
            name: service-key-secret
        prometheus:
          internal:
            enabled: true
            namespaceName: monitoring
            port: 9090
            serviceName: prometheus-kube-prometheus-prometheus
        ui:
          ingress:
            annotations:
              cert-manager.io/cluster-issuer: letsencrypt-prod
              ingress.kubernetes.io/ssl-redirect: "true"
              kubernetes.io/tls-acme: "true"
              nginx.ingress.kubernetes.io/modsecurity-snippet: |
                SecRuleEngine On
            ingressClassName: nginx
    version: v3
  chart: opencost
destination:
  server: 'https://kubernetes.default.svc'
  namespace: kubecost
syncPolicy:
  automated:
    prune: true
    selfHeal: true
  syncOptions:
    - CreateNamespace=true
    - ApplyOutOfSyncOnly=true
  retry:
    limit: 5

Logs, only this is in logs no matter what is changed:

27
2023-05-22T15:06:16.509888074Z ??? Log level set to debug
26
2023-05-22T15:06:16.510069775Z INF Starting cost-model version 1.103.1 (6dd98b305de3b1d1c826a041cf36a2db4d267ee1)
25
2023-05-22T15:06:16.510244777Z INF Prometheus/Thanos Client Max Concurrency set to 5
24
2023-05-22T15:06:16.520338258Z INF Success: retrieved the 'up' query against prometheus at: http://prometheus-kube-prometheus-prometheus.monitoring.svc:9090
23
2023-05-22T15:06:16.535087077Z INF Retrieved a prometheus config file from: http://prometheus-kube-prometheus-prometheus.monitoring.svc:9090
22
2023-05-22T15:06:16.560877685Z INF Using scrape interval of 60.000000
21
2023-05-22T15:06:16.561828193Z INF NAMESPACE: kubecost
20
2023-05-22T15:06:16.764493528Z INF Done waiting
19
2023-05-22T15:06:16.76481283Z INF Starting *v1.Deployment controller
18
2023-05-22T15:06:16.764965632Z INF Starting *v1.Namespace controller
17
2023-05-22T15:06:16.765258734Z INF Starting *v1.Node controller
16
2023-05-22T15:06:16.765370835Z INF Starting *v1.Pod controller
15
2023-05-22T15:06:16.765829738Z INF Starting *v1.StatefulSet controller
14
2023-05-22T15:06:16.765492036Z INF Starting *v1.Service controller
13
2023-05-22T15:06:16.765945539Z INF Starting *v1.ConfigMap controller
12
2023-05-22T15:06:16.76597254Z INF Starting *v1.DaemonSet controller
11
2023-05-22T15:06:16.76599914Z INF Starting *v1.Job controller
10
2023-05-22T15:06:16.76604564Z INF Starting *v1.ReplicaSet controller
9
2023-05-22T15:06:16.76606434Z INF Starting *v1.PersistentVolume controller
8
2023-05-22T15:06:16.766082241Z INF Starting *v1.PersistentVolumeClaim controller
7
2023-05-22T15:06:16.766104141Z INF Starting *v1.StorageClass controller
6
2023-05-22T15:06:16.766124241Z INF Starting *v1.ReplicationController controller
5
2023-05-22T15:06:16.766142541Z INF Starting *v1beta1.PodDisruptionBudget controller
4
2023-05-22T15:06:16.767947656Z INF Found ProviderID starting with "azure", using Azure Provider
3
2023-05-22T15:06:16.771786287Z INF No pricing-configs configmap found at install time, using existing configs: configmaps "pricing-configs" not found
2
2023-05-22T15:06:16.775200814Z INF No metrics-config configmap found at install time, using existing configs: configmaps "metrics-config" not found
1
2023-05-22T15:06:16.777853635Z INF Using ratecard query OfferDurableId eq 'MS-AZR-0003p' and Currency eq 'USD' and Locale eq 'en-US' and RegionInfo eq 'US'

kubectl describe pod -n kubecost

Name:             opencost-ff69d5985-9l9w4
Namespace:        kubecost
Priority:         0
Service Account:  opencost
Node:             aks-agentpool-40542563-vmss000003/10.224.0.4
Start Time:       Sun, 21 May 2023 18:00:35 -0400
Labels:           app.kubernetes.io/instance=opencost
                  app.kubernetes.io/name=opencost
                  pod-template-hash=ff69d5985
Annotations:      <none>
Status:           Running
IP:               10.244.0.12
IPs:
  IP:           10.244.0.12
Controlled By:  ReplicaSet/opencost-ff69d5985
Containers:
  opencost:
    Container ID:   containerd://8c9a36212c3056cbd40986bc66312109d01a89ffe1d8d5baadcec985f9759c6c
    Image:          quay.io/kubecost1/kubecost-cost-model:prod-1.103.1
    Image ID:       quay.io/kubecost1/kubecost-cost-model@sha256:2993835c1090048431f667d8e4c8bb59e561bd90c6e431f548a692a8a6a6dbe1
    Port:           9003/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Mon, 22 May 2023 11:13:29 -0400
    Last State:     Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Mon, 22 May 2023 11:06:16 -0400
      Finished:     Mon, 22 May 2023 11:08:16 -0400
    Ready:          False
    Restart Count:  231
    Limits:
      cpu:     999m
      memory:  1Gi
    Requests:
      cpu:      10m
      memory:   55Mi
    Liveness:   http-get http://:9003/healthz delay=30s timeout=1s period=10s #success=1 #failure=10
    Readiness:  http-get http://:9003/healthz delay=30s timeout=1s period=10s #success=1 #failure=200
    Environment:
      PROMETHEUS_SERVER_ENDPOINT:  http://prometheus-kube-prometheus-prometheus.monitoring.svc:9090
      CLUSTER_ID:                  default-cluster
      LOG_LEVEL:                   debug
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9lxpz (ro)
      /var/secrets from service-key-secret (rw)
  opencost-ui:
    Container ID:   containerd://a896d239f89cff03b9286e86cbf1178bcd8cf5bb19914bf1e7f4bd1a87646dd7
    Image:          quay.io/kubecost1/opencost-ui:prod-1.103.1
    Image ID:       quay.io/kubecost1/opencost-ui@sha256:526415266a674bc108d4a71284fbe04342ccbbfb3089313cb1e1421a4b218128
    Port:           9090/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sun, 21 May 2023 18:00:37 -0400
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     999m
      memory:  1Gi
    Requests:
      cpu:        10m
      memory:     55Mi
    Liveness:     http-get http://:9090/healthz delay=30s timeout=1s period=10s #success=1 #failure=10
    Readiness:    http-get http://:9090/healthz delay=30s timeout=1s period=10s #success=1 #failure=10
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9lxpz (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  service-key-secret:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  azure-service-key
    Optional:    false
  kube-api-access-9lxpz:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                     From     Message
  ----     ------     ----                    ----     -------
  Warning  Unhealthy  33m (x2700 over 17h)    kubelet  Readiness probe failed: Get "http://10.244.0.12:9003/healthz": dial tcp 10.244.0.12:9003: connect: connection refused
  Warning  Unhealthy  8m38s (x2297 over 17h)  kubelet  Liveness probe failed: Get "http://10.244.0.12:9003/healthz": dial tcp 10.244.0.12:9003: connect: connection refused
  Warning  BackOff    3m39s (x2689 over 16h)  kubelet  Back-off restarting failed container

Could anyone help with following:

1. Is Azure AKS officially supported in OpenCost Helm chart?
2. How to configure OpenCost for Azure AKS or at least how to troubleshoot it?

Any help would be highly appreciated.

feat: add priorityClassName

support for more than one replica

In some production level environments, having multiple replicas is a requirement for entry. It makes the service fault tolerant.

Ideally, more than one replica would (minimum of three) run simultaneously, failing over gracefully should one go down.

I'm not aware of any alternatives at this time, though I bet there might be some options.

Add any other context, documentation links, or screenshots about the feature request here.

Offer an external secret option for prometheus

I love the ability to specify an existing prometheus through the values file like this:

opencost:
  prometheus:
    username: "my-username"
    password: "my-password"
    external:
      enabled: true
      url: https://prometheus.example.com/api/prom

But I'd love to have the option to use an existing secret with something like:

opencost:
  prometheus:
    createSecret: false
    secretName: my-existing-secret
    usernameKey: prom_username
    passwordKey: prom_password
    external:
      enabled: true
      url: https://prometheus.example.com/api/prom

This would lead to the deployment looking like this:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
      - name: my-opencost
         env:
         - name: DB_BASIC_AUTH_USERNAME
            valueFrom:
              secretKeyRef:
                name: my-existing-secret
                key: prom_username
         - name: DB_BASIC_AUTH_PW
            valueFrom:
              secretKeyRef:
                name: my-existing-secret
                key: prom_password

The reason for this is because I am planning on deploying OpenCost along with other services that need the credentials for prometheus and I'd rather use a single secret than create multiples. That would make things like credential rotation simpler.

OpenShift support for opencost-ui

Due to some security limitations on openshift, I've faced the following problems for opencost-ui:

• permission denied on /var/www dir for nginx when starting pod. Need to change ownership and filemod
• port 80 is not available to open. It's related to default nginx configurations

These configurations should be editable with helm values

Additional labels are not assigned to a services monitor based on values.yaml config

Hey,
Noticed that additionalLabels has no effect to servicemonitor.yaml template labels

  metrics:
    serviceMonitor:
      enabled: false
      additionalLabels: {}
      ## The label to use to retrieve the job name from.
      ## jobLabel: "app.kubernetes.io/name"
      namespace: ''
      namespaceSelector: {}
      ## Default: scrape .Release.Namespace only
      ## To scrape all, use the following:
      ## namespaceSelector:
      ##   any: true
      scrapeInterval: 30s
      # honorLabels: true
      targetLabels: []
      relabelings: []
      metricRelabelings: []

Curently service monitor template contains the following regarding labels:

...
  labels:
    {{- include "opencost.labels" . | nindent 4 }}

Need to add something like this:

...
  labels:
    {{- include "opencost.labels" . | nindent 4 }}
    {{- if .Values.opencost.metrics.serviceMonitor.additionalLabels }}
    {{- toYaml .Values.opencost.metrics.serviceMonitor.additionalLabels | nindent 4 }}
    {{- end }}

Add podLabels for opencost's pods

Just like podAnnotations.

Support arguments to the exporter container such as `--log-level`

It would be great if you could add support for setting command-line flags when calling OpenCost. The currently available flags can be seen here: https://github.com/opencost/opencost/blob/3186c1eee8de0826e44f0b13d00bf06f86db787e/pkg/cmd/commands.go#L73-L75

This is useful for setting the log level of OpenCost when trying to identify a problem. For example, the current deployment can be editted with the following to set the log level to trace.

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
      - name: opencost
        args:
        - --log-level
        - trace

UI not accessible with path-based ingress

Using a path based ingress is possible, but the OpenCost UI itself does not load as it ignores the path defined within the ingress:

values.yaml

opencost:
  ui:
    ingress:
      enabled: true
      hosts:
        - host: test.intern.ch
          paths:
            - /opencost
      tls:
        - hosts:
            - test.intern.ch

The page is served at test.intern.ch/opencost but it serves an empty page as the index.js and other resources result in 404 as it is trying to load from test.intern.ch/index.js which is not a valid path.

Maybe this could be solved by providing the BASE_URL_OVERRIDE environment variable to the ui container. But I'm not quite sure. See the opencost/ui Github page for reference. The current deployment.yaml does not allow to add additional env-values. This could be solved by adding the following block to the opencost-ui container in the deployment.yaml:

{{- with .Values.opencost.ui.extraEnv }}
env:
{{- toYaml . | nindent 10 }}
{{- end }}

Not able to fix s3 bucket in helm chart

Hi all,
After setting up AWS Spot instance data feed configuration, I am not able to to add the s3 bucket configuration.
Am I missing something?
Skipping AWS spot data download: operation error S3: ListObjects, https response error StatusCode: 400, error InvalidBucketName: The specified bucket is not valid.

Helm Chart Affinity, NodeSelector and TopologySpreadConstraints

To configure how the OpenCost pods are placed on nodes, we need to add the following common configs to the Helm Chart:

• Affinity
• Node Selector
• Topology Spread Constraints

Publish the helm index to make the chart discoverable

What we need to do:

1. Create a new branch in the repository named "gh-pages"
2. Bump the chart version or delete the current release/tag (will likely fail otherwise since the run was incomplete)
3. Re run the publish workflow
4. Verify Settings/Pages > Source branch is set to gh-pages

Why we need it
Creating the new branch will allow the chart-releaser-action github action to create and pubilsh an index.yaml file with the chart information.
View here for more details: https://github.com/helm/chart-releaser-action

Rerunning the publish workflow will create an index.yaml in the gh-pages branch.

Setting the source branch to gh pages will expose that branch as a website: https://opencost.github.io/opencost-helm-chart/
View here for more details: https://pages.github.com/

Running Cloud Costs with custom pricing fails to load Cloud Costs

Related to opencost/opencost#2267

Issue with deploy by helm

was provided by deploy gcp api key and prometheus url but see some another project number

ERR Failed to lookup reserved instance data: googleapi: Error 403: Compute Engine API has not been used in project 42275347607 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project=42275347607 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Details:
[
{
"@type": "type.googleapis.com/google.rpc.Help",
"links": [
{
"description": "Google developers console API activation",
"url": "https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project=42275347607"
}
]
},
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
"domain": "googleapis.com",
"metadatas": {
"consumer": "projects/42275347607",
"service": "compute.googleapis.com"
},
"reason": "SERVICE_DISABLED"
}
]
, accessNotConfigured

Non-working default in ServiceMonitor value

When honorLabels is set to false, OpenCost metrics are being ingested with the namespace label set to the opencost namespace. This causes most of the data to be set to unallocated, as the ingested namespace labels do not match the pods.

Errors associated to setting security context values

Description
There are errors when attempting to enable the security context for the UI (commented out in the values.yaml file). The specific settings causing the errors are as follows:

• securityContext.capabilities.drop: ALL
• securityContext.runAsNonRoot: true
• securityContext.runAsUser: 1000
• securityContext.readOnlyRootFilesystem: true

1. SettingsecurityContext.capabilities.drop: ALL leads to a CrashLoopBackOff error with the message:
   

2. Setting securityContext.runAsNonRoot: true leads to a CreateContainerConfigError with the message:
   

3. Setting securityContext.runAsUser: 1000 leads to a CrashLoopBackOff error with the message:
   

4. Setting securityContext.readOnlyRootFilesystem: true leads to a CrashLoopBackOff error with the message (this also occurs in the exporter):
   

Also, version 1.105 of OpenCost apparently enabled the UI image to run as non-root by setting the user as 1001. However, by default it still runs as root. Also, when we set the chart value of runAsNonRoot to true for opencost-ui, it's still running as root (the same thing occurs when runAsUser is additionally set to 1001 with runAsNonRoot to true):

Reproduction Steps
Install the Helm chart (observed in latest release 1.18.0) with the security context settings mentioned above.
Observe the errors described above occurring in the pod logs.

Expected Behavior
I expect the security context settings to be enabled without causing these errors, as they are intended to enhance container security.

Custom pricing configmap

Description

OpenCost supports custom pricing for on-premise environments here. You can introduce this pricing by deploying a configmap in the environment sadly this isn't integrated into the helm chart and is deployed separately. If you go about it the way it is now it's not explicitly clear in the values as you have to introduce them as extraEnv and ExtraMount

  exporter:
    extraEnv:
      # The default config path is read only, for customizing we have to swap spots.
      CONFIG_PATH: "/tmp/custom-config"
    extraVolumeMounts:
      - mountPath: /tmp/custom-config
        name: custom-configs
extraVolumes:
  - name: custom-configs
    configMap:
      name: opencost-conf

Expected Behavior

As an improvement, we should include the configmap for users to be defined in their values as they see fit with the possibility to use their own configmap if they want to.

Internal prometheus URL doesn't support routePrefix

The kube-prometheus-stack helm chart supports the routePrefix for prometheusSpec.
This means that prometheus endpoint can have a route prefix.
Current code does not support this, but assuming that the internal endpoint always listens on the root path.

opencost-helm-chart/charts/opencost/templates/_helpers.tpl

Line 78 in 6ed2eb4

{{- printf "http://%s.%s.svc:%d" $host $ns $port -}}

Add repo but got index.yaml not found

Step to reproduce:

1. I put these lines in Terraform:

resource "helm_release" "opencost" {
  atomic           = "true"
  name             = "opencost"
  version          = "1.100.0"
  namespace        = "opencost"
  create_namespace = "true"
  repository       = "https://opencost.github.io/opencost-helm-chart"
  chart            = "opencost"

  values = [
    file("helm_values/opencost.values.yaml")
  ]
}

2. Then I do tf-apply

Actual result:
I got error "index.yaml not found"

Expected result:
The index.yaml file exists and accessible so adding repo get success.
Mostly repo have accessible index.yaml, for example:
https://kubecost.github.io/cost-analyzer/index.yaml

Screenshot:

Helm Chart image versioned with opencost released image

We should change the helm chart release to match the Opencost release.
At the moment in the values.yaml there is set "tag: latest".
Unfortunately in this way, the helm chart cannot be versioned by controlling what app version image has been released within

Example of the requested change
chart 1.0.3 has been set to 1.9.8 with the latest image

image:
  registry: quay.io
  repository: kubecost1/kubecost-cost-model
  tag: latest

to example:
chart 1.0.4 has been set to 1.100.0

image:
  registry: quay.io
  repository: kubecost1/kubecost-cost-model
  tag: prod-1.100.2

Container `opencost` keeps exiting with error code 2 when trying to hit RateCard API

Setup

1. We have an AKS private cluster running on which opencost is deployed using the helm-chart here
2. The secrets service-key.json, cloud-integration.json along with the service principal and associated custom roles were created as specified here

Issue

1. Upon trying to access the "Cloud Costs", seeing a "502 Bad Gateway" error in the UI
2. The opencost container keeps restarting with exit code 2 and seems to not proceed further after the log
   2023-12-21T09:52:38.134643073Z INF Using ratecard query OfferDurableId eq 'MS-AZR-0003p' and Currency eq 'USD' and Locale eq 'en-US' and RegionInfo eq 'US'

GCP Permission issues

Hey all!

I am trying to setup opencost to monitor a test gcp cluster for finical improvement etc. I have given an API key Compute Admin and Billing Project Manager. When doing this I get an error about not being able to view reserved instances. Should I be giving different permissions on this? or should I just follow the cost analyzer doc for the permissions the key or SA would need?

Thanks!

PVC `name` and Deployment `claimName` mismatch

Issue

Enabling opencost.exporter.persistence in the chart, which generates a PVC for CSV exports, causes the Opencost Pod to never start, as it fails to find the PVC with the given name.

Root Cause

The chart generates a PVC with a name that the Deployment manifest should reference in spec.template.spec.volumes.persistentVolumeClaim.claimName.

Instead, the PVC template generates a name that doesn't match the one referenced in the Deployment manifest's claimName.

Solution

Change either one of the names (PVC name or Deployment claimName) so they match across manifests.