docker image does not recognise timezone appropriately about alertmanager

[grobinson-grafana]

Hi! 👋 Your timezone in /etc/localtime is plain text, but as far as I understand it needs to be a symlink to a timezone file in /usr/share/zoneinfo:

The /etc/localtime file configures the system-wide timezone of the local system that is used by applications for presentation to the user. It should be an absolute or relative symbolic link pointing to /usr/share/zoneinfo/, followed by a timezone identifier such as "Europe/Berlin" or "Etc/UTC". The resulting link should lead to the corresponding binary tzfile(5) timezone data for the configured timezone.

https://www.freedesktop.org/software/systemd/man/latest/localtime.html

Can you try this and see if it works?

[instantdreams]

My current approach works for the other 36+ containers I run. It works for Promtail. It doesn't work for Alertmanager, Loki, Prometheus, or Node-Exporter.

I did check the state of the files in the container:

~$ docker exec alertmanager cat /etc/timezone
America/Edmonton


$ docker exec alertmanager cat /etc/localtime
TZif2[binary-data]LMTMDTMSTMWTMPT
MST7MDT,M3.2.0,M11.1.0


$ docker exec alertmanager date
Wed Apr 17 14:56:40 UTC 2024

[grobinson-grafana]

D'oh! I misread the original comment. Your /etc/localtime does look correct. When I do this though, the date is correct. For example:

/alertmanager # date
Wed Apr 17 15:01:14 UTC 2024
/alertmanager # ln -sf /usr/share/zoneinfo/America/Edmonton /etc/localtime
/alertmanager # date
Wed Apr 17 09:01:48 MDT 2024

[instantdreams]

Here's my result:

$ docker exec -it alertmanager sh
/alertmanager $ date
Wed Apr 17 15:12:26 UTC 2024
/alertmanager $ exit
$ date
Wed 17 Apr 2024 09:12:28 AM MDT

Are you running alertmanager inside a docker image? Can you share your sanitised compose file details?

Addendum: To make sure both the host and the container output the date in the same format, I used the following:

$ docker exec -it alertmanager sh
/alertmanager $ date -R
Wed, 17 Apr 2024 15:14:26 +0000
/alertmanager $ exit
$ date --rfc-email
Wed, 17 Apr 2024 09:14:30 -0600

[grobinson-grafana]

OK! I think I know what the issue is – I suspect the base image which the docker images for Prometheus and Alertmanager are built on do not have tzdata installed. This means the TZ environment variable doesn't work. Can you create an issue in https://github.com/prometheus/busybox?

[grobinson-grafana]

Hi! 👋 I opened an issue for this but I suspect it will be rejected. Please see the following in the FAQ https://prometheus.io/docs/introduction/faq/#can-i-change-the-timezone-why-is-everything-in-utc.

[jkroepke]

Reading the issues, alermanger and prometheus works only in UTC exklusive.

In our company, we are using Grafana to manage the Alertmanager alerts and Grafana will covert the UTC times into local time. Create and silence with local time in grafana will be forwarded in UTC at AM.