galaxyproject / ansible-postgresql Goto Github PK

View Code? Open in Web Editor NEW

117.0 117.0 57.0 72 KB

An Ansible role for managing a PostgreSQL server

Home Page: https://galaxy.ansible.com/galaxyproject/postgresql

Python 76.99% Shell 4.40% Jinja 18.60%

ansible-postgresql's People

Contributors

Stargazers

Watchers

ansible-postgresql's Issues

add backup rotation

https://github.com/xolox/python-rotate-backups

Backup with pg_dump

The fancy thing is very fancy, but lacks restore instructions and I'm not entirely sure what the restore process is, just replace the directory?

I'd love to have something simpler that just does pg_dump on cron, maybe with some hooks for admins to insert their own scripting, like @Slugger70 uses to post to slack + swift.

Backups not working on Postgres 15

Running /var/lib/postgresql/backup/bin/backup.py --rsync-backup-opts '\-rptg' --keep 30 --backup --clean-archive /archive will fail with

INFO:root:Initiating backup with pg_start_backup()
INFO:root:Connecting to database
INFO:root:Backup label is: 20240216T184527Z
Traceback (most recent call last):
  File "/var/lib/postgresql/backup/bin/backup.py", line 313, in <module>
    main(sys.argv[1:])
  File "/var/lib/postgresql/backup/bin/backup.py", line 300, in main
    initiate_backup()
  File "/var/lib/postgresql/backup/bin/backup.py", line 155, in initiate_backup
    state.cursor.execute(START_BACKUP_SQL, {'label': state.label})
psycopg2.errors.UndefinedFunction: function pg_start_backup(unknown, boolean, boolean) does not exist
LINE 1: SELECT pg_start_backup('20240216T184527Z', false, false)
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

because the pg_start_backup() function was removed in the PostgreSQL 15 release.

execuing rsync --ignore-existing -ptg pg_xlog/0000000100000000000000C8 /postgresql-backups/current/wal failed with code 12: rsync: change_dir#3 "/postgresql-backups/current" failed: No such file or directory (2) rsync error: errors selecting input/output files, dirs (code 3) at main.c(625) [Receiver=3.0.9] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(605) [sender=3.0.9]

I wonder if this went broken in commit de74a37e1343ded91ac6995afa21244570961a01 where creating backup dir with mkdir -p was removed.

Or do we have something else wrong here?

role fails in check mode

When trying out this role without any vars on a fresh debian 10 system I got this error:

FAILED! => {
    "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'stdout'\n\nThe error appears to be in '/home/peter/.ansible/roles/galaxyproject.postgresql/tasks/debian.yml': line 33, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set version fact\n  ^ here\n"
}

This is apparently because the task before, where the mentioned variable is to be registered, reports "msg": "skipped, running in check mode"

backup does not work in general

backup does not work in general. Look at ansible_postgresql_backup and ansible_postgresql_walbackup variables.
It is undefined.

Originally posted by @Valeronlol in #40 (comment)

Unable to find a match: postgresql*-server

I have a fresh Rocky Linux 8 VM (vggp-v60-j326-d1dfcf46c4cd-main) however installing postgresql15-server fails at the install step:

ansible-postgresql/tasks/redhat.yml

Line 50 in f23cb7a

- name: Install PostgreSQL (RedHat)

with:

All matches were filtered out by modular filtering for argument: postgresql15-server
Error: Unable to find a match: postgresql15-server

Should this case be covered by the role?

Additionally, manually installing with AppStream disabled, resulted in the same error (sudo yum --disablerepo AppStream install postgresql15-server) and the only way I've found to resolve the issue was to disable the postgresql stream module in AppStream: sudo yum module disable postgresql

Service name is incorrect on recent Debian-based systems

Recent Debian systems get a service named postgresql.service but restarting this actually does not restart PostgreSQL, for this you have to restart postgresql@<version>-<cluster> (e.g. postgresql@10-main).

custom data directory

The documentation says this:
postgresql_pgdata: Only set this if you have changed the $PGDATA directory from the package default. Note this does not configure PostgreSQL to actually use a different directory, you will need to do that yourself, it just allows the role to properly locate the directory.
So how would one go about initializing a pg cluster with a custom data directory?

galaxyproject.postgresql : Install repository key task broken

Our deployment is now failing on RHEL7 due to task Install repository key in galaxyproject.postgresql.

The task tries to fetch from: https://download.postgresql.org/pub/repos/yum/RPM-GPG-KEY-PGDG which no longer exists.

I think this is due to this change to the GPG signatures: https://yum.postgresql.org/news/pgdg-rpm-repo-gpg-key-update/

Unable to install in a vagrant box

Hi,

I have just tested your role but I am unable to install postgresql using it in my vagrant box.

I have the following error :

TASK [postgres : Set conf.d include in postgresql.conf] ************************
task path: /home/jobou/workspace/weenect-deployment/roles/postgres/tasks/main.yml:31
<weenect.dev> ESTABLISH SSH CONNECTION FOR USER: vagrant
<weenect.dev> SSH: EXEC sshpass -d14 ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/jobou/.ansible/cp/ansible-ssh-%h-%p-%r -tt weenect.dev '( umask 22 && mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455389704.21-41942075870883 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455389704.21-41942075870883 )" )'
<weenect.dev> PUT /tmp/tmpU2yMxB TO /home/vagrant/.ansible/tmp/ansible-tmp-1455389704.21-41942075870883/lineinfile
<weenect.dev> SSH: EXEC sshpass -d14 sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/jobou/.ansible/cp/ansible-ssh-%h-%p-%r '[weenect.dev]'
<weenect.dev> ESTABLISH SSH CONNECTION FOR USER: vagrant
<weenect.dev> SSH: EXEC sshpass -d14 ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o User=vagrant -o ConnectTimeout=10 -o ControlPath=/home/jobou/.ansible/cp/ansible-ssh-%h-%p-%r -tt weenect.dev '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-eavbsmhcbgzykcrtdvvrzikdgcgftlxh; LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/vagrant/.ansible/tmp/ansible-tmp-1455389704.21-41942075870883/lineinfile; rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1455389704.21-41942075870883/" > /dev/null 2>&1'"'"'"'"'"'"'"'"''"'"''
fatal: [weenect.dev]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_args": {"backrefs": false, "backup": true, "content": null, "create": false, "delimiter": null, "dest": "/etc/postgresql/9.4/main/postgresql.conf", "directory_mode": null, "follow": false, "force": null, "group": null, "insertafter": null, "insertbefore": null, "line": "include_dir 'conf.d'", "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": null, "state": "present", "validate": null}, "module_name": "lineinfile"}, "msg": "Destination /etc/postgresql/9.4/main/postgresql.conf does not exist !", "rc": 257}

For information, I am using the debian/jessie64 box.

vagrant@debian-jessie:~$ sudo lsb_release -c
Codename:   jessie

Thanks in advance

yum or pgdg option?

Right now it seems the flexible apt and pgdg options are available for debian apt-based systems, but not for redhat based ones. Any plan to fix this? Or is this related to the mirror problem,

redhat.yml file contents snippet:

# Not supported (and no good workaround) until there is a solution for https://github.com/ansible/ansible/issues/41178
# - name: Ensure that only the desired PostgreSQL version's repo is enabled
#   yum_repository:
#     name: item.repoid
#     enabled: "{{ (item.repoid == 'pgdg' ~ __postgresql_version_dotless) if item.repoid.startswith('pgdg') else item.state == 'enabled' }}"
#   # "{{ __postgresql_yum_repolist_result.results | selectattr('repoid', 'startswith', 'pgdg') | list }}" would be nice
#   # here but alas there is no `startswith` test
#   loop: "{{ __postgresql_yum_repolist_result.results }}"

/usr/bin/env python no longer works on some distributions

Arguably a distribution problem (looking at you ubuntu) but, the backup script stopped working in Dec 2021.

test the backups (also user facing documentation)

That's a pretty comprehensive list of steps, could probs do that in gh actions.

Originally posted by @hexylena in #30 (comment)

Backup fails if "mail" fails or the command is not found

This makes mailutils a dependency in Ubuntu, didn't test for other OS.

"msg": "Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user (rc: 1, err: chmod: invalid mode: ‘A+user:postgres:rx:allow’\nTry 'chmod --help' for more information.\n}). For information on working around this,

"msg": "Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user (rc: 1, err: chmod: invalid mode: ‘A+user:postgres:rx:allow’\nTry 'chmod --help' for more information.\n}). For information on working around this,

postgresql_backup_remote_rsync_path is undefined

From a role where I used the core modules for postgres to install and create users and db's, I included ansible-postgresql to make automated backups.

- name: Copias de seguridad                                           
  include_role:                                                       
    name: postgresql                                                  
  become: True                                                        
  vars:                                                               
     postgresql_backup_dir: /backups                                   
   tags:                                                               
     - a_tag                                                            
     - another_tag

Then, calling the playbook with:
$ ansible-playbook main-playbook.yml --tags another_tag -K

I get an error when installing the backup scripts:


TASK [postgresql : Install backup scripts] ***************************************************************
ok: [my_server] => (item=backup_working_wal.sh)
failed: [my_server] (item=archive_wal.sh) => {"failed": true, "item": "archive_wal.sh", "msg": "AnsibleUndefinedVariable: 'postgresql_backup_remote_rsync_path' is undefined"}
failed: [my_server] (item=scheduled_backup.sh) => {"failed": true, "item": "scheduled_backup.sh", "msg": "AnsibleUndefinedVariable: 'postgresql_backup_remote_rsync_path' is undefined"}

I guess it's just a matter of specifying that variable, but as the readme says all vars are optional, maybe it should be clarified or given a default value.

Thanks for the role, anyway, gonna save me a lot of time!

MIT PGP service unavailable

When using pgdg flavor, there may be an issue with MIT PGP server

TASK [galaxyproject.postgresql : Install pgdg package signing key (Debian/pgdg)] ************************************************************************************************
FAILED - RETRYING: Install pgdg package signing key (Debian/pgdg) (5 retries left).
FAILED - RETRYING: Install pgdg package signing key (Debian/pgdg) (4 retries left).
FAILED - RETRYING: Install pgdg package signing key (Debian/pgdg) (3 retries left).
FAILED - RETRYING: Install pgdg package signing key (Debian/pgdg) (2 retries left).
FAILED - RETRYING: Install pgdg package signing key (Debian/pgdg) (1 retries left).
fatal: [db2]: FAILED! => {"attempts": 5, "changed": false, "cmd": "/usr/bin/apt-key adv --no-tty --keyserver pgp.mit.edu --recv ACCC4CF8", "msg": "Error fetching key ACCC4CF8 from keyserver: pgp.mit.edu", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: No data\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: No data"], "stdout": "Executing: /tmp/apt-key-gpghome.PRoLspfaKj/gpg.1.sh --no-tty --keyserver pgp.mit.edu --recv ACCC4CF8\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.PRoLspfaKj/gpg.1.sh --no-tty --keyserver pgp.mit.edu --recv ACCC4CF8"]}

And there is nothing I can do with that :(

Ansible deprecation warning: Using tests as filters is deprecated.

This line occurs twice in tasks/main.yml:

when: "{{ postgresql_version | version_compare('9.3', '>=') }}"

Replacing it with:

when: postgresql_version  >= '9.3'

avoids this deprecation warning from Ansible 2.5:

Using tests as filters is deprecated.
Instead of using`result|version_compare` instead use `result is version_compare`. 
This feature will be removed in version 2.9.

Backup options have issues with `-`

I'm noticing my backups stopped a while back

root@bioinf-galaxy:/data/backups# cat /etc/cron.d/ansible_postgresql_backup
#Ansible: PostgreSQL Backup
0 1 * * * postgres '~postgres/backups'/bin/backup.py  --rsync-backup-opts -rptg --keep 30  --backup --clean-archive /data/backups

Here '~postgres..' isn't getting expanded, removing the ' fixes that, but the backup options also cause issues:

python3.8 ~postgres/backups/bin/backup.py  --keep 30  --backup --clean-archive /data/backups --rsync-backup-opts '-rptg'
usage: backup.py [-h] [--backup] [--keep KEEP] [--clean-archive] [--rsync-connect-opts RSYNC_CONNECT_OPTS] [--rsync-backup-opts RSYNC_BACKUP_OPTS] [--pg-bin-dir PG_BIN_DIR]
                 [-v]
                 backup_path
backup.py: error: argument --rsync-backup-opts: expected one argument

PGDG repo version conflicts with psycopg2

Each PGDG repo contains versions of python-psycopg2, python2-psycopg2, and python3-psycopg2 for the version of PostgreSQL in that repo. When you yum update sometimes it installs a version of psycopg2 from the wrong version's repo. It should match the version of PostgreSQL you've installed, but sometimes it doesn't.

You can fix this with yum install --disablerepo '*' --enablerepo pgdg12 python-psycopg2 (using the correct enablerepo for whatever version of PostgreSQL you've installed).

We should probably have the role set the enabled properly in the yum repo files for the desired and undesired versions of PostgreSQL.

RHEL8 - cannot install due to mismatch between versions

From @vazovn

Setting the __postgresql_version_dotless: "" seems to do the fix. The good news is that one can do this in the ansible playbook yaml file and thus not affect the role. Yet, if in the future postgresql releases the digits (postresqlXX) reappear again, this fix will be a problem.

fatal: [galaxy01-test.educloud.no]: FAILED! => {"changed": false, "failures": ["No package pgdg-redhat-repo available."], "msg": "Failed to install some of the specified packages", "rc": 1, "results": []}

We should write some molecule tests for this role

galaxyproject / ansible-postgresql Goto Github PK

ansible-postgresql's People

Contributors

Stargazers

Watchers

Forkers

ansible-postgresql's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs