aiven / aiven-db-migrate Goto Github PK

Description of the issue

I am unsuccessfully trying to install the tool following the steps in the README.

Using the module directly is possible after manually installing ''psycopg2-binary".

Issue-Type (put a x sign in the square brackets)

• bug report
• feature request
• Documentation improvement
• Other

Checklist

• Running latest version of code.
• This issue has not been reported earlier.

Your environment

• OS
• Python Version
• Release tag/commit of the code

Fedora 39
Python 3.12.2

Expected behaviour

What should happen?

I am able to run the tool

Actual behaviour

What is actually happening?

I get the errors described

Steps to reproduce

1. git clone <>
2. Run any of the two approaches specified in the documentation from the cloned directory.

Any extra info ( for eg. code snippet to reproduce, logs, screenshots etc. )

After running make, I get:

echo "__version__ = \"0.1.3-2-g14bbc70\"" > aiven_db_migrate/migrate/version.py

After running "pip install .", I get:

Defaulting to user installation because normal site-packages is not writeable
Processing /home/<redacted>/python/aiven-db-migrate
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [34 lines of output]
      Traceback (most recent call last):
        File "/home/rafa/.local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/rafa/.local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/rafa/.local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 13, in <module>
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 147, in setup
          _setup_distribution = dist = klass(attrs)
                                       ^^^^^^^^^^^^
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 314, in __init__
          self.metadata.version = self._normalize_version(self.metadata.version)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 350, in _normalize_version
          normalized = str(Version(version))
                           ^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-__768b88/overlay/lib/python3.12/site-packages/setuptools/_vendor/packaging/version.py", line 198, in __init__
          raise InvalidVersion(f"Invalid version: '{version}'")
      setuptools.extern.packaging.version.InvalidVersion: Invalid version: '0.1.3-2-g14bbc70'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Description of the issue

Currently, aiven-db-migrate does not support replication from a GCP CloudSQL database, which uses the pglogical extension in a similar way that Aiven uses the aiven-extras extension. It would be good to detect possible use of the pglogical extension and act accordingly.

Issue-Type (put a x sign in the square brackets)

• bug report
• feature request
• Documentation improvement
• Other

Checklist

• Running latest version of code.
• This issue has not been reported earlier.

Expected behaviour

Replication should be setup and happily proceed.

Actual behaviour

Numerous errors, which I tried to get past one at a time, but would be good to patch into the code.

Any extra info ( for eg. code snippet to reproduce, logs, screenshots etc. )

Things I had to do manually to get some kind of replication setup working:

1. Follow the GCP doc at https://cloud.google.com/sql/docs/postgres/replication/configure-external-replica as far as (but not including) "Configure the external replica"
2. Switch owner of the tables to be migrated to the newly-created replication_user user (not 100% certain this was the only way to do this, but it got me past the error 'must be superuser or replication role to use replication slots'). Note that this removes owner privs from the postgres (or whichever) user, so any applications need to make sure they're using users with approriately-granted roles.
3. Add pglogical to the list of removed schemas at

   aiven-db-migrate/aiven_db_migrate/migrate/pgmigrate.py

   Line 246 in 57427de

   WHERE pg_catalog.pg_namespace.nspname NOT IN ('pg_catalog', 'information_schema')""",

4. Add a filter for pglogical into the extensions loop at

   aiven-db-migrate/aiven_db_migrate/migrate/pgmigrate.py

   Line 918 in 57427de

   target_ext = next(e for e in self.target.pg_ext if e.name == source_ext.name)

5. Run the pg_migrate tool with the replication_user credentials, and the following switches: -f _aiven -f cloudsqladmin -f pglogical --with-table <list of tables>

Standard operating procedure for postgres commandline tools that leverage libpq (such as psql, pg_dump, etc) is that they will look for a password in the .pgpass file in the home directory of the running user. This allows for interacting with databases without having to supply a password in the clear on the commandline. https://www.postgresql.org/docs/12/libpq-pgpass.html

I attempted to run the tool without supplying the password expecting it would leverage .pgpass, but that did not work:

2020-10-29 16:09:39,632	PGMigrate	ERROR	fe_sendauth: no password supplied

Traceback (most recent call last):
  File "/Users/nickwilson/git/aiven-db-migrate/aiven_db_migrate/migrate/pgmigrate.py", line 947, in validate
    if self.source.version > self.target.version:
  File "/Users/nickwilson/git/aiven-db-migrate/aiven_db_migrate/migrate/pgmigrate.py", line 161, in version
    self._version = LooseVersion(self.params["server_version"])
  File "/Users/nickwilson/git/aiven-db-migrate/aiven_db_migrate/migrate/pgmigrate.py", line 154, in params
    params = self.c("SHOW ALL")
  File "/Users/nickwilson/git/aiven-db-migrate/aiven_db_migrate/migrate/pgmigrate.py", line 139, in c
    with self._cursor(dbname=dbname) as cur:
  File "/usr/local/Cellar/[email protected]/3.8.6_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/Users/nickwilson/git/aiven-db-migrate/aiven_db_migrate/migrate/pgmigrate.py", line 121, in _cursor
    conn = psycopg2.connect(**conn_info)
  File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
  File "/usr/local/lib/python3.8/site-packages/psycopg2/extras.py", line 778, in wait_select
    state = conn.poll()
psycopg2.OperationalError: fe_sendauth: no password supplied

A few things I'm noticing wrong with the docs, attempting to run this tool for the first time.

In order to run pip3 install . - there's a version file that is required that is generated by the make command, otherwise you get this nasty business:

    ERROR: Command errored out with exit status 1:
     command: /usr/local/opt/[email protected]/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/mh/dl_8qr6531q1wgqvtmybs1nw0000gn/T/pip-req-build-zs25m8bd/setup.py'"'"'; __file__='"'"'/private/var/folders/mh/dl_8qr6531q1wgqvtmybs1nw0000gn/T/pip-req-build-zs25m8bd/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/mh/dl_8qr6531q1wgqvtmybs1nw0000gn/T/pip-pip-egg-info-_5usqh3l
         cwd: /private/var/folders/mh/dl_8qr6531q1wgqvtmybs1nw0000gn/T/pip-req-build-zs25m8bd/
    Complete output (16 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/mh/dl_8qr6531q1wgqvtmybs1nw0000gn/T/pip-req-build-zs25m8bd/setup.py", line 31, in <module>
        version=get_version(),
      File "/private/var/folders/mh/dl_8qr6531q1wgqvtmybs1nw0000gn/T/pip-req-build-zs25m8bd/setup.py", line 10, in get_version
        return SourceFileLoader("version", "aiven_db_migrate/migrate/version.py").load_module().__version__
      File "<frozen importlib._bootstrap_external>", line 462, in _check_name_wrapper
      File "<frozen importlib._bootstrap_external>", line 962, in load_module
      File "<frozen importlib._bootstrap_external>", line 787, in load_module
      File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
      File "<frozen importlib._bootstrap>", line 702, in _load
      File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 779, in exec_module
      File "<frozen importlib._bootstrap_external>", line 915, in get_code
      File "<frozen importlib._bootstrap_external>", line 972, in get_data
    FileNotFoundError: [Errno 2] No such file or directory: 'aiven_db_migrate/migrate/version.py'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Running the make command generates this file. Seems like if this is the intended process, this step should be included in the docs and migrate/version.py should probably also be in the .gitignore?

➜  aiven-db-migrate git:(master) ✗ make
echo "__version__ = \"0.0.1-13-g8a8bb0b\"" > aiven_db_migrate/migrate/version.py

The second issue in the docs is that the name of the module is incorrect:

➜  aiven-db-migrate git:(master) ✗ python3 -m aiven.migrate -h
/usr/local/opt/[email protected]/bin/python3.8: Error while finding module specification for 'aiven.migrate' (ModuleNotFoundError: No module named 'aiven')

This command does work for me instead (based on the path structure in the repo):

➜  aiven-db-migrate git:(master) ✗ python3 -m aiven_db_migrate.migrate -h
Available commands: pg

On our database instance, the details of the OS are included in the version string that is reported

postgres@linus-db:~/aiven-db-migrate$ psql --version
psql (PostgreSQL) 12.1 (Ubuntu 12.1-1.pgdg18.04+1)

This interferes with the binary detection logic in the migration code as it includes that in the paths it tries to search:

2020-10-30 15:18:14,652	PGMigrate	ERROR	Couldn't find pg version '12.1 (Ubuntu 12.1-1.pgdg18.04+1)' bin dir, tried: ['/usr/pgsql-12.1 (Ubuntu 12.1-1.pgdg18.04+1)/bin', '/usr/pgsql-12.1 (Ubuntu 12.1-1.pgdg18/bin', '/usr/lib/postgresql/12.1 (Ubuntu 12.1-1.pgdg18.04+1)/bin', '/usr/lib/postgresql/12.1 (Ubuntu 12.1-1.pgdg18/bin']

Hi I was attempting to install the tool using venv but ran into this:

➜  aiven-db-migrate git:(master) ✗ python3 -m venv venv
➜  aiven-db-migrate git:(master) ✗ . venv/bin/activate
(venv) ➜  aiven-db-migrate git:(master) ✗ pip install .
Processing /Users/djb/work/aiven-db-migrate
    ERROR: Command errored out with exit status 1:
     command: /Users/djb/work/aiven-db-migrate/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/zm/4bd70yhx4m1drjwxpfd7pd_40000gn/T/pip-req-build-nv63xi38/setup.py'"'"'; __file__='"'"'/private/var/folders/zm/4bd70yhx4m1drjwxpfd7pd_40000gn/T/pip-req-build-nv63xi38/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/zm/4bd70yhx4m1drjwxpfd7pd_40000gn/T/pip-pip-egg-info-w48lrc8g
         cwd: /private/var/folders/zm/4bd70yhx4m1drjwxpfd7pd_40000gn/T/pip-req-build-nv63xi38/
    Complete output (16 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/zm/4bd70yhx4m1drjwxpfd7pd_40000gn/T/pip-req-build-nv63xi38/setup.py", line 31, in <module>
        version=get_version(),
      File "/private/var/folders/zm/4bd70yhx4m1drjwxpfd7pd_40000gn/T/pip-req-build-nv63xi38/setup.py", line 10, in get_version
        return SourceFileLoader("version", "aiven_db_migrate/migrate/version.py").load_module().__version__
      File "<frozen importlib._bootstrap_external>", line 462, in _check_name_wrapper
      File "<frozen importlib._bootstrap_external>", line 962, in load_module
      File "<frozen importlib._bootstrap_external>", line 787, in load_module
      File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
      File "<frozen importlib._bootstrap>", line 702, in _load
      File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 779, in exec_module
      File "<frozen importlib._bootstrap_external>", line 915, in get_code
      File "<frozen importlib._bootstrap_external>", line 972, in get_data
    FileNotFoundError: [Errno 2] No such file or directory: 'aiven_db_migrate/migrate/version.py'
    ----------------------------------------

Apologies if I'm missing something, I don't really have python experience.

We're about to use this to test logical replication migration from databases we fully own in our infrastructure to Aiven databases.

Two main questions:
Is a superuser ok to use on the source db?
For the target database, if we want to not use the defaultdb but a different db that is app specific, are there any permissions we should set up for the avnadmin user before running the migration tool?

Thanks!

Description of the issue

We want to claim the titles for our python projects/packages on PyPI. We currently only claimed around 2 of them which leaves it open for a bad actor to claim the name and put malicious code there, potentially tricking users of any of our repos into installing it.
We have an Aiven account that we need to claim it under in PyPi

Issue-Type (put a x sign in the square brackets)

• bug report
• feature request
• Documentation improvement
• Other

Checklist

• Running latest version of code.
• This issue has not been reported earlier.

Your environment

• OS
• Python Version
• Release tag/commit of the code

Expected behaviour

Package should be under aiven_oss in PyPI

Actual behaviour

Not present on PyPI

Steps to reproduce

Any extra info ( for eg. code snippet to reproduce, logs, screenshots etc. )

After successfully setting up logical replication, we are encountering the following error over and over again in our target database:

[webapp-db-sbx-1]2020-11-10T23:21:01.098136[postgresql-12][85-1] user=,db=,app=,client= LOG: logical replication table synchronization worker for subscription "aiven_db_migrate_dafhp6nhpahgqn_sub", table "spatial_ref_sys" has started
[webapp-db-sbx-1]2020-11-10T23:21:01.145297[postgresql-12][86-1] user=,db=,app=,client= ERROR: duplicate key value violates unique constraint "spatial_ref_sys_pkey"
[webapp-db-sbx-1]2020-11-10T23:21:01.145522[postgresql-12][86-2] user=,db=,app=,client= DETAIL: Key (srid)=(3819) already exists.
[webapp-db-sbx-1]2020-11-10T23:21:01.145589[postgresql-12][86-3] user=,db=,app=,client= CONTEXT: COPY spatial_ref_sys, line 1
[webapp-db-sbx-1]2020-11-10T23:21:01.148562[postgresql-12][85-1] user=,db=,app=,client= LOG: background worker "logical replication worker" (PID 2368) exited with exit code 1

The table in question here, spatial_ref_sys, is one of the system tables from the postgis extension. The table is populated when the extension is created, so it makes sense that trying to logically replicate the contents of this table is failing because the data is already there.

Since it is not currently possible to specify include/exclude lists of tables, there is no way to avoid this from happening when postgis is one of the extensions that is replicated. I imagine the same issue exists for other extensions that have their own built-in/system tables as well.

Description of the issue

The tool doesn't work on a Mac, because of the hard-coded list of directories where the postgres tooling is available.

Issue-Type (put a x sign in the square brackets)

• bug report
• feature request
• Documentation improvement
• Other

Checklist

• Running latest version of code.
• This issue has not been reported earlier.

Your environment

• OS - MacOS Monterey 12.1
• Python Version - 3.9.10
• Release tag/commit of the code - tag: 0.0.5 / commit 57427de

Expected behaviour

The tool should proceed as expected

Actual behaviour

aiven_db_migrate.migrate.errors.PGMigrateValidationFailedError: Couldn't find bin dir for any pg version between '14.0' and '14.2', tried ['/usr/pgsql-([0-9]+(\\.[0-9]+)*)/bin', '/usr/lib/postgresql/([0-9]+(\\.[0-9]+)*)/bin']

Steps to reproduce

Run any aiven-db-migrate command on a Mac

Any extra info ( for eg. code snippet to reproduce, logs, screenshots etc. )

The tool does work if this check is overridden. I have psql installed through Homebrew, so the binaries are available in /opt/homebrew/bin. Line 29 in pgutils.py is trying to get the version number from the path, which isn't available in the homebrew method, but if I change line 1180 in pgmigrate.py to:

self.pgbin = Path("/opt/homebrew/bin")

then the tool proceeds to work as expected. This isn't the right solution, it's only so I could check that there aren't any other problems, but it suggests that it's worth solving the issue properly.

Description of the issue

Describe the issue you are experiencing.

Issue-Type (put a x sign in the square brackets)

• bug report
• feature request
• Documentation improvement
• Other

Checklist

• Running latest version of code.
• This issue has not been reported earlier.

Your environment

• Ubuntu
• 3.9
• 0.0.5

Expected behavior

Blobs should be probably migrated

Actual behavior

The migration takes place but the blobs seems to be broken.
My server throws the following error when trying to start against the migrated target db:

2022-04-04 13:35:00.292 [main] ERROR c.u.f.jpa.searchparam.retry.Retrier [Retrier.java:79] Retry failure 8/60: org.springframework.orm.jpa.JpaSystemException: Unable to access lob stream; nested exception is org.hibernate.HibernateException: Unable to access lob stream

Migration with pg_dump works, but takes too long.

Steps to reproduce

1. Download latest version of pg_migrate / aiven migrate
2. Setup an HAPI FHIR Server (https://github.com/hapifhir/hapi-fhir-jpaserver-starter)
3. Create FHIR records or load sample data
4. Migrate the database using replication method
5. Make ur fhir server point to the target db
6. Lob stream is gonna fail

Any extra info ( for eg. code snippet to reproduce, logs, screenshots etc. )

GRANT SELECT ON TABLE aiven_extras.pg_stat_replication TO avnadmin WITH GRANT OPTION;

-- PostgreSQL database dump complete

2022-04-04 12:54:00,079 PGTarget INFO extension "aiven_extras" already exists

2022-04-04 12:54:00,931 PGSource INFO Creating publication 'aiven_db_migrate_fhir_r5_production_pub' for all tables in database 'fhir_r5_production', has_aiven_extras: True
2022-04-04 12:54:00,964 PGSource INFO Creating replication slot 'aiven_db_migrate_fhir_r5_production_slot' in database 'fhir_r5_production'
2022-04-04 12:54:01,011 PGSource INFO Created replication slot RealDictRow([('slot_name', 'aiven_db_migrate_fhir_r5_production_slot'), ('lsn', '10B/2F627D38')]) in database 'fhir_r5_production'
2022-04-04 12:54:01,072 PGTarget INFO Creating subscription 'aiven_db_migrate_fhir_r5_production_sub' to 'aiven_db_migrate_fhir_r5_production_pub' with slot 'aiven_db_migrate_fhir_r5_production_slot',
has_aiven_extras: True
2022-04-04 12:54:01,146 PGMigrate INFO Logical replication setup successful for database 'fhir_r5_production'

Roles:
rolname: 'pbi', rolpassword: None, status: 'exists', message: 'role already exists'
rolname: '_aiven', rolpassword: None, status: 'exists', message: 'role already exists'
rolname: 'avnadmin', rolpassword: None, status: 'exists', message: 'role already exists'
rolname: 'grafana', rolpassword: None, status: 'exists', message: 'role already exists'

Databases:
dbaname: 'fhir_r5_production', method: 'replication', status: 'running', message: 'created and migrated database'

In our environment, our source database superuser connection is limited only to socket connections, and not standard TCP/network connections. According to the docs, any valid libpq connection string should work, but it fails if you pass in a connection string with no host/port defined.

Traceback (most recent call last):
  File "/var/lib/postgresql/aiven-db-migrate/aiven_db_migrate/migrate/pgmigrate.py", line 943, in validate
    self.source.conn_info["host"] == self.target.conn_info["host"]
KeyError: 'host'

The connection string in question works with psql:

postgres@linus-db:~/aiven-db-migrate$ psql "postgres:///dafhp6nhpahgqn"
psql (12.1 (Ubuntu 12.1-1.pgdg18.04+1))
Type "help" for help.

dafhp6nhpahgqn=# \conninfo
You are connected to database "dafhp6nhpahgqn" as user "postgres" via socket in "/var/run/postgresql" at port "5432".
dafhp6nhpahgqn=#

Description of the issue

Describe the issue you are experiencing.

Issue-Type (put a x sign in the square brackets)

• [X ] bug report
• feature request
• Documentation improvement
• Other

Checklist

• [X ] Running latest version of code.
• [ X] This issue has not been reported earlier.

Your environment

• OS Ubuntu 22.04
• Python Version 3.10
• Release tag/commit of the code

Expected behaviour

Should work out of the box. Espcially given that aiven is the provider of both DBs

Actual behaviour

Setting up a logical replication fails and the script falls back to pg_dump.
However, I need to keep both DBs in sync.
"Logical replication failed with error: 'must be superuser to create FOR ALL TABLES publication', fallback to dump"

Steps to reproduce

setup a PSQL DB on OVH (which is powered by aiven)
Link your GCP account to aiven
Setup a PSQL DB on aiven cloud (hosted on gcp)
Use pg_migrate tool to setup logical replication

Any extra info ( for eg. code snippet to reproduce, logs, screenshots etc. )

pg_migrate -s "postgres://avnadmin:[email protected]:20184/defaultdb" -t "postgres://avnadmin:[email protected]:14837/defaultdb" 2022-12-10 10:40:07,357 PGSource WARNING Couldn't connect to database '_aiven': OperationalError('FATAL: pg_hba.conf rejects connection for host "10.5.130.98", user "avnadmin", database "_aiven", SSL encryption\nconnection to server at "postgresql-48e9e04b-o08a726b8.database.cloud.ovh.net" (10.5.131.80), port 20184 failed: FATAL: no pg_hba.conf entry for host "10.5.130.98", user "avnadmin", database "_aiven", no encryption\n') 2022-12-10 10:40:08,166 PGMigrate INFO Access to source database '_aiven' is rejected 2022-12-10 10:40:08,408 PGTarget WARNING Couldn't connect to database '_aiven': OperationalError('FATAL: pg_hba.conf rejects connection for host "141.94.211.55", user "avnadmin", database "_aiven", SSL encryption\nconnection to server at "production-de-iatros-production.aivencloud.com" (34.xxx.xxx.xxx), port 14837 failed: FATAL: no pg_hba.conf entry for host "141.94.211.55", user "avnadmin", database "_aiven", no encryption\n') 2022-12-10 10:40:08,851 PGMigrate INFO Database 'defaultdb' already exists in target 2022-12-10 10:40:08,970 PGMigrate INFO Database 'account_service_production' will be created in target 2022-12-10 10:40:09,071 PGMigrate INFO Database 'content_service_production' will be created in target 2022-12-10 10:40:09,176 PGMigrate INFO Database 'poll_service_production' will be created in target 2022-12-10 10:40:09,276 PGMigrate INFO Database 'utility_service_production' will be created in target 2022-12-10 10:40:09,393 PGMigrate INFO Database 'reff_service_production' will be created in target 2022-12-10 10:40:09,510 PGMigrate INFO Database 'iatros_sftp_production' will be created in target 2022-12-10 10:40:09,632 PGMigrate INFO Database 'randomization_production' will be created in target 2022-12-10 10:40:10,039 PGMigrate INFO Extension 'plpgsql' is installed in source and target database 'defaultdb', source version: '1.0', target version: '1.0' 2022-12-10 10:40:10,389 PGMigrate INFO Extension 'plpgsql' version '1.0' available for installation in target, source version: '1.0' 2022-12-10 10:40:10,488 PGMigrate INFO Extension 'plpgsql' is whitelisted in target 2022-12-10 10:40:10,488 PGMigrate INFO Extension 'plpgsql' version '1.0' will be installed in target database 'account_service_production' 2022-12-10 10:40:10,599 PGMigrate INFO Extension 'btree_gin' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:10,600 PGMigrate INFO Extension 'btree_gin' is whitelisted in target 2022-12-10 10:40:10,600 PGMigrate INFO Extension 'btree_gin' version '1.3' will be installed in target database 'account_service_production' 2022-12-10 10:40:10,703 PGMigrate INFO Extension 'btree_gist' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:10,703 PGMigrate INFO Extension 'btree_gist' is whitelisted in target 2022-12-10 10:40:10,703 PGMigrate INFO Extension 'btree_gist' version '1.6' will be installed in target database 'account_service_production' 2022-12-10 10:40:10,807 PGMigrate INFO Extension 'citext' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:10,807 PGMigrate INFO Extension 'citext' is whitelisted in target 2022-12-10 10:40:10,807 PGMigrate INFO Extension 'citext' version '1.6' will be installed in target database 'account_service_production' 2022-12-10 10:40:10,910 PGMigrate INFO Extension 'cube' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:10,910 PGMigrate INFO Extension 'cube' is whitelisted in target 2022-12-10 10:40:10,910 PGMigrate INFO Extension 'cube' version '1.5' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,016 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:11,016 PGMigrate INFO Extension 'fuzzystrmatch' is whitelisted in target 2022-12-10 10:40:11,017 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,121 PGMigrate INFO Extension 'hstore' version '1.8' available for installation in target, source version: '1.8' 2022-12-10 10:40:11,121 PGMigrate INFO Extension 'hstore' is whitelisted in target 2022-12-10 10:40:11,121 PGMigrate INFO Extension 'hstore' version '1.8' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,227 PGMigrate INFO Extension 'intarray' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:11,227 PGMigrate INFO Extension 'intarray' is whitelisted in target 2022-12-10 10:40:11,227 PGMigrate INFO Extension 'intarray' version '1.5' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,331 PGMigrate INFO Extension 'isn' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:11,332 PGMigrate INFO Extension 'isn' is whitelisted in target 2022-12-10 10:40:11,332 PGMigrate INFO Extension 'isn' version '1.2' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,434 PGMigrate INFO Extension 'ltree' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:11,434 PGMigrate INFO Extension 'ltree' is whitelisted in target 2022-12-10 10:40:11,434 PGMigrate INFO Extension 'ltree' version '1.2' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,551 PGMigrate INFO Extension 'pg_trgm' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:11,551 PGMigrate INFO Extension 'pg_trgm' is whitelisted in target 2022-12-10 10:40:11,551 PGMigrate INFO Extension 'pg_trgm' version '1.6' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,655 PGMigrate INFO Extension 'pgcrypto' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:11,655 PGMigrate INFO Extension 'pgcrypto' is whitelisted in target 2022-12-10 10:40:11,656 PGMigrate INFO Extension 'pgcrypto' version '1.3' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,759 PGMigrate INFO Extension 'seg' version '1.4' available for installation in target, source version: '1.4' 2022-12-10 10:40:11,760 PGMigrate INFO Extension 'seg' is trusted in target 2022-12-10 10:40:11,760 PGMigrate INFO Extension 'seg' version '1.4' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,864 PGMigrate INFO Extension 'unaccent' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:11,864 PGMigrate INFO Extension 'unaccent' is whitelisted in target 2022-12-10 10:40:11,864 PGMigrate INFO Extension 'unaccent' version '1.1' will be installed in target database 'account_service_production' 2022-12-10 10:40:11,968 PGMigrate INFO Extension 'uuid-ossp' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:11,969 PGMigrate INFO Extension 'uuid-ossp' is whitelisted in target 2022-12-10 10:40:11,969 PGMigrate INFO Extension 'uuid-ossp' version '1.1' will be installed in target database 'account_service_production' 2022-12-10 10:40:12,073 PGMigrate INFO Extension 'plpgsql' version '1.0' available for installation in target, source version: '1.0' 2022-12-10 10:40:12,074 PGMigrate INFO Extension 'plpgsql' is whitelisted in target 2022-12-10 10:40:12,074 PGMigrate INFO Extension 'plpgsql' version '1.0' will be installed in target database 'content_service_production' 2022-12-10 10:40:12,176 PGMigrate INFO Extension 'btree_gin' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:12,176 PGMigrate INFO Extension 'btree_gin' is whitelisted in target 2022-12-10 10:40:12,177 PGMigrate INFO Extension 'btree_gin' version '1.3' will be installed in target database 'content_service_production' 2022-12-10 10:40:12,278 PGMigrate INFO Extension 'btree_gist' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:12,279 PGMigrate INFO Extension 'btree_gist' is whitelisted in target 2022-12-10 10:40:12,279 PGMigrate INFO Extension 'btree_gist' version '1.6' will be installed in target database 'content_service_production' 2022-12-10 10:40:12,381 PGMigrate INFO Extension 'citext' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:12,381 PGMigrate INFO Extension 'citext' is whitelisted in target 2022-12-10 10:40:12,381 PGMigrate INFO Extension 'citext' version '1.6' will be installed in target database 'content_service_production' 2022-12-10 10:40:12,479 PGMigrate INFO Extension 'cube' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:12,479 PGMigrate INFO Extension 'cube' is whitelisted in target 2022-12-10 10:40:12,479 PGMigrate INFO Extension 'cube' version '1.5' will be installed in target database 'content_service_production' 2022-12-10 10:40:12,582 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:12,582 PGMigrate INFO Extension 'fuzzystrmatch' is whitelisted in target 2022-12-10 10:40:12,582 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' will be installed in target database 'content_service_production' 2022-12-10 10:40:12,690 PGMigrate INFO Extension 'hstore' version '1.8' available for installation in target, source version: '1.8' 2022-12-10 10:40:12,691 PGMigrate INFO Extension 'hstore' is whitelisted in target 2022-12-10 10:40:12,691 PGMigrate INFO Extension 'hstore' version '1.8' will be installed in target database 'content_service_production' 2022-12-10 10:40:12,794 PGMigrate INFO Extension 'intarray' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:12,794 PGMigrate INFO Extension 'intarray' is whitelisted in target 2022-12-10 10:40:12,794 PGMigrate INFO Extension 'intarray' version '1.5' will be installed in target database 'content_service_production' 2022-12-10 10:40:12,897 PGMigrate INFO Extension 'isn' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:12,898 PGMigrate INFO Extension 'isn' is whitelisted in target 2022-12-10 10:40:12,898 PGMigrate INFO Extension 'isn' version '1.2' will be installed in target database 'content_service_production' 2022-12-10 10:40:13,000 PGMigrate INFO Extension 'ltree' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:13,000 PGMigrate INFO Extension 'ltree' is whitelisted in target 2022-12-10 10:40:13,000 PGMigrate INFO Extension 'ltree' version '1.2' will be installed in target database 'content_service_production' 2022-12-10 10:40:13,118 PGMigrate INFO Extension 'pg_trgm' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:13,118 PGMigrate INFO Extension 'pg_trgm' is whitelisted in target 2022-12-10 10:40:13,118 PGMigrate INFO Extension 'pg_trgm' version '1.6' will be installed in target database 'content_service_production' 2022-12-10 10:40:13,222 PGMigrate INFO Extension 'pgcrypto' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:13,223 PGMigrate INFO Extension 'pgcrypto' is whitelisted in target 2022-12-10 10:40:13,223 PGMigrate INFO Extension 'pgcrypto' version '1.3' will be installed in target database 'content_service_production' 2022-12-10 10:40:13,324 PGMigrate INFO Extension 'seg' version '1.4' available for installation in target, source version: '1.4' 2022-12-10 10:40:13,324 PGMigrate INFO Extension 'seg' is trusted in target 2022-12-10 10:40:13,324 PGMigrate INFO Extension 'seg' version '1.4' will be installed in target database 'content_service_production' 2022-12-10 10:40:13,432 PGMigrate INFO Extension 'unaccent' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:13,432 PGMigrate INFO Extension 'unaccent' is whitelisted in target 2022-12-10 10:40:13,432 PGMigrate INFO Extension 'unaccent' version '1.1' will be installed in target database 'content_service_production' 2022-12-10 10:40:13,535 PGMigrate INFO Extension 'uuid-ossp' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:13,536 PGMigrate INFO Extension 'uuid-ossp' is whitelisted in target 2022-12-10 10:40:13,536 PGMigrate INFO Extension 'uuid-ossp' version '1.1' will be installed in target database 'content_service_production' 2022-12-10 10:40:13,639 PGMigrate INFO Extension 'plpgsql' version '1.0' available for installation in target, source version: '1.0' 2022-12-10 10:40:13,639 PGMigrate INFO Extension 'plpgsql' is whitelisted in target 2022-12-10 10:40:13,639 PGMigrate INFO Extension 'plpgsql' version '1.0' will be installed in target database 'poll_service_production' 2022-12-10 10:40:13,740 PGMigrate INFO Extension 'btree_gin' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:13,740 PGMigrate INFO Extension 'btree_gin' is whitelisted in target 2022-12-10 10:40:13,741 PGMigrate INFO Extension 'btree_gin' version '1.3' will be installed in target database 'poll_service_production' 2022-12-10 10:40:13,841 PGMigrate INFO Extension 'btree_gist' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:13,842 PGMigrate INFO Extension 'btree_gist' is whitelisted in target 2022-12-10 10:40:13,842 PGMigrate INFO Extension 'btree_gist' version '1.6' will be installed in target database 'poll_service_production' 2022-12-10 10:40:13,944 PGMigrate INFO Extension 'citext' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:13,944 PGMigrate INFO Extension 'citext' is whitelisted in target 2022-12-10 10:40:13,945 PGMigrate INFO Extension 'citext' version '1.6' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,061 PGMigrate INFO Extension 'cube' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:14,062 PGMigrate INFO Extension 'cube' is whitelisted in target 2022-12-10 10:40:14,062 PGMigrate INFO Extension 'cube' version '1.5' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,164 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:14,165 PGMigrate INFO Extension 'fuzzystrmatch' is whitelisted in target 2022-12-10 10:40:14,165 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,270 PGMigrate INFO Extension 'hstore' version '1.8' available for installation in target, source version: '1.8' 2022-12-10 10:40:14,271 PGMigrate INFO Extension 'hstore' is whitelisted in target 2022-12-10 10:40:14,271 PGMigrate INFO Extension 'hstore' version '1.8' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,376 PGMigrate INFO Extension 'intarray' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:14,376 PGMigrate INFO Extension 'intarray' is whitelisted in target 2022-12-10 10:40:14,376 PGMigrate INFO Extension 'intarray' version '1.5' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,481 PGMigrate INFO Extension 'isn' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:14,481 PGMigrate INFO Extension 'isn' is whitelisted in target 2022-12-10 10:40:14,482 PGMigrate INFO Extension 'isn' version '1.2' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,586 PGMigrate INFO Extension 'ltree' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:14,586 PGMigrate INFO Extension 'ltree' is whitelisted in target 2022-12-10 10:40:14,586 PGMigrate INFO Extension 'ltree' version '1.2' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,688 PGMigrate INFO Extension 'pg_trgm' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:14,688 PGMigrate INFO Extension 'pg_trgm' is whitelisted in target 2022-12-10 10:40:14,688 PGMigrate INFO Extension 'pg_trgm' version '1.6' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,791 PGMigrate INFO Extension 'pgcrypto' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:14,791 PGMigrate INFO Extension 'pgcrypto' is whitelisted in target 2022-12-10 10:40:14,791 PGMigrate INFO Extension 'pgcrypto' version '1.3' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,889 PGMigrate INFO Extension 'seg' version '1.4' available for installation in target, source version: '1.4' 2022-12-10 10:40:14,889 PGMigrate INFO Extension 'seg' is trusted in target 2022-12-10 10:40:14,889 PGMigrate INFO Extension 'seg' version '1.4' will be installed in target database 'poll_service_production' 2022-12-10 10:40:14,990 PGMigrate INFO Extension 'unaccent' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:14,990 PGMigrate INFO Extension 'unaccent' is whitelisted in target 2022-12-10 10:40:14,990 PGMigrate INFO Extension 'unaccent' version '1.1' will be installed in target database 'poll_service_production' 2022-12-10 10:40:15,092 PGMigrate INFO Extension 'uuid-ossp' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:15,093 PGMigrate INFO Extension 'uuid-ossp' is whitelisted in target 2022-12-10 10:40:15,093 PGMigrate INFO Extension 'uuid-ossp' version '1.1' will be installed in target database 'poll_service_production' 2022-12-10 10:40:15,207 PGMigrate INFO Extension 'plpgsql' version '1.0' available for installation in target, source version: '1.0' 2022-12-10 10:40:15,208 PGMigrate INFO Extension 'plpgsql' is whitelisted in target 2022-12-10 10:40:15,208 PGMigrate INFO Extension 'plpgsql' version '1.0' will be installed in target database 'utility_service_production' 2022-12-10 10:40:15,316 PGMigrate INFO Extension 'btree_gin' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:15,317 PGMigrate INFO Extension 'btree_gin' is whitelisted in target 2022-12-10 10:40:15,317 PGMigrate INFO Extension 'btree_gin' version '1.3' will be installed in target database 'utility_service_production' 2022-12-10 10:40:15,425 PGMigrate INFO Extension 'btree_gist' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:15,426 PGMigrate INFO Extension 'btree_gist' is whitelisted in target 2022-12-10 10:40:15,426 PGMigrate INFO Extension 'btree_gist' version '1.6' will be installed in target database 'utility_service_production' 2022-12-10 10:40:15,528 PGMigrate INFO Extension 'citext' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:15,529 PGMigrate INFO Extension 'citext' is whitelisted in target 2022-12-10 10:40:15,529 PGMigrate INFO Extension 'citext' version '1.6' will be installed in target database 'utility_service_production' 2022-12-10 10:40:15,634 PGMigrate INFO Extension 'cube' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:15,634 PGMigrate INFO Extension 'cube' is whitelisted in target 2022-12-10 10:40:15,634 PGMigrate INFO Extension 'cube' version '1.5' will be installed in target database 'utility_service_production' 2022-12-10 10:40:15,738 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:15,738 PGMigrate INFO Extension 'fuzzystrmatch' is whitelisted in target 2022-12-10 10:40:15,738 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' will be installed in target database 'utility_service_production' 2022-12-10 10:40:15,838 PGMigrate INFO Extension 'hstore' version '1.8' available for installation in target, source version: '1.8' 2022-12-10 10:40:15,838 PGMigrate INFO Extension 'hstore' is whitelisted in target 2022-12-10 10:40:15,838 PGMigrate INFO Extension 'hstore' version '1.8' will be installed in target database 'utility_service_production' 2022-12-10 10:40:15,943 PGMigrate INFO Extension 'intarray' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:15,943 PGMigrate INFO Extension 'intarray' is whitelisted in target 2022-12-10 10:40:15,943 PGMigrate INFO Extension 'intarray' version '1.5' will be installed in target database 'utility_service_production' 2022-12-10 10:40:16,049 PGMigrate INFO Extension 'isn' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:16,049 PGMigrate INFO Extension 'isn' is whitelisted in target 2022-12-10 10:40:16,049 PGMigrate INFO Extension 'isn' version '1.2' will be installed in target database 'utility_service_production' 2022-12-10 10:40:16,153 PGMigrate INFO Extension 'ltree' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:16,154 PGMigrate INFO Extension 'ltree' is whitelisted in target 2022-12-10 10:40:16,154 PGMigrate INFO Extension 'ltree' version '1.2' will be installed in target database 'utility_service_production' 2022-12-10 10:40:16,257 PGMigrate INFO Extension 'pg_trgm' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:16,257 PGMigrate INFO Extension 'pg_trgm' is whitelisted in target 2022-12-10 10:40:16,258 PGMigrate INFO Extension 'pg_trgm' version '1.6' will be installed in target database 'utility_service_production' 2022-12-10 10:40:16,358 PGMigrate INFO Extension 'pgcrypto' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:16,358 PGMigrate INFO Extension 'pgcrypto' is whitelisted in target 2022-12-10 10:40:16,359 PGMigrate INFO Extension 'pgcrypto' version '1.3' will be installed in target database 'utility_service_production' 2022-12-10 10:40:16,463 PGMigrate INFO Extension 'seg' version '1.4' available for installation in target, source version: '1.4' 2022-12-10 10:40:16,463 PGMigrate INFO Extension 'seg' is trusted in target 2022-12-10 10:40:16,463 PGMigrate INFO Extension 'seg' version '1.4' will be installed in target database 'utility_service_production' 2022-12-10 10:40:16,568 PGMigrate INFO Extension 'unaccent' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:16,568 PGMigrate INFO Extension 'unaccent' is whitelisted in target 2022-12-10 10:40:16,568 PGMigrate INFO Extension 'unaccent' version '1.1' will be installed in target database 'utility_service_production' 2022-12-10 10:40:16,672 PGMigrate INFO Extension 'uuid-ossp' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:16,672 PGMigrate INFO Extension 'uuid-ossp' is whitelisted in target 2022-12-10 10:40:16,672 PGMigrate INFO Extension 'uuid-ossp' version '1.1' will be installed in target database 'utility_service_production' 2022-12-10 10:40:16,777 PGMigrate INFO Extension 'plpgsql' version '1.0' available for installation in target, source version: '1.0' 2022-12-10 10:40:16,778 PGMigrate INFO Extension 'plpgsql' is whitelisted in target 2022-12-10 10:40:16,778 PGMigrate INFO Extension 'plpgsql' version '1.0' will be installed in target database 'reff_service_production' 2022-12-10 10:40:16,882 PGMigrate INFO Extension 'btree_gin' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:16,882 PGMigrate INFO Extension 'btree_gin' is whitelisted in target 2022-12-10 10:40:16,882 PGMigrate INFO Extension 'btree_gin' version '1.3' will be installed in target database 'reff_service_production' 2022-12-10 10:40:16,989 PGMigrate INFO Extension 'btree_gist' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:16,989 PGMigrate INFO Extension 'btree_gist' is whitelisted in target 2022-12-10 10:40:16,989 PGMigrate INFO Extension 'btree_gist' version '1.6' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,090 PGMigrate INFO Extension 'citext' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:17,090 PGMigrate INFO Extension 'citext' is whitelisted in target 2022-12-10 10:40:17,090 PGMigrate INFO Extension 'citext' version '1.6' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,197 PGMigrate INFO Extension 'cube' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:17,197 PGMigrate INFO Extension 'cube' is whitelisted in target 2022-12-10 10:40:17,198 PGMigrate INFO Extension 'cube' version '1.5' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,303 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:17,303 PGMigrate INFO Extension 'fuzzystrmatch' is whitelisted in target 2022-12-10 10:40:17,304 PGMigrate INFO Extension 'fuzzystrmatch' version '1.1' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,409 PGMigrate INFO Extension 'hstore' version '1.8' available for installation in target, source version: '1.8' 2022-12-10 10:40:17,409 PGMigrate INFO Extension 'hstore' is whitelisted in target 2022-12-10 10:40:17,409 PGMigrate INFO Extension 'hstore' version '1.8' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,511 PGMigrate INFO Extension 'intarray' version '1.5' available for installation in target, source version: '1.5' 2022-12-10 10:40:17,512 PGMigrate INFO Extension 'intarray' is whitelisted in target 2022-12-10 10:40:17,512 PGMigrate INFO Extension 'intarray' version '1.5' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,615 PGMigrate INFO Extension 'isn' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:17,615 PGMigrate INFO Extension 'isn' is whitelisted in target 2022-12-10 10:40:17,615 PGMigrate INFO Extension 'isn' version '1.2' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,715 PGMigrate INFO Extension 'ltree' version '1.2' available for installation in target, source version: '1.2' 2022-12-10 10:40:17,715 PGMigrate INFO Extension 'ltree' is whitelisted in target 2022-12-10 10:40:17,715 PGMigrate INFO Extension 'ltree' version '1.2' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,822 PGMigrate INFO Extension 'pg_trgm' version '1.6' available for installation in target, source version: '1.6' 2022-12-10 10:40:17,822 PGMigrate INFO Extension 'pg_trgm' is whitelisted in target 2022-12-10 10:40:17,823 PGMigrate INFO Extension 'pg_trgm' version '1.6' will be installed in target database 'reff_service_production' 2022-12-10 10:40:17,930 PGMigrate INFO Extension 'pgcrypto' version '1.3' available for installation in target, source version: '1.3' 2022-12-10 10:40:17,930 PGMigrate INFO Extension 'pgcrypto' is whitelisted in target 2022-12-10 10:40:17,930 PGMigrate INFO Extension 'pgcrypto' version '1.3' will be installed in target database 'reff_service_production' 2022-12-10 10:40:18,039 PGMigrate INFO Extension 'seg' version '1.4' available for installation in target, source version: '1.4' 2022-12-10 10:40:18,039 PGMigrate INFO Extension 'seg' is trusted in target 2022-12-10 10:40:18,039 PGMigrate INFO Extension 'seg' version '1.4' will be installed in target database 'reff_service_production' 2022-12-10 10:40:18,142 PGMigrate INFO Extension 'unaccent' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:18,142 PGMigrate INFO Extension 'unaccent' is whitelisted in target 2022-12-10 10:40:18,142 PGMigrate INFO Extension 'unaccent' version '1.1' will be installed in target database 'reff_service_production' 2022-12-10 10:40:18,246 PGMigrate INFO Extension 'uuid-ossp' version '1.1' available for installation in target, source version: '1.1' 2022-12-10 10:40:18,246 PGMigrate INFO Extension 'uuid-ossp' is whitelisted in target 2022-12-10 10:40:18,246 PGMigrate INFO Extension 'uuid-ossp' version '1.1' will be installed in target database 'reff_service_production' 2022-12-10 10:40:18,353 PGMigrate INFO Extension 'plpgsql' version '1.0' available for installation in target, source version: '1.0' 2022-12-10 10:40:18,353 PGMigrate INFO Extension 'plpgsql' is whitelisted in target 2022-12-10 10:40:18,354 PGMigrate INFO Extension 'plpgsql' version '1.0' will be installed in target database 'iatros_sftp_production' 2022-12-10 10:40:18,456 PGMigrate INFO Extension 'plpgsql' version '1.0' available for installation in target, source version: '1.0' 2022-12-10 10:40:18,456 PGMigrate INFO Extension 'plpgsql' is whitelisted in target 2022-12-10 10:40:18,456 PGMigrate INFO Extension 'plpgsql' version '1.0' will be installed in target database 'randomization_production' 2022-12-10 10:40:18,736 PGMigrate INFO Logical replication available in source (14.5) 2022-12-10 10:40:18,736 PGMigrate INFO Source: max_replication_slots = 20, max_logical_replication_workers = 4, max_worker_processes = 8, wal_level = logical, max_wal_senders = 20 2022-12-10 10:40:18,736 PGMigrate INFO Target: max_replication_slots = 20, max_logical_replication_workers = 4, max_worker_processes = 8, wal_level = logical, max_wal_senders = 20 2022-12-10 10:40:18,737 PGMigrate INFO Using max 2 workers 2022-12-10 10:40:18,878 PGMigrate INFO Creating role PGRole(rolname='grafana', rolsuper=False, rolinherit=True, rolcreaterole=False, rolcreatedb=False, rolcanlogin=True, rolreplication=False, rolconnlimit=-1, rolpassword='placeholder_kqcundrqavunfmji', rolvaliduntil=None, rolbypassrls=False, rolconfig=[], safe_rolname='grafana') to target 2022-12-10 10:40:18,979 PGMigrate WARNING Role '_aiven' already exists in target 2022-12-10 10:40:18,979 PGMigrate WARNING Role 'avnadmin' already exists in target 2022-12-10 10:40:19,252 PGMigrate INFO Dumping schema from database 'defaultdb' 2022-12-10 10:40:19,353 PGMigrate INFO Dumping schema from database 'account_service_production' ERROR: database "defaultdb" already exists STATEMENT: CREATE DATABASE defaultdb WITH TEMPLATE = template0 ENCODING = 'UTF8' LOCALE = 'en_US.UTF-8'; 2022-12-10 10:40:20,367 PGTarget INFO no results to fetch 2022-12-10 10:40:20,742 PGSource INFO Creating publication 'aiven_db_migrate_defaultdb_pub' for all tables in database 'defaultdb', has_aiven_extras: False 2022-12-10 10:40:20,771 PGMigrate ERROR Encountered error: InsufficientPrivilege('must be superuser to create FOR ALL TABLES publication\n'), cleaning up 2022-12-10 10:40:20,772 PGMigrate WARNING Logical replication failed with error: 'must be superuser to create FOR ALL TABLES publication', fallback to dump 2022-12-10 10:40:20,772 PGMigrate INFO Dumping data from database 'defaultdb' 2022-12-10 10:40:21,057 PGMigrate INFO Dumping schema from database 'content_service_production' ERROR: must be owner of extension btree_gin STATEMENT: COMMENT ON EXTENSION btree_gin IS 'support for indexing common datatypes in GIN'; ERROR: must be owner of extension btree_gist STATEMENT: COMMENT ON EXTENSION btree_gist IS 'support for indexing common datatypes in GiST'; ERROR: must be owner of extension citext STATEMENT: COMMENT ON EXTENSION citext IS 'data type for case-insensitive character strings'; ERROR: must be owner of extension cube STATEMENT: COMMENT ON EXTENSION cube IS 'data type for multidimensional cubes'; ERROR: must be owner of extension fuzzystrmatch STATEMENT: COMMENT ON EXTENSION fuzzystrmatch IS 'determine similarities and distance between strings'; ERROR: must be owner of extension hstore STATEMENT: COMMENT ON EXTENSION hstore IS 'data type for storing sets of (key, value) pairs'; ERROR: must be owner of extension intarray STATEMENT: COMMENT ON EXTENSION intarray IS 'functions, operators, and index support for 1-D arrays of integers'; ERROR: must be owner of extension isn STATEMENT: COMMENT ON EXTENSION isn IS 'data types for international product numbering standards'; ERROR: must be owner of extension ltree STATEMENT: COMMENT ON EXTENSION ltree IS 'data type for hierarchical tree-like structures'; ERROR: must be owner of extension pg_trgm STATEMENT: COMMENT ON EXTENSION pg_trgm IS 'text similarity measurement and index searching based on trigrams'; ERROR: must be owner of extension pgcrypto STATEMENT: COMMENT ON EXTENSION pgcrypto IS 'cryptographic functions'; ERROR: must be owner of extension unaccent STATEMENT: COMMENT ON EXTENSION unaccent IS 'text search dictionary that removes accents'; ERROR: must be owner of extension uuid-ossp STATEMENT: COMMENT ON EXTENSION "uuid-ossp" IS 'generate universally unique identifiers (UUIDs)'; ERROR: must be owner of extension btree_gin STATEMENT: COMMENT ON EXTENSION btree_gin IS 'support for indexing common datatypes in GIN'; ERROR: must be owner of extension btree_gist STATEMENT: COMMENT ON EXTENSION btree_gist IS 'support for indexing common datatypes in GiST'; ERROR: must be owner of extension citext STATEMENT: COMMENT ON EXTENSION citext IS 'data type for case-insensitive character strings'; ERROR: must be owner of extension cube STATEMENT: COMMENT ON EXTENSION cube IS 'data type for multidimensional cubes'; ERROR: must be owner of extension fuzzystrmatch STATEMENT: COMMENT ON EXTENSION fuzzystrmatch IS 'determine similarities and distance between strings'; ERROR: must be owner of extension hstore STATEMENT: COMMENT ON EXTENSION hstore IS 'data type for storing sets of (key, value) pairs'; ERROR: must be owner of extension intarray STATEMENT: COMMENT ON EXTENSION intarray IS 'functions, operators, and index support for 1-D arrays of integers'; ERROR: must be owner of extension isn STATEMENT: COMMENT ON EXTENSION isn IS 'data types for international product numbering standards'; ERROR: must be owner of extension ltree STATEMENT: COMMENT ON EXTENSION ltree IS 'data type for hierarchical tree-like structures'; ERROR: must be owner of extension pg_trgm STATEMENT: COMMENT ON EXTENSION pg_trgm IS 'text similarity measurement and index searching based on trigrams'; ERROR: must be owner of extension pgcrypto STATEMENT: COMMENT ON EXTENSION pgcrypto IS 'cryptographic functions'; ERROR: must be owner of extension unaccent STATEMENT: COMMENT ON EXTENSION unaccent IS 'text search dictionary that removes accents'; ERROR: must be owner of extension uuid-ossp STATEMENT: COMMENT ON EXTENSION "uuid-ossp" IS 'generate universally unique identifiers (UUIDs)'; 2022-12-10 10:40:28,412 PGTarget INFO no results to fetch 2022-12-10 10:40:28,805 PGSource INFO Creating publication 'aiven_db_migrate_content_service_production_pub' for all tables in database 'content_service_production', has_aiven_extras: False 2022-12-10 10:40:28,848 PGMigrate ERROR Encountered error: InsufficientPrivilege('must be superuser to create FOR ALL TABLES publication\n'), cleaning up 2022-12-10 10:40:28,848 PGMigrate WARNING Logical replication failed with error: 'must be superuser to create FOR ALL TABLES publication', fallback to dump 2022-12-10 10:40:28,848 PGMigrate INFO Dumping data from database 'content_service_production' pg_dump: warning: there are circular foreign-key constraints among these tables: pg_dump: huey_monitor_signalinfomodel pg_dump: huey_monitor_taskmodel pg_dump: You might not be able to restore the dump without using --disable-triggers or temporarily dropping the constraints. pg_dump: Consider using a full dump instead of a --data-only dump to avoid this problem. pg_dump: warning: there are circular foreign-key constraints among these tables: pg_dump: oauth2_provider_accesstoken pg_dump: oauth2_provider_refreshtoken pg_dump: You might not be able to restore the dump without using --disable-triggers or temporarily dropping the constraints. pg_dump: Consider using a full dump instead of a --data-only dump to avoid this problem. pg_dump: warning: there are circular foreign-key constraints on this table: pg_dump: huey_monitor_taskmodel pg_dump: You might not be able to restore the dump without using --disable-triggers or temporarily dropping the constraints. pg_dump: Consider using a full dump instead of a --data-only dump to avoid this problem. 2022-12-10 10:40:29,803 PGTarget INFO no results to fetch ERROR: insert or update on table "huey_monitor_signalinfomodel" violates foreign key constraint "huey_monitor_signali_task_id_68fa68bd_fk_huey_moni" DETAIL: Key (task_id)=(27282c08-9502-404d-bb97-cd3329358967) is not present in table "huey_monitor_taskmodel". STATEMENT: COPY public.huey_monitor_signalinfomodel (id, signal_name, exception_line, exception, create_dt, task_id, hostname, pid, thread, progress_count) FROM stdin; ERROR: insert or update on table "huey_monitor_taskmodel" violates foreign key constraint "huey_monitor_taskmod_state_id_b20b1eca_fk_huey_moni" DETAIL: Key (state_id)=(b0312851-15be-4cad-a8a7-7b0ff49fea45) is not present in table "huey_monitor_signalinfomodel". STATEMENT: COPY public.huey_monitor_taskmodel (create_dt, update_dt, task_id, name, state_id, parent_task_id, "desc", total, unit, unit_divisor, finished, progress_count, cumulate_progress) FROM stdin; 2022-12-10 10:40:30,210 PGSource INFO Creating publication 'aiven_db_migrate_account_service_production_pub' for all tables in database 'account_service_production', has_aiven_extras: False 2022-12-10 10:40:30,235 PGMigrate ERROR Encountered error: InsufficientPrivilege('must be superuser to create FOR ALL TABLES publication\n'), cleaning up 2022-12-10 10:40:30,236 PGMigrate WARNING Logical replication failed with error: 'must be superuser to create FOR ALL TABLES publication', fallback to dump