[Bug]: db:convert-type uses systemConfig instead of command line parameters for connection to new target database about server

[Infamousbugg]

I also have a migration on hold waiting for this issue to be resolved.

[juliushaertl]

Possible patch candidate, but haven't had time to verify and test this yet:

diff --git a/core/Command/Db/ConvertType.php b/core/Command/Db/ConvertType.php
index 333f29625f6..078fd47965b 100644
--- a/core/Command/Db/ConvertType.php
+++ b/core/Command/Db/ConvertType.php
@@ -227,15 +227,22 @@ class ConvertType extends Command implements CompletionAwareInterface {
        protected function getToDBConnection(InputInterface $input, OutputInterface $output) {
                $type = $input->getArgument('type');
                $connectionParams = $this->connectionFactory->createConnectionParams();
-               $connectionParams = array_merge($connectionParams, [
+
+               $overrideConnectionParams = [
                        'host' => $input->getArgument('hostname'),
                        'user' => $input->getArgument('username'),
                        'password' => $input->getOption('password'),
                        'dbname' => $input->getArgument('database'),
-               ]);
+               ];
                if ($input->getOption('port')) {
-                       $connectionParams['port'] = $input->getOption('port');
+                       $overrideConnectionParams['port'] = $input->getOption('port');
                }
+
+               $connectionParams = array_merge($connectionParams, [
+                       'primary' => $overrideConnectionParams,
+                       'replica' => [$overrideConnectionParams],
+               ]);
+
                return $this->connectionFactory->getConnection($type, $connectionParams);
        }

[jiriks74]

@szaimen I'll be probably playing around with a new (old) server this evening. I'll back up my database and try the patch out. I'll know practically immediately as I get the duplicate entry error right at the start if it tries to write into the original DB.

[zackgomez]

Any progress?

Is the current guidance to create a fresh system losing an existing data?

[joshtrichards]

Yes, unfortunately. 😢 The end result is the parameters get swapped so it likely dropped the source database.

Fortunately, recovery should be a simple matter of restoring from backup (just the db portion): https://docs.nextcloud.com/server/latest/admin_manual/maintenance/restore.html#restore-database (or whatever other mechanism you utilize in your environment for database backup/recovery)

[joshtrichards]

@Delagen Mind pushing that as PR? I know there may be additional changes needed, but it'll start things in motion.

[Delagen]

@joshtrichards I don't mind that my changes solve the problem correctly. It only makes possible for me successful conversion from MySQL to Postgres

[rotdrop]

Would not be it the simplest solution, if the ConnectionFactory::getConnection() method would first check if the $additionalConnectionParams argument contains explicit connection parameters and in that case simply skip that fancy primary / replica stuff? At least for the case were the explicitly specified connection differs from the "internal" database connection configured in config/config.php. In principle this should restore the previous behaviour when using the factory for alternate database connections.

[szaimen]

Hi @intersectRaven @ksmonkey123 @Delagen @HammyHavoc @chimpboy @rotdrop are you able to test the above patch? Thanks in advance! :)

[jiriks74]

The conversion script gets stuck on this line

server/core/Command/Db/ConvertType.php

Line 210 in 34f5cba

$toMS->migrate($currentMigration);

[ksmonkey123]

Logging the Parameter $additionalConnectionParams in ConnectionFactory.php on Line 166 shows the following values for the "fromDB" and the "toDB":

FromDB

Array
(
    [adapter] => OC\DB\AdapterMySQL
    [charset] => utf8mb4
    [driver] => pdo_mysql
    [wrapperClass] => OC\DB\Connection
    [driverOptions] => Array
        (
            [1005] => 1
        )

    [user] => nextcloud
    [password] => *************
    [host] => nextcloud-db
    [dbname] => nextcloud
    [tablePrefix] => oc_
    [sqlite.journal_mode] => WAL
    [defaultTableOptions] => Array
        (
            [collate] => utf8mb4_bin
            [charset] => utf8mb4
            [tablePrefix] => oc_
        )

    [primary] => Array
        (
            [adapter] => OC\DB\AdapterMySQL
            [charset] => utf8mb4
            [driver] => pdo_mysql
            [wrapperClass] => OC\DB\Connection
            [driverOptions] => Array
                (
                    [1005] => 1
                )

            [user] => nextcloud
            [password] => ****************
            [host] => nextcloud-db
            [dbname] => nextcloud
            [tablePrefix] => oc_
            [sqlite.journal_mode] => WAL
            [defaultTableOptions] => Array
                (
                    [collate] => utf8mb4_bin
                    [charset] => utf8mb4
                    [tablePrefix] => oc_
                )

        )

    [replica] => Array
        (
            [0] => Array
                (
                    [adapter] => OC\DB\AdapterMySQL
                    [charset] => utf8mb4
                    [driver] => pdo_mysql
                    [wrapperClass] => OC\DB\Connection
                    [driverOptions] => Array
                        (
                            [1005] => 1
                        )

                    [user] => nextcloud
                    [password] => *************
                    [host] => nextcloud-db
                    [dbname] => nextcloud
                    [tablePrefix] => oc_
                    [sqlite.journal_mode] => WAL
                    [defaultTableOptions] => Array
                        (
                            [collate] => utf8mb4_bin
                            [charset] => utf8mb4
                            [tablePrefix] => oc_
                        )

                )

        )

)

ToDB

Array
(
    [adapter] => OC\DB\AdapterMySQL
    [charset] => utf8mb4
    [driver] => pdo_mysql
    [wrapperClass] => OC\DB\Connection
    [driverOptions] => Array
        (
            [1005] => 1
        )

    [user] => nextcloud_app
    [password] => ***************
    [host] => postgres
    [dbname] => nextcloud
    [tablePrefix] => oc_
    [sqlite.journal_mode] => WAL
    [defaultTableOptions] => Array
        (
            [collate] => utf8mb4_bin
            [charset] => utf8mb4
            [tablePrefix] => oc_
        )

    [primary] => Array
        (
            [adapter] => OC\DB\AdapterMySQL
            [charset] => utf8mb4
            [driver] => pdo_mysql
            [wrapperClass] => OC\DB\Connection
            [driverOptions] => Array
                (
                    [1005] => 1
                )

            [user] => nextcloud
            [password] => *************
            [host] => nextcloud-db
            [dbname] => nextcloud
            [tablePrefix] => oc_
            [sqlite.journal_mode] => WAL
            [defaultTableOptions] => Array
                (
                    [collate] => utf8mb4_bin
                    [charset] => utf8mb4
                    [tablePrefix] => oc_
                )

        )

    [replica] => Array
        (
            [0] => Array
                (
                    [adapter] => OC\DB\AdapterMySQL
                    [charset] => utf8mb4
                    [driver] => pdo_mysql
                    [wrapperClass] => OC\DB\Connection
                    [driverOptions] => Array
                        (
                            [1005] => 1
                        )

                    [user] => nextcloud
                    [password] => **********
                    [host] => nextcloud-db
                    [dbname] => nextcloud
                    [tablePrefix] => oc_
                    [sqlite.journal_mode] => WAL
                    [defaultTableOptions] => Array
                        (
                            [collate] => utf8mb4_bin
                            [charset] => utf8mb4
                            [tablePrefix] => oc_
                        )

                )

        )

    [port] => 5432
)

It is notable that in the second case (which should be the postgres-database configured via the CLI arguments), the host, dbname and port in the top-level array are set correctly from the CLI, Both the primary and the replica fields and even some top-level fields obviously use the DB-configs from config.php

[joshtrichards]

Possibly related: #45097

I don't have time to look tonight, but your analysis seems sound (and, given the added evidence from the log output, even more so).

Cc: @juliushaertl

[intersectRaven]

I can verify but can't contribute logs. I used the --clear-schema option in my attempt and my original MariaDB DB got deleted. Good thing I have backups.

[Delagen]

I modified https://github.com/nextcloud/server/blob/master/core/Command/Db/ConvertType.php#L247
to

$connectionParams = $this->connectionFactory->getDefaultConnectionParams($type);

and seems all converted

Also have to remove mysql.utf8mb4 parameter from config.php to perform conversion to remove client_encoding from params of DbFactory

[HammyHavoc]

Got hit by this a week ago. Is that particular Nextcloud instance completely screwed?

[chimpboy]

I have just experienced this problem on a new installation.

[szaimen]

Hi, is this #45013 fixing the problem?

[jiriks74]

I just looked into it a bit and so far it looks like the databases get mixed up somewhere in the script.

When I hardcoded the toDB values it tried to connect to the target database (PostgreSQL) with the credentials of my current database (MariaDB/MySQL). It used the username from the old database on the new one.

[jiriks74]

I tried it out, it doesn't work. I get stuck on Creating schema in new database even after hardcoding the connections for my databases to be sure that it's passed right. I'll try to find out where the issue is.

[leon1995]

Are there any new information on this?