ihanick / anydbver Goto Github PK
View Code? Open in Web Editor NEWLXD+Ansible setup to install Percona/Oracle MySQL/MongoDB/Postgresql database products with exact version specified.
License: MIT License
anydbver's People
Contributors
Stargazers
Watchers
Forkers
simhaonline lazzyfu ctrl4 puneet0191 85stallions anton-bystrov michaelcoburn guriandoro rkd-learn codeforall ylacancellera venkatesh-prasad-vanydbver's Issues
Please add option to include pgbouncer instance in PostgreSQL replication and Patroni cluster setup.
Please add an option to include a pgbouncer instance in PostgreSQL replication and Patroni cluster setup.
Thank you.
Allow deploy of pmm-server/client from perconalab
Hi,
For testing purposes, we create feature builds of PMM server and client and we run the server like this:
#!/bin/bash
IMAGE="${1:-perconalab/pmm-server:dev-latest}"
docker create -v /srv --name pmm-data ${IMAGE} /bin/true
docker run -d \
-p 80:80 \
-p 443:443 \
--volumes-from pmm-data \
--name pmm-server \
-e PERCONA_TEST_DBAAS=1 \
-e PERCONA_TEST_VERSION_SERVICE_URL=https://check-dev.percona.com/versions/v1 \
${IMAGE}
It would be nice if there is a way to use the perconalab docker images instead of percona so we can test PMM inside anydbver.
./anydbver fix ip
when restarting the db containers created with anydbver might have different IP. in that case,, need to sync up the new IP manually with ansible_host / ssh_config to log with ./anydbver ssh node1 command
$ ./anydbver ssh default
No such host, try to deploy or add it
$
$ ./anydbver ssh node1
No such host, try to deploy or add it
So an automatic fix from ./anydbver side kie fix ip should help here
Dependency conflicts
./anydbver deploy mysql:8.0.18 group-replication \
node1 hn:mysql_rs0_gr1 mysql:8.0.18 group-replication master:default \
node2 hn:mysql_rs0_gr2 mysql:8.0.18 group-replication master:default \
node3 hn:mysql_rs0_gr3 mysql:8.0.18 group-replication master:default \
node4 hn:mysql_rs0_router mysql:8.0.18 master:default
Results in dependency conflicts:
Error: mysql-community-client-plugins conflicts with mysql-community-client-8.0.18-1.el7.x86_64
", "rc": 1, "results": ["Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.cc.columbia.edu
* extras: repos-va.psychz.net
* updates: linux-mirrors.fnal.gov
Resolving Dependencies
--> Running transaction check
---> Package mysql-community-client.x86_64 0:8.0.18-1.el7 will be installed
--> Processing Dependency: mysql-community-libs(x86-64) >= 8.0.11 for package: mysql-community-client-8.0.18-1.el7.x86_64
--> Running transaction check
---> Package mysql-community-libs.x86_64 0:8.0.23-1.el7 will be installed
--> Processing Dependency: mysql-community-client-plugins = 8.0.23-1.el7 for package: mysql-community-libs-8.0.23-1.el7.x86_64
--> Running transaction check
---> Package mysql-community-client-plugins.x86_64 0:8.0.23-1.el7 will be installed
--> Processing Conflict: mysql-community-client-plugins-8.0.23-1.el7.x86_64 conflicts mysql-community-client < 8.0.22
--> Finished Dependency Resolution
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
"]}
failed: [marcos.albe.default] (item=mysql-community-server-8.0.18-1.el7.x86_64) => {"ansible_loop_var": "item", "changed": false, "changes": {"installed": ["mysql-community-server-8.0.18-1.el7.x86_64"]}, "item": "mysql-community-server-8.0.18-1.el7.x86_64", "msg": "Error: mysql-community-client-plugins conflicts with mysql-community-server-8.0.18-1.el7.x86_64
"
Full output
[COR-SER] [marcos.albe@tp-support03 malbe-anydbver]$ ./anydbver deploy mysql:8.0.18 group-replication node1 hn:mysql_rs0_gr1 mysql:8.0.18 group-replication master:default node2 hn:mysql_rs0_gr2 mysql:8.0.18 group-replication master:default node3 hn:mysql_rs0_gr3 mysql:8.0.18 group-replication master:default node4 hn:mysql_rs0_router mysql:8.0.18 master:default
./lxdctl --nodes 5 --hostname node1=mysql_rs0_gr1 --hostname node2=mysql_rs0_gr2 --hostname node3=mysql_rs0_gr3 --hostname node4=mysql_rs0_router
Creating marcos-albe-default
Starting marcos-albe-default
Package iproute-4.11.0-30.el7.x86_64 already installed and latest version
Creating marcos-albe-node1
Starting marcos-albe-node1
Package iproute-4.11.0-30.el7.x86_64 already installed and latest version
Creating marcos-albe-node2
Starting marcos-albe-node2
Package iproute-4.11.0-30.el7.x86_64 already installed and latest version
Creating marcos-albe-node3
Starting marcos-albe-node3
Package iproute-4.11.0-30.el7.x86_64 already installed and latest version
Creating marcos-albe-node4
Starting marcos-albe-node4
Package iproute-4.11.0-30.el7.x86_64 already installed and latest version
MYSQL=8.0.18-1 REPLICATION_TYPE=group DB_USER=root DB_PASS=secret START=1 ansible-playbook -i ansible_hosts --limit marcos.albe.default playbook.yml
PLAY [install percona server] ****************************************************************************************************************************************************************
TASK [Gathering Facts] ***********************************************************************************************************************************************************************
ok: [marcos.albe.default]
TASK [Include package descriptions for each OS] **********************************************************************************************************************************************
ok: [marcos.albe.default]
TASK [vagrant compat] ************************************************************************************************************************************************************************
ok: [marcos.albe.default]
TASK [Check if /vagrant directory is mounted] ************************************************************************************************************************************************
ok: [marcos.albe.default]
TASK [sync configs and tools] ****************************************************************************************************************************************************************
included: /home/marcos.albe/malbe-anydbver/common/tasks/copy_files.yaml for marcos.albe.default
TASK [file] **********************************************************************************************************************************************************************************
changed: [marcos.albe.default]
TASK [synchronize] ***************************************************************************************************************************************************************************
changed: [marcos.albe.default]
TASK [synchronize] ***************************************************************************************************************************************************************************
changed: [marcos.albe.default]
TASK [synchronize] ***************************************************************************************************************************************************************************
changed: [marcos.albe.default]
TASK [package] *******************************************************************************************************************************************************************************
ok: [marcos.albe.default]
TASK [MariaDB Role] **************************************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [install percona yum repo] **************************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [install unlisted deps for percona release] *********************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [install percona-release.deb] ***********************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [refresh apt repo] **********************************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [enable Percona MySQL 8.0 repo] *********************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [MySQL 8.0 yum repo] ********************************************************************************************************************************************************************
changed: [marcos.albe.default]
TASK [Switch mysql repo 5.7] *****************************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [Switch mysql repo 5.6] *****************************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [Disable Redhat MySQL module] ***********************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [Enable EPEL] ***************************************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [Install symas.com repo for openldap-servers] *******************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [Postgresql Role] ***********************************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [MongoDB Role] **************************************************************************************************************************************************************************
skipping: [marcos.albe.default]
TASK [Install db packages] *******************************************************************************************************************************************************************
included: /home/marcos.albe/malbe-anydbver/common/tasks/install_db_packages.yml for marcos.albe.default
TASK [install apt db packages] ***************************************************************************************************************************************************************
skipping: [marcos.albe.default] => (item=mysql-community-common-8.0.18-1.el7.x86_64)
skipping: [marcos.albe.default] => (item=mysql-community-client-8.0.18-1.el7.x86_64)
skipping: [marcos.albe.default] => (item=mysql-community-server-8.0.18-1.el7.x86_64)
TASK [install db packages] *******************************************************************************************************************************************************************
changed: [marcos.albe.default] => (item=mysql-community-common-8.0.18-1.el7.x86_64)
failed: [marcos.albe.default] (item=mysql-community-client-8.0.18-1.el7.x86_64) => {"ansible_loop_var": "item", "changed": false, "changes": {"installed": ["mysql-community-client-8.0.18-1.el7.x86_64"]}, "item": "mysql-community-client-8.0.18-1.el7.x86_64", "msg": "Error: mysql-community-client-plugins conflicts with mysql-community-client-8.0.18-1.el7.x86_64\n", "rc": 1, "results": ["Loaded plugins: fastestmirror\nLoading mirror speeds from cached hostfile\n * base: mirror.cc.columbia.edu\n * extras: repos-va.psychz.net\n * updates: linux-mirrors.fnal.gov\nResolving Dependencies\n--> Running transaction check\n---> Package mysql-community-client.x86_64 0:8.0.18-1.el7 will be installed\n--> Processing Dependency: mysql-community-libs(x86-64) >= 8.0.11 for package: mysql-community-client-8.0.18-1.el7.x86_64\n--> Running transaction check\n---> Package mysql-community-libs.x86_64 0:8.0.23-1.el7 will be installed\n--> Processing Dependency: mysql-community-client-plugins = 8.0.23-1.el7 for package: mysql-community-libs-8.0.23-1.el7.x86_64\n--> Running transaction check\n---> Package mysql-community-client-plugins.x86_64 0:8.0.23-1.el7 will be installed\n--> Processing Conflict: mysql-community-client-plugins-8.0.23-1.el7.x86_64 conflicts mysql-community-client < 8.0.22\n--> Finished Dependency Resolution\n You could try using --skip-broken to work around the problem\n You could try running: rpm -Va --nofiles --nodigest\n"]}
failed: [marcos.albe.default] (item=mysql-community-server-8.0.18-1.el7.x86_64) => {"ansible_loop_var": "item", "changed": false, "changes": {"installed": ["mysql-community-server-8.0.18-1.el7.x86_64"]}, "item": "mysql-community-server-8.0.18-1.el7.x86_64", "msg": "Error: mysql-community-client-plugins conflicts with mysql-community-server-8.0.18-1.el7.x86_64\n", "rc": 1, "results": ["Loaded plugins: fastestmirror\nLoading mirror speeds from cached hostfile\n * base: mirror.cc.columbia.edu\n * extras: repos-va.psychz.net\n * updates: linux-mirrors.fnal.gov\nResolving Dependencies\n--> Running transaction check\n---> Package mysql-community-server.x86_64 0:8.0.18-1.el7 will be installed\n--> Processing Dependency: mysql-community-client(x86-64) >= 8.0.11 for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: /usr/bin/perl for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: libaio.so.1(LIBAIO_0.1)(64bit) for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: libaio.so.1(LIBAIO_0.4)(64bit) for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: libnuma.so.1(libnuma_1.1)(64bit) for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: libnuma.so.1(libnuma_1.2)(64bit) for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: net-tools for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: perl(Getopt::Long) for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: perl(strict) for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: libaio.so.1()(64bit) for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Processing Dependency: libnuma.so.1()(64bit) for package: mysql-community-server-8.0.18-1.el7.x86_64\n--> Running transaction check\n---> Package libaio.x86_64 0:0.3.109-13.el7 will be installed\n---> Package mysql-community-client.x86_64 0:8.0.23-1.el7 will be installed\n--> Processing Dependency: mysql-community-client-plugins = 8.0.23-1.el7 for package: mysql-community-client-8.0.23-1.el7.x86_64\n--> Processing Dependency: mysql-community-libs(x86-64) >= 8.0.11 for package: mysql-community-client-8.0.23-1.el7.x86_64\n---> Package net-tools.x86_64 0:2.0-0.25.20131004git.el7 will be installed\n---> Package numactl-libs.x86_64 0:2.0.12-5.el7 will be installed\n---> Package perl.x86_64 4:5.16.3-297.el7 will be installed\n--> Processing Dependency: perl-libs = 4:5.16.3-297.el7 for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Socket) >= 1.3 for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Scalar::Util) >= 1.10 for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl-macros for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl-libs for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(threads::shared) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(threads) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(constant) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Time::Local) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Time::HiRes) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Storable) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Socket) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Scalar::Util) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Pod::Simple::XHTML) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Pod::Simple::Search) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Filter::Util::Call) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(File::Temp) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(File::Spec::Unix) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(File::Spec::Functions) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(File::Spec) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(File::Path) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Exporter) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Cwd) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: perl(Carp) for package: 4:perl-5.16.3-297.el7.x86_64\n--> Processing Dependency: libperl.so()(64bit) for package: 4:perl-5.16.3-297.el7.x86_64\n---> Package perl-Getopt-Long.noarch 0:2.40-3.el7 will be installed\n--> Processing Dependency: perl(Pod::Usage) >= 1.14 for package: perl-Getopt-Long-2.40-3.el7.noarch\n--> Processing Dependency: perl(Text::ParseWords) for package: perl-Getopt-Long-2.40-3.el7.noarch\n--> Running transaction check\n---> Package mysql-community-client-plugins.x86_64 0:8.0.23-1.el7 will be installed\n---> Package mysql-community-libs.x86_64 0:8.0.23-1.el7 will be installed\n---> Package perl-Carp.noarch 0:1.26-244.el7 will be installed\n---> Package perl-Exporter.noarch 0:5.68-3.el7 will be installed\n---> Package perl-File-Path.noarch 0:2.09-2.el7 will be installed\n---> Package perl-File-Temp.noarch 0:0.23.01-3.el7 will be installed\n---> Package perl-Filter.x86_64 0:1.49-3.el7 will be installed\n---> Package perl-PathTools.x86_64 0:3.40-5.el7 will be installed\n---> Package perl-Pod-Simple.noarch 1:3.28-4.el7 will be installed\n--> Processing Dependency: perl(Pod::Escapes) >= 1.04 for package: 1:perl-Pod-Simple-3.28-4.el7.noarch\n--> Processing Dependency: perl(Encode) for package: 1:perl-Pod-Simple-3.28-4.el7.noarch\n---> Package perl-Pod-Usage.noarch 0:1.63-3.el7 will be installed\n--> Processing Dependency: perl(Pod::Text) >= 3.15 for package: perl-Pod-Usage-1.63-3.el7.noarch\n--> Processing Dependency: perl-Pod-Perldoc for package: perl-Pod-Usage-1.63-3.el7.noarch\n---> Package perl-Scalar-List-Utils.x86_64 0:1.27-248.el7 will be installed\n---> Package perl-Socket.x86_64 0:2.010-5.el7 will be installed\n---> Package perl-Storable.x86_64 0:2.45-3.el7 will be installed\n---> Package perl-Text-ParseWords.noarch 0:3.29-4.el7 will be installed\n---> Package perl-Time-HiRes.x86_64 4:1.9725-3.el7 will be installed\n---> Package perl-Time-Local.noarch 0:1.2300-2.el7 will be installed\n---> Package perl-constant.noarch 0:1.27-2.el7 will be installed\n---> Package perl-libs.x86_64 4:5.16.3-297.el7 will be installed\n---> Package perl-macros.x86_64 4:5.16.3-297.el7 will be installed\n---> Package perl-threads.x86_64 0:1.87-4.el7 will be installed\n---> Package perl-threads-shared.x86_64 0:1.43-6.el7 will be installed\n--> Running transaction check\n---> Package perl-Encode.x86_64 0:2.51-7.el7 will be installed\n---> Package perl-Pod-Escapes.noarch 1:1.04-297.el7 will be installed\n---> Package perl-Pod-Perldoc.noarch 0:3.20-4.el7 will be installed\n--> Processing Dependency: perl(parent) for package: perl-Pod-Perldoc-3.20-4.el7.noarch\n--> Processing Dependency: perl(HTTP::Tiny) for package: perl-Pod-Perldoc-3.20-4.el7.noarch\n--> Processing Dependency: groff-base for package: perl-Pod-Perldoc-3.20-4.el7.noarch\n---> Package perl-podlators.noarch 0:2.5.1-3.el7 will be installed\n--> Running transaction check\n---> Package groff-base.x86_64 0:1.22.2-8.el7 will be installed\n---> Package perl-HTTP-Tiny.noarch 0:0.033-3.el7 will be installed\n---> Package perl-parent.noarch 1:0.225-244.el7 will be installed\n--> Processing Conflict: mysql-community-client-plugins-8.0.23-1.el7.x86_64 conflicts mysql-community-server < 8.0.21\n--> Finished Dependency Resolution\n You could try using --skip-broken to work around the problem\n You could try running: rpm -Va --nofiles --nodigest\n"]}
changed: [marcos.albe.default] => (item=https://dev.mysql.com/get/Downloads/MySQL-Shell/mysql-shell-8.0.18-1.el7.x86_64.rpm)
Add Bind DNS
If possible,
- Add Bind DNS node that has a caching DNS server configured
- A master zone and reverse zone that contain the hostname of every node installed
- The resolv.conf of every node points to the DNS node
Error on deploy mariadb-cluster:10.3.26
Hi Nickolay,
I tried to deploy the Maria Cluster:
./anydbver deploy mariadb-cluster:10.3.26
But it is throwing this error:
fatal: [vinicius.grippa.default]: FAILED! => {"changed": false, "msg": "Failure downloading https://archive.mariadb.org/mariadb-10.3.26/yum/centos7-amd64/rpms/galera-25.3.31-1.rhel7.el7.centos.x86_64.rpm, HTTP Error 404: Not Found"}
Checking the repo and the only available for 25.3.31 is galera-25.3.31-1.el7.centos.x86_64.rpm.
Vanilla mysql version in el7 is only recognized up to 8.0.21
Hello,
Thanks for creating a great tool.
I tried to deploy with the latest version of Vanilla mysql, but I got the following error.
$ ./anydbver deploy mysql:8.0.25 xtrabackup
No such version mysql for el7: 'mysql:8.0.25'
$ ./anydbver deploy mysql:8.0.24 xtrabackup
No such version mysql for el7: 'mysql:8.0.24'
I see, this is how the script gets the version information from the Yum repository.
curl -s \
http://repo.mysql.com/yum/mysql-5.6-community/el/7/x86_64/ \
http://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/ \
http://repo.mysql.com/yum/mysql-8.0-community/el/7/x86_64/ \
| sed -rne 's/^.*HREF="mysql-community-server-([0-9].*)\.el.\.x86_64\.rpm".*$/\1/p' \
| sort -t. -k 1,1n -k 2,2n -k 3,3n \
> .version-info/mysql.el7.txt
When I access the URL of the repository, the page does indeed show only up to 8.0.21.
http://repo.mysql.com/yum/mysql-8.0-community/el/7/x86_64/
In the reference manual, it was an HTTPS URL.
https://dev.mysql.com/doc/mysql-yum-repo-quick-guide/en/#repo-enable-and-disable-sub-repositories
If I access the site via HTTPS, I see the version 8.0.22 or later.
https://repo.mysql.com/yum/mysql-8.0-community/el/7/x86_64/
[FILE] mysql-community-client-8.0.22-..> 25-Sep-2020 08:58 48.2M
[FILE] mysql-community-client-8.0.22-..> 25-Sep-2020 08:58 47.5M
[FILE] mysql-community-client-8.0.23-..> 16-Dec-2020 09:14 48.4M
[FILE] mysql-community-client-8.0.23-..> 16-Dec-2020 09:14 47.8M
[FILE] mysql-community-client-8.0.24-..> 23-Mar-2021 11:53 46.0M
[FILE] mysql-community-client-8.0.24-..> 23-Mar-2021 11:53 45.5M
[FILE] mysql-community-client-8.0.25-..> 26-Apr-2021 07:36 46.1M
[FILE] mysql-community-client-8.0.25-..> 26-Apr-2021 07:36 45.6M
I actually changed it to HTTPS and the deployment went well.
$ ./anydbver update
$ tail .version-info/mysql.el7.txt
8.0.16-2
8.0.17-1
8.0.18-1
8.0.19-1
8.0.20-1
8.0.21-1
8.0.22-1
8.0.23-1
8.0.24-1
8.0.25-1
$ ./anydbver deploy mysql:8.0.25
(...)
$ ./anydbver ssh default 'mysqld --version'
/usr/sbin/mysqld Ver 8.0.25 for Linux on x86_64 (MySQL Community Server - GPL)
sysbench for postgres needs postgres libs >= 10
When we create a pg setup with a "dedicated app" node like
./anydbver deploy \
node0 pg:13 \
node1 sysbench sysbench-pg:node0 oltp_read_write \
sysbench will need to connect remotely and fails with
FATAL: Connection to database failed: SCRAM authentication requires libpq version 10 or above
For now, we can work around it by using the sysbench directives in the db node itself too, like
./anydbver deploy \
node0 pg:13 sysbench sysbench-pg:node0 oltp_read_write \
node1 sysbench sysbench-pg:node0 oltp_read_write \
and then in the app node enabling percona repo for pg and installing the lib
./anydbver ssh node1
percona-release setup ppg-11
yum -y install percona-postgresql11-libs
In this way, the initialization will work per the sysbench setup in node0, and then we run it remotely from node1 with /usr/local/bin/run_sysbench.sh with a minimal amount of manual steps.
reimplement list command
With multiple namespaces it's hard to see just required containers, let's re-implement
./anydbver --namespace test list
MariaDB version could not found
Command: ./anydbver deploy maria:latest node1 maria:latest,master=node0,galera node2 maria:latest,master=node0,galera
Output:
TASK [mariadb : Package info sql] ******************************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [yunus.uyanik.default] => {
"msg": "select * from mariadb_version where version like 'latest%' and os='el8' ORDER BY CAST(SUBSTR(version,1,INSTR(version,'.')-1) as integer) DESC, CAST(SUBSTR(version,INSTR(version,'.')+1, INSTR(SUBSTR(version,INSTR(version,'.')+1),'.')-1 ) AS INTEGER) DESC, CAST( SUBSTR(version, INSTR(version,'.')+1+ INSTR(SUBSTR(version,INSTR(version,'.')+1),'.') ) AS INTEGER) DESC LIMIT 1"
}
True
replication in mysql
For MySQLreplication is not automatically configure with the deploy command
FR: add KMIP server and integration
Here's basic Docker that Marce has put together to run PyKMIP:
docker run -d --security-opt seccomp=unconfined --cap-add=NET_ADMIN --rm -p 5696:5696 --name kmip altmannmarcelo/kmip:latest
mkdir /tmp/certs
docker cp kmip:/opt/certs/root_certificate.pem /tmp/certs/
docker cp kmip:/opt/certs/client_key_jane_doe.pem /tmp/certs/
docker cp kmip:/opt/certs/client_certificate_jane_doe.pem /tmp/certs/
You can use the following configuration for component_keyring_kmip.cnf:
# component_keyring_kmip.cnf
{
"server_addr": "127.0.0.1",
"server_port": "5696",
"client_ca": "/tmp/certs/client_certificate_jane_doe.pem",
"client_key": "/tmp/certs/client_key_jane_doe.pem",
"server_ca": "/tmp/certs/root_certificate.pem"
}
Would be awesome to have this as a component for anydbver
Can you please add support for postgresql version 9.6
Can you please add support for postgresql version 9.6
Thanks
Add dedicated network interface for SSH
Hey Nickolay! Here's one more idea:
Add a second network interface to handle only the ssh traffic, so that we use iptables/qdisc to break the network connecting the services and clients , and still be able to access the nodes.
Allow to point sysbench to mysql router or proxysql
Hey Nickolay! Here's one idea that came up today while doing some tests:
Currently sysbench can point to a mysql or pg node, but it would be very useful if you coudl allow to point the sysbench to a mysql-router or proxysql node, or a pgBouncer node.
An alternative would be to allow sysbench to specify a port for the target mysql/pg node.
Another alternative would be to use port 3306 for the proxies/routers (this seems less tempting)
Add support for Hashicorp Vault
Would be amazing to have vault automagically deployed, and perhaps have integration with deployed MySQL done or printed out as the Vault is created and opened.
Generate screen/tmux session for the deployment
Hey Nickolay! Here's another idea:
At the end of deployment generate screen/tmux config lines to have a session to each deployed node, for example in my ~/.screenrc I added:
screen -t Client 0 /home/marcos.albe/anydbver/anydbver --namespace=gr ssh node0
screen -t DC1_N1 1 /home/marcos.albe/anydbver/anydbver --namespace=gr ssh node1
screen -t DC1_N2 2 /home/marcos.albe/anydbver/anydbver --namespace=gr ssh node2
screen -t DC1_N3 3 /home/marcos.albe/anydbver/anydbver --namespace=gr ssh node3
screen -t DC2_N1 4 /home/marcos.albe/anydbver/anydbver --namespace=gr ssh node4
screen -t DC2_N2 5 /home/marcos.albe/anydbver/anydbver --namespace=gr ssh node5
screen -t DC2_N3 6 /home/marcos.albe/anydbver/anydbver --namespace=gr ssh node6
For extra-coolness points it would be awesome to also have split window with tail -F <log for the service deployed in the node> in each tab.
Allow port forwarding to expose nodes
In order to be able to connect to the different nodes from outside the vagrant box, it would be useful to expose/forward the ports used by each node.
For example, if I start a 3 nodes PS cluster, anydbver reports this:
./anydbver list
+-----------------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| vagrant-default | RUNNING | 10.56.52.62 (eth0) | fd42:4c11:eee0:8c13:216:3eff:fe90:d784 (eth0) | CONTAINER | 0 |
+-----------------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| vagrant-node1 | RUNNING | 10.56.52.24 (eth0) | fd42:4c11:eee0:8c13:216:3eff:feea:352d (eth0) | CONTAINER | 0 |
+-----------------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| vagrant-node2 | RUNNING | 10.56.52.37 (eth0) | fd42:4c11:eee0:8c13:216:3eff:fe07:6b70 (eth0) | CONTAINER | 0 |
+-----------------+---------+--------------------+-----------------------------------------------+-----------+-----------+
The IP addresses 10.56.52.xxx cannot be accessed from a test running in my localhost.
It would be useful to be able to forward, for example, 10.56.52.62:3306 to the vagrant box and being able to connect to MySQL from the outside
Reimplement Patroni support in new anydbver impl
We used to be able to have Patroni deployments:
./anydbver deploy \
node0 pg:13 patroni \
node1 pg:13 master:node0 patroni etcd-ip:node0 \
node2 pg:13 master:node0 patroni etcd-ip:node0
But it seems after the migration to the new script, we are having issues:
[root@node1 ~]# patronictl -c /etc/patroni/cluster1-1.yml list
+ Cluster: stampede (7298832024534509405) ---------+----+-----------+-----------------+
| Member | Host | Role | State | TL | Lag in MB | Pending restart |
+------------+------------+---------+--------------+----+-----------+-----------------+
| cluster1-0 | 172.28.0.2 | Leader | running | 1 | | * |
| cluster1-1 | 172.28.0.3 | Replica | start failed | | unknown | |
+------------+------------+---------+--------------+----+-----------+-----------------+
The errors are due to:
[root@node1 ~]# cat /var/lib/pgsql/13/data/log/postgresql-Tue.log
...
2023-11-07 21:17:27.884 UTC [2848] ERROR: cannot execute CREATE ROLE in a read-only transaction
2023-11-07 21:17:27.884 UTC [2848] STATEMENT: CREATE USER replicator WITH REPLICATION ENCRYPTED PASSWORD 'verysecretpassword1^';
2023-11-07 21:17:28.750 UTC [2045] LOG: received fast shutdown request
2023-11-07 21:17:28.752 UTC [2045] LOG: aborting any active transactions
2023-11-07 21:17:28.752 UTC [2882] FATAL: terminating connection due to administrator command
2023-11-07 21:17:28.752 UTC [2051] FATAL: terminating walreceiver process due to administrator command
2023-11-07 21:17:28.754 UTC [2048] LOG: shutting down
2023-11-07 21:17:28.839 UTC [2045] LOG: database system is shut down
Let me know if I can help with anything.
Please (re?)add MySQL Router support
What the subject says! :) It's certainly needed
default: HTTP Error 404: Not Found http://repo.percona.com/ppg-10/yum/release/7/RPMS/x86_64 [
Encountering following error. Any idea how to fix it. Never observed before.
default: Loaded settings: {'PROVIDER': 'lxd', 'LXD_PROFILE': 'vagrant', 'K3S_FLANNEL_BACKEND': 'host-gw'}
default: Loaded settings: {'PROVIDER': 'lxd', 'LXD_PROFILE': 'vagrant', 'K3S_FLANNEL_BACKEND': 'host-gw'}
default: HTTP Error 404: Not Found http://repo.percona.com/ppg-10/yum/release/7/RPMS/x86_64 []
default: HTTP Error 404: Not Found http://repo.percona.com/ppg-10/yum/release/8/RPMS/x86_64 []
default: Traceback (most recent call last):
default: File "./anydbver", line 1227, in <module>
default: main()
default: File "./anydbver", line 1072, in main
default: handle_non_deployment_commands()
default: File "./anydbver", line 1047, in handle_non_deployment_commands
default: update_versions()
default: File "/home/vagrant/anydbver/anydbver_updater.py", line 773, in update_versions
default: save_percona_postgresql_versions_to_sqlite(osver)
default: File "/home/vagrant/anydbver/anydbver_updater.py", line 147, in save_percona_postgresql_versions_to_sqlite
default: vers2_list = list(open(".version-info/ppg2.{os}.txt".format(os=osver)))
default: FileNotFoundError: [Errno 2] No such file or directory: '.version-info/ppg2.el8.txt'
default: Process install dependency map
default: Starting collection install process
default: Installing 'theredgreek.sqlite:1.2.3' to '/home/vagrant/.ansible/collections/ansible_collections/theredgreek/sqlite'
default: vm.overcommit_memory = 1
default: vm.overcommit_ratio = 10000
default: kernel.panic = 10
default: kernel.panic_on_oops = 1
FR: data-loading for MySQL
Hey Nickolay! :)
I was wondering if it would be too much effort to implement a new parameter like "load-sample-dbs master:node0 samples:sakila,world,menagerie" that would download and load the example databases found in https://dev.mysql.com/doc/index-other.html
Would come super handy for quick correctness verifications and simple testing of backups and many of the pt-* tools.
Ping me on slack and let me know if I can help!
Patroni fails on node2.
In a 3-node PG/Patroni environment, the same node name is being used in nodes 1 and 2, so this is creating conflicts.
I noticed the yaml file is named the same in both nodes, too.
Node1:
[root@node1 ~]# head /etc/patroni/cluster1-1.yml
scope: stampede
#namespace: /service/
name: cluster1-1
restapi:
listen: 0.0.0.0:8008
connect_address: 192.168.48.3:8008
etcd:
host: 192.168.48.3:2379
bootstrap:
# this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
Node2
[root@node2 ~]# head /etc/patroni/cluster1-1.yml
scope: stampede
#namespace: /service/
name: cluster1-1
restapi:
listen: 0.0.0.0:8008
connect_address: 192.168.48.4:8008
etcd:
host: 192.168.48.4:2379
bootstrap:
# this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
Everything else but the name is ok (like IP addresses).
MongoDB sharded cluster wrong keyFile contents
When creating a sharded cluster like this:
./anydbver deploy \
psmdb:4.2 replica-set:rs0 shardsrv \
node1 psmdb:4.2 master:default replica-set:rs0 shardsrv \
node2 psmdb:4.2 master:default replica-set:rs0 shardsrv \
node3 psmdb:4.2 configsrv replica-set:cfg0 \
node4 psmdb:4.2 configsrv replica-set:cfg0 master:node3 \
node5 psmdb:4.2 configsrv replica-set:cfg0 master:node3 \
node6 psmdb:4.2 mongos-cfg:cfg0/node3,node4,node5 mongos-shard:rs0/default,node1,node2
anydbver creates /etc/rs0-keyfile on shard servers and /etc/cfg0-keyfile on config. File name doesn't matter, but content does, and shard servers have different content from config.
This results in sh.addShard() failing.
Copying keyFile contents from cfg to shard servers and restarting fixes it.
mydumper is failing
Hi Nickolay!
Doing this over here, since you are out :) no rush on it.
./anydbver deploy ps:5.7 mydumper
is failing with the following, even though I tried to curl -LO that url and it works:
[...]
TASK [mysql : Install mydumper] **********************************************************************************************************************************
fatal: [agustin.default]: FAILED! => {"changed": false, "msg": "Failure downloading https://github.com/maxbube/mydumper/releases/download/v0.12.5-3-zstd/mydumper-0.12.5-3-zstd.el7.x86_64.rpm, HTTP Error 404: Not Found"}
PLAY RECAP *******************************************************************************************************************************************************
agustin.default : ok=17 changed=8 unreachable=0 failed=1 skipped=32 rescued=0 ignored=0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.