saltstack-formulas / mysql-formula Goto Github PK
View Code? Open in Web Editor NEWInstall the MySQL client and/or server
Home Page: http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
License: Other
Install the MySQL client and/or server
Home Page: http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
License: Other
I am not sure if this is intended but it can be fixed by adding name to the database state
Using this formula it's only possible to configure one mysql server per minion.
Does anyone have ideas of how to simply adapt it for several ones per minion?
I ran into two related issue when trying to install mysql on an AWS instance.
Current pillar.example:
mysql:
__server:
__lookup:
____python: python-mysqldb
Should be in pillar.example:
mysql:
__server:
____lookup:
______python: python-mysqldb
you can see from python.sls lookup is being referenced as a child of server:
{%- set mysql = salt['grains.filter_by'](rawmap, grain='os', merge=salt['pillar.get']%28'mysql:server:lookup'%29) %}
Edit: put underscores for spaces to show indentation in pillar.example snippets
salt.modules.mysql.db_create
has a collate
attribute which allows to specify a default collation to be used when creating a new database.
It would be nice if this formula allows to specify it.
Problem: the current Pillar configuration doesn't allow to specify such kind of settings:
mysql:
database:
- foo
- bar
as this is just a list of database names.
If you have a suggestion on how to configure this, I'm willing to submit a pull request for this feature.
This issue is a "duplicate" of #43 as I have also the problem to install MariaDB on CentOS 7.
The mysql formula should use map.as defaults store with grains.filter_by mechanism to provide a layered defaults store.
I'm working on a patch on my repo https://github.com/refnode/mysql-formula.git and send a pull request soon.
It would be nice if you could adapt this to work with MariaDB - especially since CentOS/RedHat has dropped MySQL in favor of MariaDB.
Using latest develop branch. Tried %, '%', "%" and same variants with *
mysql_user_0:
mysql_user.present:
- name: redacted
- host: % <======================
- password: 'redacted'
- connection_host: localhost
- connection_user: root
- connection_pass: 'redacted'
I'm new to Salt and I'm having trouble using salt to create a db, user and assign privileges to my user. I've added the mysql directory from the mysql-formula to salt/roots/salt.
In the bottom of minion.conf I've added:
mysql.host: 'localhost'
mysql.port: 3306
mysql.user: 'root'
mysql.pass: 'somepass'
mysql.db: 'mysql'
mysql.unix_socket: '/tmp/mysql.sock'
mysql.charset: 'utf8'
In my mysql.sls
for the host I'm provisioning using vagrant I have:
include:
- mysql.server
- mysql.client
mysql_database:
- present
- name: intranet
mysql_user:
- present
- name: intranet
- password_hash: '*65F24BF0453608D25F9E6942F73F8182B90D3780'
- require:
- service: mysqld
mysql_grants:
- present
- grant: all privileges
- database: {{ pillar['mysql']['db'] }}.*
- user: {{ pillar['mysql']['user'] }}
- host: localhost
When I run vagrant provision
I get:
[DEBUG ] Rendered data from file: /srv/salt/intranet/mysql.sls:
include:
- mysql.server
- mysql.client
mysql_database:
- present
- name: intranet
mysql_user:
- present
- name: intranet
- password_hash: '*65F24BF0453608D25F9E6942F73F8182B90D3780'
- require:
- service: mysqld
mysql_grants:
- present
- grant: all privileges
- database: intranet.*
- user: intranet
- host: localhost
[CRITICAL] Rendering SLS intranet.mysql failed, render error: while parsing a block collection
in "<unicode string>", line 2, column 3:
- mysql.server
^
expected <block end>, but found '?'
in "<unicode string>", line 4, column 3:
mysql_database:
^
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/salt/state.py", line 2095, in render_state
sls, rendered_sls=mods
File "/usr/lib/pymodules/python2.7/salt/template.py", line 74, in compile_template
ret = render(input_data, saltenv, sls, **render_kwargs)
File "/usr/lib/pymodules/python2.7/salt/renderers/yaml.py", line 43, in render
data = load(yaml_data, Loader=get_yaml_loader(argline))
File "/usr/lib/python2.7/dist-packages/yaml/__init__.py", line 71, in load
return loader.get_single_data()
File "/usr/lib/python2.7/dist-packages/yaml/constructor.py", line 37, in get_single_data
node = self.get_single_node()
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 36, in get_single_node
document = self.compose_document()
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 55, in compose_document
node = self.compose_node(None, None)
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 82, in compose_node
node = self.compose_sequence_node(anchor)
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 110, in compose_sequence_node
while not self.check_event(SequenceEndEvent):
File "/usr/lib/python2.7/dist-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
File "/usr/lib/python2.7/dist-packages/yaml/parser.py", line 393, in parse_block_sequence_entry
"expected <block end>, but found %r" % token.id, token.start_mark)
ParserError: while parsing a block collection
in "<unicode string>", line 2, column 3:
- mysql.server
^
expected <block end>, but found '?'
in "<unicode string>", line 4, column 3:
mysql_database:
^
[DEBUG ] Loaded no_out as virtual quiet
[DEBUG ] Loaded json_out as virtual json
[DEBUG ] Loaded yaml_out as virtual yaml
[DEBUG ] Loaded pprint_out as virtual pprint
local:
Data failed to compile:
----------
Rendering SLS intranet.mysql failed, render error: while parsing a block collection
in "<unicode string>", line 2, column 3:
- mysql.server
^
expected <block end>, but found '?'
in "<unicode string>", line 4, column 3:
mysql_database:
^
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/salt/state.py", line 2095, in render_state
sls, rendered_sls=mods
File "/usr/lib/pymodules/python2.7/salt/template.py", line 74, in compile_template
ret = render(input_data, saltenv, sls, **render_kwargs)
File "/usr/lib/pymodules/python2.7/salt/renderers/yaml.py", line 43, in render
data = load(yaml_data, Loader=get_yaml_loader(argline))
File "/usr/lib/python2.7/dist-packages/yaml/__init__.py", line 71, in load
return loader.get_single_data()
File "/usr/lib/python2.7/dist-packages/yaml/constructor.py", line 37, in get_single_data
node = self.get_single_node()
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 36, in get_single_node
document = self.compose_document()
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 55, in compose_document
node = self.compose_node(None, None)
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 82, in compose_node
node = self.compose_sequence_node(anchor)
File "/usr/lib/python2.7/dist-packages/yaml/composer.py", line 110, in compose_sequence_node
while not self.check_event(SequenceEndEvent):
File "/usr/lib/python2.7/dist-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
File "/usr/lib/python2.7/dist-packages/yaml/parser.py", line 393, in parse_block_sequence_entry
"expected <block end>, but found %r" % token.id, token.start_mark)
ParserError: while parsing a block collection
in "<unicode string>", line 2, column 3:
- mysql.server
^
expected <block end>, but found '?'
in "<unicode string>", line 4, column 3:
mysql_database:
^
Obviously my mysql.sls
isn't correct, any suggestions?
Install fails on ubuntu (14.04) due to debconf-utils not being installed by default. It should install it (or at least have a toggle on/off)
After I implemented the mysql-formula
, my vagrant provision was failing. giving me the following error:
----------
ID: mysql_debconf
Function: debconf.set
Name: mysql-server
Result: False
Comment: State debconf.set found in sls mysql.server is unavailable
Changes:
----------
After some research I found that the debconf-utils
package was missing on the vagrant box. After doing a sudo apt-get install debconf-utils
the provision was working fine again. I think this formula is missing a dependency to this package.
i noticed something interesting with mysql 5.7:
in the init script, it looks for the pid file by searching for "pid-file" in the output of my_print_defaults
.
but the mysql-formula doesn't set that option. it sets "pid_file" and by default it only sets that option for the [mysqld]
section and not for [mysqld_safe]
. mysqld doesn't care but the init script waits forever. the fallback to $MYSQLDATA/$(hostname).pid
doesn't work as well because the pid-file is located at /var/run/mysqld/mysqld.pid
/etc/init.d/mysql:
...
get_mysql_option() {
RESULT=$(my_print_defaults "$1" | sed -n "s/^--$2=//p" | tail -n 1)
if [ -z "$RESULT" ];
then
RESULT="$3"
fi
echo $RESULT
}
get_running () {
PIDFILE=$(get_mysql_option mysqld_safe pid-file "")
if [ -z "$PIDFILE" ];
then
PIDFILE=$(get_mysql_option mysqld pid-file "$MYSQLDATA/$(hostname).pid")
fi
if [ -e "$PIDFILE" ] && [ -d "/proc/$(cat "$PIDFILE")" ];
then
echo 1
else
echo 0
fi
}
...
Since 04c8db5 I'm seeing the following error while using this formula to build MariaDB on CentOS 7. The user grant appears to fail while applying a minion highstate but running the command manually works fine.
----------
ID: mysql_user_user_localhost_0
Function: mysql_grants.present
Name: admin_database_all
Result: False
Comment: Failed to execute: "GRANT all privileges ON database.* TO user@localhost" (MySQL Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1)
Started: 23:11:53.928118
Duration: 5.062 ms
Changes:
# mysql -psecret
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 127
Server version: 5.5.50-MariaDB MariaDB Server
Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> use database
Database changed
MariaDB [database]> GRANT all privileges ON database.* TO user@localhost;
Query OK, 0 rows affected (0.03 sec)
MariaDB [database]>
Pillar data:
mysql:
server:
root_user: 'root'
root_password: 'secret'
mysqld:
bind-address: localhost
log_error: /var/log/mariadb/mariadb.log
pid-file: /var/run/mariadb/mariadb.pid
mysqld_safe:
bind-address: localhost
log_error: /var/log/mariadb/mariadb.log
pid-file: /var/run/mariadb/mariadb.pid
database:
- database
schema:
database:
load: True
source: salt://files/database.sql
lookup:
server: mariadb-server
client: mariadb
service: mariadb
user:
user:
password: 'secret'
host: localhost
databases:
- database: database
grants: ['all privileges']
Master:
# salt --versions-report
Salt Version:
Salt: 2016.3.2
Dependency Versions:
cffi: 0.8.6
cherrypy: 3.2.2
dateutil: 1.5
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: 0.21.0
libnacl: Not Installed
M2Crypto: 0.21.1
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.7
mysql-python: Not Installed
pycparser: 2.14
pycrypto: 2.6.1
pygit2: 0.21.4
Python: 2.7.5 (default, Aug 18 2016, 15:58:25)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.2.1511 Core
machine: x86_64
release: 3.10.0-327.18.2.el7.x86_64
system: Linux
version: CentOS Linux 7.2.1511 Core
Minion:
$ salt-call --versions-report
Salt Version:
Salt: 2016.3.2
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.7
mysql-python: 1.2.3
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
Python: 2.7.5 (default, Aug 18 2016, 15:58:25)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.2.1511 Core
machine: x86_64
release: 3.10.0-327.22.2.el7.x86_64
system: Linux
version: CentOS Linux 7.2.1511 Core
Hello,
I'm using this formula with an option to change the InnoDB log file size.
innodb_log_file_size: 256M
It causes problem on my Debian VM: mysql is installed, configured but can't (re)start. The error tells about the current log file size which is different from the one specified in my.cnf.
I think what happens is the following:
I modified the formula to make mysql_config state run before the mysql.server package installation, and it solved my problem.
What do you think about this? Should it be possible, and safe, to change the states order?
Thanks.
> salt-call state.apply mysql.server
[CRITICAL] Rendering SLS 'base:mysql.server' failed: Conflicting ID 'mysql_delete_anonymous_user_localhost.localdomain'
local:
Data failed to compile:
----------
Rendering SLS 'base:mysql.server' failed: Conflicting ID 'mysql_delete_anonymous_user_localhost.localdomain'
See mysql/server.sls:
{% for host in ['localhost', 'localhost.localdomain', salt['grains.get']('fqdn')] %}
mysql_delete_anonymous_user_{{ host }}:
mysql_user:
- absent
- host: {{ host or "''" }}
- name: ''
- connection_host: '{{ mysql_host }}'
- connection_user: '{{ mysql_salt_user }}'
{% if mysql_salt_password %}
- connection_pass: '{{ mysql_salt_password }}'
{% endif %}
- connection_charset: utf8
- require:
- service: mysqld
- pkg: mysql_python
{%- if (mysql_salt_user == mysql_root_user) and mysql_root_password %}
- cmd: mysql_root_password
{%- endif %}
{% endfor %}
The mysql formula fails on openSUSE with
Rendering SLS 'production:mysql.server' failed: Jinja variable 'None' has no attribute 'server'
Rendering SLS 'production:mysql.server' failed: Jinja variable 'None' has no attribute 'server'
The reason is that default.yaml gets filtered by the 'os' grain - but doesn't include a matching key.
# salt '*' grains.get os
tux: # openSUSE Tumbleweed
openSUSE Tumbleweed
salt-test-1: # openSUSE Leap 42.2
SUSE
However default.yaml has "openSUSE" as a key, which doesn't match any of the above.
I can't speak for other distributions, but at least for openSUSE it would be a good idea to use the 'os_family' grain:
# salt '*' grains.get os_family
tux: # openSUSE Tumbleweed
Suse
salt-test-1: # openSUSE Leap 42.2
Suse
(CC @ccboltz)
To reproduce, set mysql:server:root_password
to False
in pillar and assert any state, for example mysql
. salt reports The following requisites were not found: require: debconf: mysql_debconf
Cause:
server.sls
def's mysql_debconf
inside of {% if mysql_root_password %}
so if root password is False
, mysql_debconf
is not defined. However, it is required by most states.
After installing mysql-server with this formula, with a root_password set in pillar, I change the password in pillar and run state.highstate, but the root password in mysql stays the same. I see salt changing the mysql root password in debconf, but that is only used during install of the package.
Unsure if this is intentional, but a variety of mysqld options are missing from the supported_params.yaml
file, such as skip-networking
and a whole bunch more on https://dev.mysql.com/doc/refman/5.7/en/security-options.html and https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html.
Is this by design? If it's just that options are missing and haven't been added, perhaps that second link's table could be trimmed down into YAML and included in the given file -- that would ensure all options are covered in one fell swoop.
Formulas iterating over dictionaries should use the items()
method instead of iteritems()
because the latter method has been removed from Python 3. While the items()
method in Python 2 uses more memory than iteritems()
, these dictionaries are typically small (e.g., sets of settings from Pillar).
Premise: Our saltmaster is stateless, so we can just delete the machine our autoscaling group will recreate it and all minions going to connect.
Our saltmaster will create the database, setup users and fetches the mysql dump from s3 and load it.
If i understand the SLS correctly schema load ({{ state_id }}_load
) will always be triggered when the schema file is being handled ({{ state_id }}_schema
). This is because of - watch: - file: {{ state_id }}_schema
.
Is this expected behaviour? Once the database is created it should load the schema. It shouldn't load the schema file or execute another schema import.
Thoughts?
Hey guys,
I was working on this formula and wanted to have a way to test in multiple OS at once so I come up with a solution that involves TravisCI and KitchenCI.
I tried to draft a global
solution that can be easily ported to other formulas because it reads the FORMULA
file to get the formula name and based on that knows the name of the state to execute.
At this time for this particular formula it tests on the following OS:
Once #147 is fixed we can add support for CentOS and Fedora, and once test-kitchen/kitchen-docker#156 is merged we will be able to support openSuSE too.
I think this feature will allow contributors to test their commits on multiple scenarios to ensure formula quality.
Let me know if you have any question/improvement!
Thanks,
Andres
[CRITICAL] Rendering SLS "base:mysql" failed: Jinja variable 'dict object' has no attribute 'test.rand_str'
/srv/formulas/mysql-formula/mysql/database.sls(3):
---
{% from "mysql/map.jinja" import mysql with context %}
{% set mysql_root_pass = salt['pillar.get']('mysql:server:root_password', salt['test.rand_str'](64)) %} <======================
The formula is the original one and it has no custom pillar data. It is a masterless setup with version 2014.1.6 salt-call.
Right now, there's no handling at all for non-default locations of datadir
.
datadir
datadir
to its new location (shutdown of mysql service, moving datadir
, starting mysql service again)There might be some difficulties involved in this, as the package management of some distributions adds mechanisms (postinst etc.) to the MySQL package which expects the default path (/var/lib/mysql
) and otherwise might break the package installation/always re-create the datadir, etc.
The database.sls file is pre-pending "mysql_db" to the database name.
Just needs a name declaration added..
e.g.
I can't install MySQL server because Jinja can't find mysql/defaults.yaml
(related with this not solved but closed issue)
Currently I've tested this with a clean new VM deploying from my notebook (thanks to salt-ssh) without success.
I hope that this can be solved.
etc/salt/master
root_dir: .
file_roots:
base:
- srv/salt/states
- srv/salt/formulas/mysql-formula
pillar_roots:
base:
- srv/salt/pillars
srv/salt/states/top.sls
base:
'*':
- mysql.server
srv/salt/pillars/top.sls
base:
'*':
- database
srv/salt/pillars/database.sls
mysql:
server:
root_user: 'root'
root_password: 'pass'
user: mysql
{% if grains['os_family'] == 'RedHat' %}
lookup:
server: mariadb-server
client: mariadb
{% endif %}
Error:
ID: mysql_config
Function: file.managed
Name: /etc/my.cnf
Result: False
Comment: Unable to manage file: Jinja error: mysql/defaults.yaml
Traceback (most recent call last):
File "/tmp/.root_b91e95_salt/salt/utils/templates.py", line 368, in render_jinja_tmpl
output = template.render(**decoded_context)
File "/tmp/.root_b91e95_salt/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/tmp/.root_b91e95_salt/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "<template>", line 7, in top-level template code
File "/tmp/.root_b91e95_salt/salt/utils/jinja.py", line 144, in get_source
raise TemplateNotFound(template)
TemplateNotFound: mysql/defaults.yaml
; line 7
---
[...]
# This config is generated by SALTSTACK
# and all change will be overrided on next salt call
{#-
===== FETCH DATA =====
-#}
{%- from "mysql/defaults.yaml" import rawmap with context -%} <======================
{%- from "mysql/supported_sections.yaml" import supported_sections with context -%}
{%- set datamap = salt['grains.filter_by'](rawmap, grain='os', merge=salt['pillar.get']('mysql:server:lookup')) -%}
{#-
===== COMBINE DATA =====
-#}
[...]
---
Started: 05:25:44.050898
Duration: 30.905 ms
Changes:
salt-ssh --versions-report
Salt Version:
Salt: 2015.8.0
Dependency Versions:
Jinja2: 2.7.3
M2Crypto: Not Installed
Mako: 1.0.0
PyYAML: 3.11
PyZMQ: 14.4.1
Python: 2.7.9 (default, Apr 2 2015, 15:33:21)
RAET: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.2
gitdb: 0.5.4
gitpython: Not Installed
ioflo: Not Installed
libnacl: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.2
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
python-gnupg: Not Installed
smmap: 0.8.2
timelib: Not Installed
System Versions:
dist: Ubuntu 15.04 vivid
machine: x86_64
release: 3.19.0-30-generic
system: Ubuntu 15.04 vivid
When using a non-standard datadir
on Ubuntu, changes to AppArmor are required, otherwise MySQL won't start anymore.
See also: https://blogs.oracle.com/jsmyth/entry/apparmor_and_mysql
Similar changes are most likely required for SELinux enabled systems.
when provisioning a mysql server, the root_password is written in clear in the minion's log, which is not so good...
2015-06-08 18:27:33,049 [salt.state ][INFO ][22761] Running state [mysqladmin --user root password 'XXX'] at time 18:27:33.049208
2015-06-08 18:27:33,050 [salt.state ][INFO ][22761] Executing state cmd.run for mysqladmin --user root password 'XXX'
2015-06-08 18:27:33,051 [salt.loaded.int.module.cmdmod][INFO ][22761] Executing command 'mysql --user root --password=\'XXX\' --execute="SELECT 1;"' in directory '/root'
2015-06-08 18:27:33,129 [salt.loaded.int.module.cmdmod][INFO ][22761] Executing command "mysqladmin --user root password XXX'" in directory '/root'
2015-06-08 18:27:33,227 [salt.state ][INFO ][22761] {'pid': 23051, 'retcode': 0, 'stderr': '', 'stdout': ''}
2015-06-08 18:27:33,227 [salt.state ][INFO ][22761] Completed state [mysqladmin --user root password 'XXX'] at time 18:27:33.227229
I'm not sure how to avoid that beside creating a custom module...
If I set in my pillar something like this:
mysql:
server:
mysqld:
bind-address: 0.0.0.0
... then the configuration file gets properly updated and the server gets reloaded but it's actually not sufficient and MySQL continues to listen on its default localhost
address.
It has to be manually restarted for the change to take effect.
That would be cool if the formula takes care of this by itself.
The python package specified in the map.jinja file is incorrect for CentOS 5.
I'm not sure if this also affects Redhat
The correct package name is 'python26-mysqldb'
Not sure of the best way to fix this.. Obviously the grain "osmajorrelease" will allow determination of the os version..
Is there a way to add a further condition to the grains.filter_by in the map?
mysql_grants.present
supports a boolean attribute to override the default escaping of string values. The formula should support this as well.
See luciddg#1
my.cnf
requires some parameters to be specified multiple times. An example would be binlog-ignore-db
. This formula cannot support that due to the my.cnf
sections in the pillar being a dict.
Using a list of dicts would solve this, but would break the current interface of the formula. Any guidance on how to resolve this?
When changing innodb_log_file_size
, mysqld
won't start anymore without moving /var/lib/mysql/ib_logfile*
out of the way.
See also this article for details.
I think it could be basically implemented like this:
{% for section, data in mysql.config.sections.iteritems() %}
{% if 'innodb_log_file_size' in data %}
{% set innodb_log_file_workaround = true %}
{% endif %}
{% endfor %}
{% if innodb_log_file_workaround is defined and innodb_log_file_workaround is sameas true %}
mysqld-stopped-for-logfile-workaround:
service.dead:
- name: {{ mysql.service }}
- prereq:
- module: innodb-logfile-workaround
- require:
- pkg: mysqld
{% set curtime = None | strftime("%Y-%m-%d-%H:%M:%S") %}
{% set backup_dir = mysql.config.sections.mysqld.datadir ~ '/../ib_logfile-backup-' ~ curtime ~ '/' %}
innodb-logfile-backupdir:
file.directory:
- name: {{ backup_dir }}
- makedirs: true
innodb-logfile-workaround:
module.run:
- name: cmd.run
- cmd: "mv {{ mysql.config.sections.mysqld.datadir }}/ib_logfile* {{ backup_dir }}"
- onlyif: "ls {{ mysql.config.sections.mysqld.datadir }}/ib_logfile*"
- prereq:
- file: mysql_config
- require:
- file: innodb-logfile-backupdir
{% endif %}
This is not fully working yet and a bit hackish - other ideas/sugestions/PRs welcome!
Hi!
My mysql user "foo" can connect from localhost and from another ip. Same username, different ip.
At the moment I can't produce two entries in mysql.user table with the same "User" field and different "Host" field
In Ubuntu 14.04 i'm setting lookup.server value in my pillar file as: mysql-server-5.6. This value is shown with the pillar.tems. However running salt-call --local state.highstate, installs the value from the defaults.yml, which is 'mysql-server'.
Changing the value in the defaults.yml, to mysql-server-5.6 works and the correct version gets installed.
On a Debian VM, Salt is printing the below messages:
[ERROR ] Rendering exception occurred: Jinja variable 'dict object' has no attribute 'version' [CRITICAL] Rendering SLS 'base:mysql.server' failed: Jinja variable 'dict object' has no attribute 'version' [ERROR ] Data passed to highstate outputter is not a valid highstate return: {'local': ["Rendering SLS 'base:mysql.server' failed: Jinja variable 'dict object' has no attribute 'version'"]}
The pillar.example doesn't include a 'version' entry, nor does the defaults.yaml . The 'version' value is only noted on lines 67, 91, and 108 of server.sls . Further, should lines 91 and 108 be
{% if ( os_family == 'RedHat' or os_family == 'Suse' ) and mysql.version >= 5.7 %}
or
{% if os_family == 'RedHat' or ( os_family == 'Suse' and mysql.version >= 5.7 ) %}
instead? As currently written, it appears to cause a check of mysql.version regardless of the OS family.
When using the root_password
option in the pillar.example to set the root password, I am unable to login as root using the root password:
sudo salt-call pillar.get mysql
local:
----------
server:
----------
root_password:
xxx
root_user:
root
The login attempt:
mysql -u root -p
Enter password:
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
The output from debconf:
----------
ID: mysql_debconf
Function: debconf.set
Name: mysql-server
Result: True
Comment:
Started: 14:59:59.839264
Duration: 1488.64 ms
Changes:
----------
mysql-server/root_password:
(password hidden)
mysql-server/root_password_again:
(password hidden)
mysql-server/start_on_boot:
true
skip-name-resolve
is an important option but I am not sure how this one should configured from pillar.
What is the proper way to add this in pillar?
We are using Oracle Enterprise Linux, which is basically the same as RedHat Enterprise Linux - the problem is that the "os" grain value is "OEL" for Oracle Linux, but the defaults.yaml is only prepared for RedHat (and CentOS and Fedora).
Duplicating the RedHat part in defaults.yaml and changing the key to "OEL" works fine.
My question is, could you include this in the formula?
Thanks in advance.
I can not log in to mysql after I used this formula and set the following in pillar:
I have put in fake pass words and logins in this file:
Otherwise Ive made no changes to the salt state files.
then ran it like so:
sudo salt [Server_IP_Address] state.apply mysql
Then I try to login from either the jump server or the server where mysql was installed it fails:
Enter password:
ERROR 2003 (HY000): Can't connect to MySQL server on '[Server_IP_Address]' (111)
I'm assuming I have failed to set some needed value in the pillar.
Any help would be appreciated.
Jeff Wilson
Or be able to give 'hosts' as a list
With multiple mysql servers to setup it would be more useful to be able to specify interface service is listening on then ip address, as it can be easily obtained in runtime from specified interface.
Add a functionality to remove a database user:
Example pillar:
mysql:
# Manage users
# you can get pillar for existing server using scripts/import_users.py script
user:
frank:
absent: true ## <===== Add this, the user will be removed.
password: 'somepass'
host: localhost
databases:
- database: foo
grants: ['select', 'insert', 'update']
- database: bar
grants: ['all privileges']
Run:
salt 'db*' state.apply mysql.user
part of mysql/config.sls
included by mysql/client.sls
, from here
mysql_config:
file.managed:
- name: {{ mysql.config.file }}
- require:
- pkg: {{ mysql.server }}
- template: jinja
{% if "config_directory" in mysql %}
- source: salt://mysql/files/my-include.cnf
{% else %}
- source: salt://mysql/files/my.cnf
{% endif %}
{% if os_family in ['Debian', 'Gentoo', 'RedHat'] %}
- user: root
- group: root
- mode: 644
{% endif %}
If I just want to install the client on a webserver, to probe mysql connection from the webserver to the dbserver, I don't need to have the server package to be installed.
On Debian jessie there'are, 2 packages available:
mariadb-common
dpkg -L mariadb-common
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/mariadb-common
/usr/share/doc/mariadb-common/copyright
/usr/share/doc/mariadb-common/changelog.Debian.gz
/etc
/etc/mysql
/etc/mysql/conf.d
/etc/mysql/conf.d/mariadb.cnf
mysql-common
root@web0:~# dpkg -L mysql-common
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/mysql-common
/usr/share/doc/mysql-common/NEWS.Debian.gz
/usr/share/doc/mysql-common/changelog.gz
/usr/share/doc/mysql-common/copyright
/usr/share/doc/mysql-common/changelog.Debian.gz
/etc
/etc/mysql
/etc/mysql/my.cnf
/etc/mysql/conf.d
/etc/mysql/conf.d/.keepme
When setting up MySQL from scratch (without a pre-existing /var/lib/mysql/), I get errors like
ID: mysql_delete_anonymous_user_localhost.localdomain
Function: mysql_user.absent
Name:
Result: False
Comment: MySQL Error 1045: Access denied for user 'salt'@'localhost' (using password: YES)
This happens because mysql_delete_anonymous_user_* runs before the salt_user gets created.
Additionally, mysql.salt-user isn't included in init.sls, which means by default salt_user doesn't get created at all.
Proof of concept-patch:
--- a/mysql/init.sls
+++ b/mysql/init.sls
@@ -11,6 +11,7 @@
include:
- mysql.server
+ - mysql.salt-user
- mysql.database
- mysql.user
{% if mysql_dev %}
--- a/mysql/server.sls
+++ b/mysql/server.sls
@@ -52,6 +52,7 @@ mysql_delete_anonymous_user_{{ host }}:
{% endif %}
- connection_charset: utf8
- require:
+ - mysql_root_password
- service: mysqld
- pkg: mysql_python
{%- if (mysql_salt_user == mysql_root_user) and mysql_root_password %}
Note that this patch is not perfect - requiring mysql_root_password delays mysql_delete_anonymous_user_* enough to get it working. However it uses the mysql_salt_user, not root - so the dependency on mysql_root_password is technically wrong ;-)
(CC @ccboltz)
Using the mysql-formula on Ubuntu >=15.04, using systemd, the installation of mysql-server
will fail as described in Launchpad#1478849.
Still investigating workarounds for this issue. Will report here in case I find a way to get it working in a more-or-less hackish way.
Hey guys,
I'm opening this ticket to fix a couple of things I discovered when testing this formula on CentOS/RHEL 7.
I will be pushing a PR in a bit.
Thanks!
I'm getting the following error when I run the mysql state:
----------
ID: mysql_config
Function: file.managed
Name: /etc/mysql/my.cnf
Result: False
Comment: Unable to manage file: Jinja error: mysql/defaults.yaml
Traceback (most recent call last):
File "/tmp/.root_4e6e95__salt/salt/utils/templates.py", line 306, in render_jinja_tmpl
output = template.render(**decoded_context)
File "/tmp/.root_4e6e95__salt/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/tmp/.root_4e6e95__salt/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "<template>", line 7, in top-level template code
File "/tmp/.root_4e6e95__salt/salt/utils/jinja.py", line 143, in get_source
raise TemplateNotFound(template)
TemplateNotFound: mysql/defaults.yaml
; line 7
---
[...]
# This config is generated by SALTSTACK
# and all change will be overrided on next salt call
{#-
===== FETCH DATA =====
-#}
{%- from "mysql/defaults.yaml" import rawmap with context -%} <======================
{%- from "mysql/supported_sections.yaml" import supported_sections with context -%}
{%- set datamap = salt['grains.filter_by'](rawmap, grain='os', merge=salt['pillar.get']('mysql:server:lookup')) -%}
{#-
===== COMBINE DATA =====
-#}
[...]
---
Started: 16:42:01.829849
Duration: 40.14 ms
Changes:
When you install this formula you may expect that the defined password for mysql's root
on the managed server will be available on the salt master for mysql.module call:
salt 'db*' mysql.db_list
salt 'db*' mysql.user_create bob 10.0.0.3 pipo
# etc.
It requires to have setup in /etc/salt/minion
as said in the doc above:
mysql.host: 'localhost'
mysql.port: 3306
mysql.user: 'root'
mysql.pass: 'S3retPass4root!'
mysql.db: 'mysql'
mysql.unix_socket: '/tmp/mysql.sock'
mysql.charset: 'utf8'
or
mysql.default_file: '/etc/mysql/debian.cnf'
I propose to use the second form in the file /root/.my.cnf
I accomplished a prototype here: server.sls
I will post more in this ticket about security. And sum up discussion in #salt IRC channel.
For Debian OS family, debconf is used to set root user password, which is fine, but root user cannot be specified this way thus it is set to root for all Debian systems.
When the formula attemps to do further changes to mysql like creating a database, etc, it fails if root_user
is set to anything else than root.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.