saltstack-formulas / salt-formula Goto Github PK

View Code? Open in Web Editor NEW

198.0 59.0 420.0 1.44 MB

Yes, Salt can Salt itself!

Home Page: http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html

License: Other

Shell 3.54% SaltStack 57.87% Ruby 15.61% JavaScript 8.19% Jinja 14.80%

salt-formula's People

Contributors

Stargazers

Watchers

Forkers

auser basepi lucernex kennethwilke tiger-seo lmzopq gravyboat tmandry johtso viq worlds-enough-studios adcade libermentix spsoit timjones crucialcloudhosting jacksoncage westurner joroy virtru-ops plainlystated iggy darren-alexander juiceinc kev009 andrew-vant restlessbandit tzu3l robmessick stooj somenick iamtew ckng openenvi-formulas shearer12345 rhertzog packetloop qbcode mriedmann ianwu2014 eedgar voxxit bpflanz rpatterson maytech bebo jcftang qdubious renoirb mohae jbouse amustafa marclop robertfach daschatten ugns cloud-ee volantisops luciddg rowhit gpcsolutions rabidcicada aukan quixoten tony dyejon alkivi-sas mbehrle vdcow sjd-risca-saltpin blbradley txomon kfdm predatorvi evicertia pruiz kurt108 puneetk juicemobile puluto peterdemin geekcrowds nrgaway cyclefusion oro suporten3 innerdaze chqcvs hoonetorg martinhoefling jor123 alexeiswirth jpic cyaliven bradthurber kaiwisdom pravcorp null9beta dafyddj marccardinal

salt-formula's Issues

Not obvious how to use it

I'm new to Salt. I've read the documentation fairly extensively but without a "here's how to run this" in the README.md I'm unlikely to have time to figure out how to use this, even though I would really like to.

Can I use salt-ssh from my local machine to use this to deploy salt onto a remote machine? If so, how? Can it be explained in the README?

Default value for minion "master" parameter does not work

When the master is also being configured. The problem is this line in salt/files/minion:

{{ get_config('master', 'salt') }}

If pillars['salt']['minion']['master'] does not exist, it will try pillars['salt']['master'], which of course is itself a dict.

pillar key "master" clashes with config name "master"

Due to how the get_config macro works, if you don't have the master option set in salt.minion, and you have a salt.master config, then the entire salt.master config is mistakenly used as the value for the master option for the minion.

move changed config files to master.d and minion.d

I would like to propose the following change:

if a config file is changed by pillar data put the file in:

/etc/salt/master.d/master.conf
/etc/salt/minion.d/minion.conf

So not to overwrite the defaults salt package provides ?
If i'm not mistaken salt will prefer the files in master.d and minion.d
because all defaults are commented.

Ability to run from Git clone

It would be awesome if the salt-formula could be configured to clone Salt from the Git repo (e.g. giving it a version tag), and then it set up a virtualenv and made sure the minion/master were running from source, simultaneously disabling any distro-wide installations.

[CentOS 6] `mapping` test not supported, master config cannot be generated

Unfortunately, since CentOS 6 only ships with Jinja2 2.2.1, the mapping test is not supported. Looking through the documentation, I don't see anything else equivalent to it (both iterable and sequence test true for dicts and lists), so I'm not sure how to maintian functionality without that test.

Tapping @Sjd-Risca, just in case they have some idea

salt version 2015.8.0rc1 can't reference {{ slspath }}

in the latest salt version 2015.8.0rc1 the source salt://{{ slspath }} doesn't work anymore

file_|-salt-master_|-/etc/salt/master.d_|-recurse:
        ----------
        __run_num__:
            1
        changes:
            ----------
        comment:
            #### /etc/salt/master.d/reactor.conf ####
            Unable to manage file: [Errno 2] No such file or directory: u'|/var/cache/salt/minion/files/base/salt/files/master.d/reactor.conf'

            #### /etc/salt/master.d/f_defaults.conf ####
            Unable to manage file: [Errno 2] No such file or directory: u'|/var/cache/salt/minion/files/base/salt/files/master.d/f_defaults.conf'
        duration:
            52.152 ms
        name:
            /etc/salt/master.d
        result:
            False
        start_time:
            21:36:23.706337

if I change the {{ slspath }} back to the hardcoded salt://salt/files/ it works

formula fails on CentOS when run from gitfs

I've recently converted a bunch of formulas I cloned and then referenced via file_roots to local git repos referenced from gitfs_remotes. A couple of formulas now break whenever a dictionary item can't be resolved.

I am using CentOS, and this formula fails because it can't resolve salt.pkrepo.centos. Since this was working fine when pulled from file_roots I am not sure what to say, although wrapping the contents of salt/pkgrepo/init.sls in an if checking for os_family == 'Debian' eliminates the problem for me.

Jinja2 mapping test failure on RHEL 6

Salting minion fails on all RHEL 6 boxes with python-jinja2-2.2.1-2.el6_5.x86_64. Same run succeeds on RHEL 7 boxes with python-jinja2-2.7.2-2.el7.noarch.

----------
          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: False
     Comment: #### /etc/salt/minion.d/f_defaults.conf ####
              Unable to manage file: Jinja syntax error: no test named 'mapping'
              /var/cache/salt/minion/files/prod/salt/formulas.jinja(14):
              ---
              [...]

              {%- macro formulas_git_opt(env, opt) -%}
              {%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
                   salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
                     defaults[opt])) -%}
              {%- if value is mapping -%}    <======================
              {{ value|yaml }}
              {%- else -%}
              {{ value }}
              {%- endif -%}
              {%- endmacro -%}
              [...]
              ---
     Started: 22:55:39.879651
    Duration: 133.819 ms
     Changes:

"dev" directory at the top level of the repo is too ambiguous

The dev directory (the vagrant files, etc.) gets plopped down at the top level of your salt:// fileserver root. Seems like it could cause a name conflict. Maybe it should be namespaced?

salt.minion fails with syntax error: no test named 'mapping' RHEL6/Jinja 2.2.1

It looks like mapping test requires Jinja2 v2.6 or higher and I'm on 2.2.1 - which is the best RHEL 6 provides.

          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: False
     Comment: #### /etc/salt/minion.d/f_defaults.conf ####
              Unable to manage file: Jinja syntax error: no test named 'mapping'
              /var/cache/salt/minion/files/base/salt/formulas.jinja(14):
              ---
              [...]

              {%- macro formulas_git_opt(env, opt) -%}
              {%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
                   salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
                     defaults[opt])) -%}
              {%- if value is mapping -%}    <======================
              {{ value|yaml }}
              {%- else -%}
              {{ value }}
              {%- endif -%}
              {%- endmacro -%}

# salt-master --versions
           Salt: 2015.5.3
         Python: 2.6.6 (r266:84292, Nov 21 2013, 10:50:32)
         Jinja2: 2.2.1
       M2Crypto: 0.20.2
 msgpack-python: 0.4.6
   msgpack-pure: Not Installed
       pycrypto: 2.0.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 14.5.0
           RAET: Not Installed
            ZMQ: 4.0.5
           Mako: 0.3.4
        Tornado: Not Installed

salt-cloud provider configuration file permissions are too open

The salt-cloud provider configuration files (.../cloud.providers.d/*.conf) may contain keying material (e.g., passwords or secret keys). The permissions of these files should be restricted to some group of authorized users (e.g., root or wheel) in order to prevent unauthorized access.

Use `repo.saltstack.com` where appropriate/possible

Currently, the following distributions/releases are covered by https://repo.saltstack.com:

salt.pkgrepo should make use of repo.saltstack.com where appropriate/possible.

I might take care of providing a PR for Ubuntu. Can't promise yet to add support for other distributions as well.

Windows is missing

How about salting/upgrading salt minions on windows in a stable way? I cant see support for that in this formula.

I have some work implemented for this but its very "hacky" due to that nssm.exe often hangs and locks files after an uninstall. To come around this I'm rebooting the server and continue the installation afterwards but this is far from optimal.

I can however, share the work if anyone is interested.

map.jinja uses feature not available in released Salt

77522d6 added the base keyword to filter_by() in map.jinja. This is not available until the next release.

Use as submodule

I can not see how to use this has a git submodule.
I'm forced to create a state to override all salt:// declarations with include and extend to get things work.
Not being able to use this as submodule reduce the interest of the project from "plug and play" to "cherry pick". And cherry pick won't bring contributions..
I've found no solution for now. But I'm new to saltstack. I may have missed something.

Use salt PPA for ubuntu installations

salt.syndic fails to compile

# salt-call state.sls salt.syndic test=True
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'base', ** done ** 'salt/syndic.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/master.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
local:
    Data failed to compile:
----------
    A recursive requisite was found, SLS "salt.syndic" ID "salt-syndic" ID "salt-syndic"

Error when running salt.minion

The following error occurs:

          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: False
     Comment: #### /etc/salt/minion.d/_defaults.conf ####
              Unable to manage file: Jinja syntax error: no test named 'mapping'
              /var/cache/salt/minion/files/base/salt/formulas.jinja(14):
              ---
              [...]

              {%- macro formulas_git_opt(env, opt) -%}
              {%- set value = salt['pillar.get']('salt_formulas:git_opts:{}:{}'.format(env, opt),
                   salt['pillar.get']('salt_formulas:git_opts:default:{}'.format(opt),
                     defaults[opt])) -%}
              {%- if value is mapping -%}    <======================
              {{ value|yaml }}
              {%- else -%}
              {{ value }}
              {%- endif -%}
              {%- endmacro -%}
              [...]
              ---

Which appears to come from line 14 in ../salt/formulas.jinja

Not sure what the right value should be.

Manage salt-key as well

Accepting minions after installation with this formula will make provisioning much easier.

Master template gitfs_remotes must have children

salt-formula/salt/files/master.d/_defaults.conf

Lines 486 to 502 in 0f5f976

 {% if 'gitfs_remotes' in master -%} 

 gitfs_remotes: 

 {%- for remote in master['gitfs_remotes'] %} 

 {% if remote is iterable %} 

 {%- for repo, children in remote.iteritems() -%} 

 - {{ repo }}: 

 {%- for child in children %} 

 {% for key, value in child.iteritems() -%} 

 - {{ key }}: {{ value }} 

 {%- endfor %} 

 {%- endfor %} 

 {%- endfor %} 

 {%- else -%} 

 - {{ remote }} 

 {%- endif %} 

 {%- endfor %} 

 {%- endif %}

is iterable doesn't seem to be a deep enough check.

My salt.sls pillar looks something like this:

salt:
  master:
    fileserver_backend:
      - git
      - roots
    gitfs_remotes:
      - [email protected]:saltstack-formulas/salt-formula.git

And if the remote item doesn't have a child as the above doesn't (note that there's no data about the remote, just the address) it fails with the following jinja error:

I get the following jinja error when running state.highstate:

          ID: salt-master
    Function: file.recurse
        Name: /etc/salt/master.d
      Result: False
     Comment: #### /etc/salt/master.d/_defaults.conf ####
              Unable to manage file: Jinja variable 'str object' has no attribute 'iteritems'
     Started: 03:18:47.515161
    Duration: 153.94 ms
     Changes:

If I add a remote child, it starts working:

    gitfs_remotes:
      - [email protected]:saltstack-formulas/salt-formula.git:
        - base: master

salt versions-report is:

           Salt: 2014.7.0
         Python: 2.7.5 (default, Jun 17 2014, 18:11:42)
         Jinja2: 2.7.2
       M2Crypto: 0.21.1
 msgpack-python: 0.4.2
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 14.3.1
           RAET: Not Installed
            ZMQ: 4.0.4
           Mako: Not Installed

Restart salt-minion after end of run

Hi all,

Is there a reason why something like : http://docs.saltstack.com/en/latest/faq.html#what-is-the-best-way-to-restart-a-salt-daemon-using-salt is not used in this formula ?

I changed salt-formula (locally), it's a huge win : I can now spawn a new salt minion (with salt-cloud) in one highstate run, without interruption.

salt/minion.sls :

  cmd.wait:
    - name: echo salt-call --local service.restart salt-minion | at now + 1 minute
    - python_shell: True
    - order: last
    - watch:
{% if salt_settings.install_packages %}
      - pkg: salt-minion
{% endif %}
      - file: salt-minion
      - file: remove-old-minion-conf-file

I didn't manage "atd" since it's on by default on Debian, but it's definitely doable.

salt_cloud_certs broken by pull #106 ("Don't overwrite salt variable")

In context, see cb3aa80#diff-cc4a14e8ec642fa03a6038c172bd52b0L46.

The original Pillar lookup had nothing to do with the salt variable that was renamed to salt_settings. Instead, it was querying the value of the Pillar key "salt_cloud_certs". This commit changes the Pillar key to "salt:salt_cloud_certs" without also updating https://github.com/saltstack-formulas/salt-formula/blob/master/salt/files/key and https://github.com/saltstack-formulas/salt-formula/blob/master/pillar.example, and it violates POLA in that it breaks working salt-cloud configurations managed by salt-formula.

salt.minion is not working centos5

2015-07-29 15:09:16,376 [salt.state                                 ][ERROR   ][8517] #### /etc/salt/minion.d/f_defaults.conf ####
Unable to manage file: Jinja syntax error: no test named 'mapping'
/var/cache/salt/minion/files/itg/salt/formulas.jinja(14):

---
[...]

{%- macro formulas_git_opt(env, opt) -%}
{%- set value = salt['pillar.get']('salt_formulas:git_opts:{0}:{1}'.format(env, opt),
     salt['pillar.get']('salt_formulas:git_opts:default:{0}'.format(opt),
       defaults[opt])) -%}
{%- if value is mapping -%}    <======================
{{ value|yaml }}
{%- else -%}
{{ value }}
{%- endif -%}
{%- endmacro -%}
[...]

Mapping test is available from jinja version >= 2.6. But for centos5 available is only 2.5 version.

10 failed states when provisioning Vagrant with all defaults

(Warning: I'm a Salt newb)

Running the following from a clean checkout of salt-formula:

vagrant up
vagrant ssh
vagrant@salt:~$ sudo su -
root@salt:~# salt-call state.highstate

I get the following failed states:

----------
          ID: salt-minion
    Function: file.recurse
        Name: /etc/salt/minion.d
      Result: False
     Comment: Recurse failed: none of the specified sources were found
     Started: 23:03:24.860639
    Duration: 8292.687 ms
     Changes:   
----------
          ID: salt-minion
    Function: service.running
      Result: False
     Comment: One or more requisite failed: {'salt.minion.salt-minion': 'Recurse failed: none of the specified sources were found'}
     Started: 
    Duration: 
     Changes:   
----------
          ID: cloud-cert-aws-pem
    Function: file.managed
        Name: /etc/salt/cloud.providers.d/key/aws.pem
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.656005
    Duration: 8.735 ms
     Changes:   
----------
          ID: cloud-cert-gce-pem
    Function: file.managed
        Name: /etc/salt/cloud.providers.d/key/gce.pem
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.666232
    Duration: 11.163 ms
     Changes:   
----------
          ID: salt-cloud-profiles-ec2
    Function: file.managed
        Name: /etc/salt/cloud.profiles.d/ec2.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.677963
    Duration: 7.45 ms
     Changes:   
----------
          ID: salt-cloud-profiles-gce
    Function: file.managed
        Name: /etc/salt/cloud.profiles.d/gce.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.686129
    Duration: 12.509 ms
     Changes:   
----------
          ID: salt-cloud-providers-ec2
    Function: file.managed
        Name: /etc/salt/cloud.providers.d/ec2.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.699382
    Duration: 3.817 ms
     Changes:   
----------
          ID: salt-cloud-providers-gce
    Function: file.managed
        Name: /etc/salt/cloud.providers.d/gce.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.703812
    Duration: 5.745 ms
     Changes:   
----------
          ID: salt-cloud-maps-ec2
    Function: file.managed
        Name: /etc/salt/cloud.maps.d/ec2.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.710319
    Duration: 6.986 ms
     Changes:   
----------
          ID: salt-cloud-maps-gce
    Function: file.managed
        Name: /etc/salt/cloud.maps.d/gce.conf
      Result: False
     Comment: Unable to manage file: string indices must be integers, not str
     Started: 23:04:46.722172
    Duration: 10.791 ms
     Changes:

After this failure if I run:

salt-run fileserver.update
salt-call state.highstate

all states succeed.

What is the ideal set of commands to go from 'vagrant up' to all states passing with no failures? Not sure if I am missing something... for a new person seeing a successful run of this formula from a clean repo would be a good confidence builder :).

Watch master.d and minion.d

As reported on this mailinglist post, I was hoping that salt-master would automatically restart when a config file changes there.

Consider a way to install Salt from a specific GitHub pull request.

In terms of testing the Salt project in general it would be cool if this formula offered a streamlined way to install the Salt codebase from a GitHub Pull Request ref point.

Just a thought. Feel free to close this up if that is too crazy an idea!

minion/master file.recurse needs to stop using clean: True

The salt code (and myself) put files in /etc/salt/{master,minion}.d

The "clean: True" is killing all that. It probably shouldn't do that.

salt-minion:
  file.recurse:
    - name: {{ salt.get('config-path', '/etc/salt') }}/minion.d
    - template: jinja
    - source: salt://salt/files/minion.d
    - clean: True
    - context:
        standalone: False

salt variables are overwritten all over the place

i.e. {% set salt = .... BAD!~!!!

`gitfs` support doesn't ensure dependency on `pip` is fulfilled

gitfs support makes use of a pip.installed state in several places.
When pip isn't installed, applying the salt.gitfs state will fail like this:

# salt-call state.sls salt.gitfs.pygit2 test=true
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'base', ** done ** u'salt/gitfs/pygit2.sls'
[INFO    ] Fetching file from saltenv 'base', ** done ** u'salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** done ** u'salt/defaults.yaml'
[ERROR   ] You should upgrade pyOpenSSL to at least 0.14.1 to enable the use of X509 extensions
[INFO    ] Running state [git] at time 11:41:13.008687
[INFO    ] Executing state pkg.installed for git
[INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}\n', '-W'] in directory '/root'
[INFO    ] Package git is already installed
[INFO    ] Completed state [git] at time 11:41:13.499474
[INFO    ] Running state [libgit2-22] at time 11:41:13.499758
[INFO    ] Executing state pkg.installed for libgit2-22
[INFO    ] The following packages are set to be installed/updated: libgit2-22
[INFO    ] Completed state [libgit2-22] at time 11:41:13.504759
[INFO    ] Running state [pygit2 == 0.22.1] at time 11:41:13.505978
[INFO    ] Executing state pip.installed for pygit2 == 0.22.1
[ERROR   ] An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
[INFO    ] Completed state [pygit2 == 0.22.1] at time 11:41:13.506796
local:
----------
          ID: pygit2-git
    Function: pkg.installed
        Name: git
      Result: True
     Comment: Package git is already installed
     Started: 11:41:13.008687
    Duration: 490.787 ms
     Changes:   
----------
          ID: libgit2-22
    Function: pkg.installed
      Result: None
     Comment: The following packages are set to be installed/updated: libgit2-22
     Started: 11:41:13.499758
    Duration: 5.001 ms
     Changes:   
----------
          ID: install-pygit2
    Function: pip.installed
        Name: pygit2 == 0.22.1
      Result: False
     Comment: An importable pip module is required but could not be found on your system. This usually means that the systems pip package is not installed properly.
     Started: 11:41:13.505978
    Duration: 0.818 ms
     Changes:   

Summary for local
------------
Succeeded: 2 (unchanged=1)
Failed:    1
------------
Total states run:     3
Total run time: 496.606 ms

Alter the formula to support pygit2 on FreeBSD

The process of installing from source must be completely overhauled.

A binary package named py27-pygit2 is available on FreeBSD.

Doesn't entirely play nice with salt-cloud

Ran into an issue today: salt.minion does all of its management in minion.d and doesn't touch the /etc/salt/minion file. salt-cloud does the opposite. Usually the settings in the former override the latter so everything is okay. However, for features that can be configured either from the minion config or the pillar root (such as mine_functions), the cloud-deployed file takes precedence over pillar, and salt.minion has no way of cleaning it up after the fact.

I don't think this is really a bug, but removing those settings after the fact is surprisingly irritating given that salt.minion won't touch that file.

I don't know the correct way to deal with this, but it seems like some option for managing /etc/salt/minion or otherwise neutering it (perhaps off by default) would be a helpful thing to have.

salt.master fails in salt-ssh: TemplateNotFound: salt/formulas.jinja

I'm attempting to use salt-ssh 2015.5.0 to set up a salt master. The salt.master state is failing.

----------
          ID: salt-master
    Function: file.recurse
        Name: /etc/salt/master.d
      Result: False
     Comment: #### /etc/salt/master.d/f_defaults.conf ####
              Unable to manage file: Jinja error: salt/formulas.jinja
              Traceback (most recent call last):
                File "/tmp/.root_679d2e__salt/salt/utils/templates.py", line 306, in render_jinja_tmpl
                  output = template.render(**decoded_context)
                File "/tmp/.root_679d2e__salt/jinja2/environment.py", line 969, in render
                  return self.environment.handle_exception(exc_info, True)
                File "/tmp/.root_679d2e__salt/jinja2/environment.py", line 742, in handle_exception
                  reraise(exc_type, exc_value, tb)
                File "<template>", line 15, in top-level template code
                File "/tmp/.root_679d2e__salt/salt/utils/jinja.py", line 110, in get_source
                  raise TemplateNotFound(template)
              TemplateNotFound: salt/formulas.jinja

              ; line 15

              ---
              [...]
              {{ configname }}: {{ cfg_salt[configname] }}
              {%- else -%}
              #{{ configname }}: {{ default_value }}
              {%- endif -%}
              {%- endmacro -%}
              {%- from 'salt/formulas.jinja' import file_roots, formulas with context -%}    <======================
              ##### Primary configuration settings #####
              ##########################################
              # This configuration file is used to manage the behavior of the Salt Master
              # Values that are commented out but have no space after the comment are
              # defaults that need not be set in the config. If there is a space after the
              [...]
              ---
     Started: 04:18:21.410948
    Duration: 271.237 ms
     Changes:

I cloned the formula manually and have it referenced in my file_roots. Not sure if that would have an impact over bringing it in via gitfs_remotes, but I'm trying to avoid adding the gitfs dependencies.

Rename to avoid fork conflict

One can not fork "saltstack/salt" and " saltstack-formulas/salt".
And having a "salt" outside saltstack-formulas is misleading.
This issue applies to saltstack-formulas/*
Maybe prefix with something like salt-formula-* ?

Errors rendering salt/formulas.sls or salt/files/master.d/f_defaults.conf

Running the salt.formulas state results in the following error:

root@uxeprdbsdsalt01:~ # salt-call state.sls salt.formulas saltenv=production test=True
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'production', ** skipped ** latest already in cache 'salt://salt/formulas.sls'
[INFO    ] Fetching file from saltenv 'production', ** done ** 'salt/formulas.jinja'
[CRITICAL] Rendering SLS 'production:salt.formulas' failed: mapping values are not allowed here; line 18

---
[...]

# Setup the directory hosting the Git repository


/usr/local/etc/salt/formulas
...:    <======================
  file.directory:
    - makedirs: True
    - group: wheel
    - user: root
    - mode: 755
[...]

---
local:
    Data failed to compile:
----------
    Rendering SLS 'production:salt.formulas' failed: mapping values are not allowed here; line 18

---
[...]

# Setup the directory hosting the Git repository


/usr/local/etc/salt/formulas
...:    <======================
  file.directory:
    - makedirs: True
    - group: wheel
    - user: root
    - mode: 755
[...]

---

Running the salt.master state results in the following incorrect rendering (truncated for brevity's sake):

root@uxeprdbsdsalt01:~ # salt-call state.sls salt.master saltenv=development test=True
[INFO    ] Loading fresh modules for state activity
[INFO    ] Fetching file from saltenv 'development', ** skipped ** latest already in cache 'salt://salt/master.sls'
[INFO    ] Fetching file from saltenv 'development', ** done ** 'salt/map.jinja'
[INFO    ] Fetching file from saltenv 'development', ** done ** 'salt/defaults.yaml'
[INFO    ] Executing command '/sbin/zfs -?' in directory '/root'
[INFO    ] Running state [py27-salt] at time 14:50:01.749847
[INFO    ] Executing state pkg.installed for py27-salt
[INFO    ] Executing command 'pkg info -ao' in directory '/root'
[INFO    ] Package py27-salt is already installed.
[INFO    ] Completed state [py27-salt] at time 14:50:01.963569
[INFO    ] Running state [/usr/local/etc/salt/master.d] at time 14:50:01.968229
[INFO    ] Executing state file.recurse for /usr/local/etc/salt/master.d
[INFO    ] Fetching file from saltenv 'development', ** skipped ** latest already in cache u'salt://salt/files/master.d/f_defaults.conf'
[INFO    ] Fetching file from saltenv 'development', ** skipped ** latest already in cache 'salt://salt/formulas.jinja'
[INFO    ] Fetching file from saltenv 'development', ** skipped ** latest already in cache u'salt://salt/files/master.d/reactor.conf'
[INFO    ] {'/usr/local/etc/salt/master.d/f_defaults.conf': {'diff': '--- \n+++ \n@@ -431,77 +431,146 @@\n     - /code/twgs-formula\n     - /code/users-formula\n   production:\n-    - /usr/local/etc/salt/formulas/active-directory-formula\n-    - /usr/local/etc/salt/formulas/epel-formula\n-    - /usr/local/etc/salt/formulas/fail2ban-formula\n-    - /usr/local/etc/salt/formulas/firewalld-formula\n-    - /usr/local/etc/salt/formulas/git-formula\n-    - /usr/local/etc/salt/formulas/jenkins-formula\n-    - /usr/local/etc/salt/formulas/mysql-formula\n-    - /usr/local/etc/salt/formulas/ntp-formula\n-    - /usr/local/etc/salt/formulas/nux-formula\n-    - /usr/local/etc/salt/formulas/openstack-formula\n-    - /usr/local/etc/salt/formulas/openssh-formula\n-    - /usr
...
----------
          ID: salt-master
    Function: file.recurse
        Name: /usr/local/etc/salt/master.d
      Result: None
     Comment: #### /usr/local/etc/salt/master.d/f_defaults.conf ####
              The file /usr/local/etc/salt/master.d/f_defaults.conf is set to be changed
     Started: 14:50:01.968229
    Duration: 890.319 ms
     Changes:
              ----------
              /usr/local/etc/salt/master.d/f_defaults.conf:
                  ----------
                  diff:
                      ---
                      +++
                      @@ -431,77 +431,146 @@
                           - /code/twgs-formula
                           - /code/users-formula
                         production:
                      -    - /usr/local/etc/salt/formulas/active-directory-formula
                      -    - /usr/local/etc/salt/formulas/epel-formula
                      -    - /usr/local/etc/salt/formulas/fail2ban-formula
                      -    - /usr/local/etc/salt/formulas/firewalld-formula
                      -    - /usr/local/etc/salt/formulas/git-formula
                      -    - /usr/local/etc/salt/formulas/jenkins-formula
                      -    - /usr/local/etc/salt/formulas/mysql-formula
                      -    - /usr/local/etc/salt/formulas/ntp-formula
                      -    - /usr/local/etc/salt/formulas/nux-formula
                      -    - /usr/local/etc/salt/formulas/openstack-formula
                      -    - /usr/local/etc/salt/formulas/openssh-formula
                      -    - /usr/local/etc/salt/formulas/os-hardening-formula
                      -    - /usr/local/etc/salt/formulas/owncloud-formula
                      -    - /usr/local/etc/salt/formulas/postgres-formula
                      -    - /usr/local/etc/salt/formulas/rabbitmq-formula
                      -    - /usr/local/etc/salt/formulas/salt-formula
                      -    - /usr/local/etc/salt/formulas/shibboleth-formula
                      -    - /usr/local/etc/salt/formulas/snmp-formula
                      -    - /usr/local/etc/salt/formulas/spigotmc-formula
                      -    - /usr/local/etc/salt/formulas/sudoers-formula
                      -    - /usr/local/etc/salt/formulas/tomcat-formula
                      -    - /usr/local/etc/salt/formulas/twgs-formula
                      -    - /usr/local/etc/salt/formulas/users-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../active-directory-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../epel-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../fail2ban-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../firewalld-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../git-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../jenkins-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../mysql-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../ntp-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../nux-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../openstack-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../openssh-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../os-hardening-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../owncloud-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../postgres-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../rabbitmq-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../salt-formula
                      +    - /usr/local/etc/salt/formulas
                      +.../shibboleth-formula

configuration files named incorrectly (missing .conf)

Am I missing something, or should the configuration files be /etc/salt/minion.d/minion.conf rather than the current /etc/salt/minion.d/minion?

http://docs.saltstack.com/en/latest/ref/configuration/master.html#default-include

Per default the master will automatically include all config files from master.d/*.conf where master.d is relative to the directory of the master configuration file.

Deprecated iteritems() method not forward-compatible with Python 3

Formulas iterating over dictionaries should use the items() method instead of iteritems() because the latter method has been removed from Python 3. While the items() method in Python 2 uses more memory than iteritems(), these dictionaries are typically small (e.g., sets of settings from Pillar).

Add installation of gitfs dependencies (GitPython)

Please remove extra/outdated branches

i.e. style-short-dec branch

Update the salt formulas to leverage the new 2014.7.0 features out of the box.

I am looking to have the state do the steps outlined in this doc out of the box.

apt-get install python-service-identity python-cherrypy
salt-call --local tls.create_self_signed_cert
http://docs.saltstack.com/en/latest/ref/netapi/all/salt.netapi.rest_cherrypy.html

no salt-cloud vmware provider

as of 2015.8.x, vmware as a provider is available, as such i've manually altered the formula to add that functionality locally:

added a cloud.provider.d vmware.conf template
added a cloud.profile.d vmware.conf template
altered cloud.sls to install pyvmomi if required

I'm raising this issue, because this functionality should probably exist in the cloud formula going forward.

if people are interested, i can prep a PR for review.

Error while trying to use salt.gitfs.gitpython

I'm getting an error when trying use this state.

johncc@aws-dev-c-salt-1:~⟫ sudo salt-call -l debug state.sls salt.gitfs.gitpython                                                                                                                                                              
[sudo] password for johncc: 
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG   ] Configuration file path: /etc/salt/minion
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG   ] Initializing new SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG   ] Decrypting the current master AES key
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded state.sls
[DEBUG   ] LazyLoaded saltutil.is_running
[DEBUG   ] LazyLoaded grains.get
[DEBUG   ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Including configuration from '/etc/salt/minion.d/f_defaults.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/f_defaults.conf
[DEBUG   ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[INFO    ] Loading fresh modules for state activity
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/gitfs/gitpython.sls'
[DEBUG   ] LazyLoaded cmd.run
[DEBUG   ] Jinja search path: ['/var/cache/salt/minion/files/base']
[DEBUG   ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[DEBUG   ] LazyLoaded grains.filter_by
[DEBUG   ] LazyLoaded pillar.get
[CRITICAL] Rendering SLS 'base:salt.gitfs.gitpython' failed: Jinja variable 'dict object' has no attribute 'gitpython'
[DEBUG   ] Re-using SAuth for ('/etc/salt/pki/minion', 'aws-dev-c-salt-1', 'tcp://192.168.101.7:4506')
[DEBUG   ] LazyLoaded highstate.output
local:
    Data failed to compile:
----------
    Rendering SLS 'base:salt.gitfs.gitpython' failed: Jinja variable 'dict object' has no attribute 'gitpython'

It was suggested in the IRC channel by AndreasLutro that the problem is caused by incorrect merging of the default and OS specific settings dictionaries.

In particular, it was suggested that this line is at fault https://github.com/saltstack-formulas/salt-formula/blob/master/salt/map.jinja#L79 and should be changed to do a deep merge instead of a shallow merge.

Add support for Halite configuration

http://salt.readthedocs.org/en/latest/topics/tutorials/halite.html#configuring-halite-settings

salt.minion fails when applied with salt-ssh

I just posted this issue with SaltStack as I believe the bug exists in salt-ssh rather than in this formula directly. This is really just for visibility.

Brief summary:
When calling salt.minion via salt-ssh, the formulas.jinja file is not included and so the minion.d/f_defaults.conf file fails to render.

Workaround is to simply modify the following lines in salt/files/minion.d/f_defaults.conf:

@@ -12,7 +12,6 @@
 #{{ configname }}: {{ default_value }}
 {%- endif -%}
 {%- endmacro -%}
-{%- from 'salt/formulas.jinja' import file_roots, formulas with context -%}
 ##### Primary configuration settings #####
 ##########################################

@@ -235,8 +234,6 @@ file_client: local
 {{ file_roots(cfg_minion['file_roots']) }}
 {%- elif 'file_roots' in cfg_salt -%}
 {{ file_roots(cfg_salt['file_roots']) }}
-{%- elif formulas|length -%}
-{{ file_roots({'base': ['/srv/salt']}) }}
 {%- endif %}

 # limit fileserver traversal

Salt-cloud crypto dependency fails.

I rebuilt my salt master today and the salt.cloud state refuses to run, saying that the attempt to pip.install package 'crypto' failed. I tried it manually and it also failed. I looked at the package index located here: https://pypi.python.org/pypi?%3Aaction=index and see no package named 'crypto.'

I am confused, because this worked a month ago. Did the package get renamed in pip recently, or something? Shouldn't the distribution-provided crypto package be getting used anyway, if possible?

[salt-master-2015.8.0-2.el6] salt.minion ends up with values from salt.master's grains in config file, cannot start

Configuration:

This is surprisingly easy to reproduce:

`/srv/salt/top.sls`:

base:
  '*':
    - salt.minion

  'G@roles:master':
    - salt.master

`/srv/pillar/master.sls`:

salt:
  master:
    fileserver_backend:
      - roots
      - git

    file_roots:
      base:
        - /srv/salt

    pillar_roots:
      base:
        - /srv/pillar

    gitfs_provider: pygit2

    gitfs_remotes:
      - git://github.com/saltstack-formulas/salt-formula

`/srv/pillar/minion.sls`:

salt:
  minion:
    - master: 'salt'

/srv/pillar/top.sls:

base:
  '*':
    - minion

  'G@roles:master':
    - master

Results:

Given those values, I end up with the following in /etc/salt/minion.d/f_defaults.conf:

[root@pluto ~]# grep -v '^#' /etc/salt/minion.d/f_defaults.conf | uniq

master:
  - fileserver_backend
  - gitfs_remotes
  - pillar_roots
  - file_roots
  - gitfs_provider


[root@pluto ~]#

salt.formulas fails on highstate if not data

I removed the salt_formulas: thing from my pillar, but forgot the salt/top.sls with the salt.formulas there and it started to fail.

[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/master.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/minion.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/map.jinja'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/defaults.yaml'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/formulas.sls'
[INFO    ] Fetching file from saltenv 'base', ** skipped ** latest already in cache 'salt://salt/formulas.jinja'
[CRITICAL] Rendering SLS 'base:salt.formulas' failed: Jinja variable 'str object' has no attribute 'iteritems'
local:
    Data failed to compile:
----------
    Rendering SLS 'base:salt.formulas' failed: Jinja variable 'str object' has no attribute 'iteritems'

Take spaces out of IDs

Looks like this is most prevalent here: https://github.com/saltstack-formulas/salt-formula/blob/master/salt/ssh.sls (shame on you @timoguin ;) ). We should fix that.

Reduce amount of configuration in rendered configuration files

For example in this one we can see many default comments.
Although this is important for new users, the comments make much harder to navigate among all this lines.
Moreover if meaning of some parameter changes and we don't syncronize the change with original, our words will be confusing.

What do you think about removing these comments?

salt minion configuration does not support multiple master

See: http://docs.saltstack.com/en/latest/topics/tutorials/multimaster.html#configure-minions

	{% if 'gitfs_remotes' in master -%}
	gitfs_remotes:
	{%- for remote in master['gitfs_remotes'] %}
	{% if remote is iterable %}
	{%- for repo, children in remote.iteritems() -%}
	- {{ repo }}:
	{%- for child in children %}
	{% for key, value in child.iteritems() -%}
	- {{ key }}: {{ value }}
	{%- endfor %}
	{%- endfor %}
	{%- endfor %}
	{%- else -%}
	- {{ remote }}
	{%- endif %}
	{%- endfor %}
	{%- endif %}

saltstack-formulas / salt-formula Goto Github PK

salt-formula's People

Contributors

Stargazers

Watchers

Forkers

salt-formula's Issues

Configuration:

/srv/salt/top.sls:

/srv/pillar/master.sls:

/srv/pillar/minion.sls:

Results:

Recommend Projects

Recommend Topics

Recommend Org

Jobs

`/srv/salt/top.sls`:

`/srv/pillar/master.sls`:

`/srv/pillar/minion.sls`: