salopensource / sal Goto Github PK

Modular reporting for Endpoints

License: Apache License 2.0

Python 40.19% Makefile 0.18% HTML 12.05% CSS 14.96% JavaScript 22.63% Shell 0.36% Dockerfile 0.18% Less 5.37% SCSS 4.09% Procfile 0.01%

sal's People

Contributors

Stargazers

Watchers

sal's Issues

facter3 results in 502 on checkin

As soon as the .7.5 version of sal-scripts is told to use facter3, even though I get some data for that part of the report, it's only 13 of the 38 keys in that data. I can't see anything wrong between https://github.com/salopensource/sal/blob/master/server/views.py#L1511 and https://github.com/salopensource/sal/blob/master/server/models.py#L237, since facter2 was sending the same sorts of data types, so I can't find where the issue is happening while I'm not running with DEBUG on. The current result is a 502 on the checkin section, but at least the catalogs submission in the same run go through. If it helps for correlation, the keys in that dict that are missing in Sal(including some custom facts we use and PE-specific ones) are:

 'aio_agent_build'
 'custom_auth_conf'
 'dmi'
 'facterversion'
 'filesystems'
 'identity'
 'is_virtual'
 'kernelmajversion'
 'kernelversion'
 'mac_admin_users'
 'memory'
 'mountpoints'
 'munki_catalog'
 'munki_included_manifest'
 'mysql_server_id'
 'os'
 'path'
 'pe_concat_basedir'
 'processors'
 'puppet_files_dir_present'
 'ruby'
 'ssh'
 'staging_http_get'
 'system_profiler'
 'system_uptime'

MunkiInstalls plugin query issue

[12/Dec/2016 09:45:18] ERROR [django.request:124] Internal Server Error: /pluginload/MunkiInstalls/
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/docker/sal/server/views.py", line 574, in plugin_load
    html = plugin.plugin_object.widget_content(page, machines, theID)
  File "/home/docker/sal/server/plugins/munkiinstalls/munkiinstalls.py", line 60, in widget_content
    update_history__machine=machines, update_history__update_type='third_party').count()
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 369, in count
    return self.query.get_count(using=self.db)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 476, in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 457, in get_aggregation
    result = compiler.execute_sql(SINGLE)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
ProgrammingError: more than one row returned by a subquery used as an expression

This could indeed just be that my data is loco, and I'm not familiar enough with this plugin to really see what the issue is here. It is also most definitely not a big deal to me whether this works or not. Just figured I'd report the issue.

Passing DOCKER_SAL_BASIC_AUTH does not set BASIC_AUTH variable

Passing DOCKER_SAL_BASIC_AUTH does not set BASIC_AUTH variable - instead it looks like the DEBUG variable is set.

https://github.com/salopensource/sal/blob/master/docker/settings_import.py#L33

# Read the BASIC_AUTH setting from env var
try:
    if getenv('DOCKER_SAL_BASIC_AUTH').lower() == 'true':
        DEBUG = True
    else:
        DEBUG = False
except:
    DEBUG = True

License Tracking

We should 'borrow' the license tracking code from MWA.

application inventory ignores too much apple software

https://github.com/salopensource/sal/blob/django1.10/inventory/views.py#L248-L252

it ignores anything that is com.apple.* except for a short whitelist.

I'd like to know Safari versions, who has the Sierra installer in their applications folder, etc.

Perhaps if you want to hide things then hide the ones in the System folder or something.

facter3 uptime not reflected in plugin

facter is being called on postflight with the --puppet and yaml options, which would only contain the new format by default. The way we flattened facter output turns that into system_uptime=>days, which isn't what the plugin can therefore pull out.
We could handle this by throwing a conditional in the plugin (I guess, but facter version isn't a top-level value in the machine model), dropping facter2 support and changing the [fact_name] that is checked, or add the --show-legacy option and put either of those above changes off for now.

MariaDB Specified key was too long

Hello,

I've been playing with Sal on my testing machine. When trying to configure it with MariaDB I got following error:

    execute_from_command_line(sys.argv)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/core/management/base.py", line 305, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/core/management/base.py", line 356, in execute
    output = self.handle(*args, **options)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 202, in handle
    targets, plan, fake=fake, fake_initial=fake_initial
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 97, in migrate
    state = self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 132, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 237, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/migrations/operations/models.py", line 525, in database_forwards
    getattr(new_model._meta, self.option_name, set()),
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 333, in alter_unique_together
    self.execute(self._create_unique_sql(model, columns))
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 112, in execute
    cursor.execute(sql, params)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 112, in execute
    return self.cursor.execute(query, args)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/Volumes/HangarDeck/repos/github/sal/sal_env/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.OperationalError: (1071, 'Specified key was too long; max key length is 767 bytes')

It did not matter if configured with strict mode or not.

Docker Image

Hey,

Was going to update my docker container to the latest release and newest version is Sal-LDAP is 2.1.1. When will you be pushing the new image?

500 Internal Server Error

Getting an internal server error page when trying to access Sal after just completing the install instructions for Ubuntu 14. I created a gist of the error.log here: https://gist.github.com/chasetb/3e867e455638cb0e932a

Looks like the issues is either an issue with mod_wsgi or something in the AppRegistry. Any ideas?

Empty conditions seems to break the DB

--- Too quick to hit enter, sorry ---

I've a munki condition who report the license version for Office, it test the existence of license v1 or v2 file and report 2011, 2016 or no value.

It's the only condition I use actually.

It seems to break SAL with the following error.

[30/May/2016 12:33:22] ERROR [django.request:256] Internal Server Error: /checkin/
Traceback (most recent call last):
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/mnt/data/www/sal_env/sal/server/views.py", line 1884, in checkin
    condition.save()
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/models/base.py", line 734, in save
    force_update=force_update, update_fields=update_fields)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/models/base.py", line 762, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/models/base.py", line 846, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/models/base.py", line 885, in _do_insert
    using=using, raw=raw)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 974, in execute_sql
    cursor.execute(sql, params)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 129, in execute
    six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
  File "/mnt/data/www/sal_env/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
IntegrityError: (1048, "Column 'condition_data' cannot be null")

Business Unit Dashboard fails to load plugins

When a Business Unit has more than one machine group, and plugins that use the machines query, it fails to load, throwing an exception:

[14/Dec/2016 10:45:58] ERROR [django.request:124] Internal Server Error: /id_pluginload/Status/bu_dashboard/1/
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/docker/sal/server/views.py", line 574, in plugin_load
    html = plugin.plugin_object.widget_content(page, machines, theID)
  File "/home/docker/sal/server/plugins/status/status.py", line 40, in widget_content
    errors = machines.filter(errors__gt=0).count()
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 369, in count
    return self.query.get_count(using=self.db)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 476, in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 457, in get_aggregation
    result = compiler.execute_sql(SINGLE)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
ProgrammingError: more than one row returned by a subquery used as an expression

The root of this problem is actually in the /server/views.py machines query for BusinessUnit filtered machines.

9687da3 rewrote the querying, and mistakenly tries to do a "exact" query on machine groups (which is why we see the "more than one row" error in the logs, and the problem disappears for Business Units with only one Machine Group.

PR incoming.

CSV export is slow

Inventory csv export on SQLite is very slow. Other csv exports are pretty slow.

@grahamgilbert and I talked about me doing a pass through this at some point in the future.

Search not working as expected

It would be nice to search for machines by the Munki ClientIdentifier/Manifest. From my research it doesn't look like this is possible without getting into the database.

Back button in Application Inventory doesn't go back if pressed Next page.

Here is what I do:

Click 'Application Inventory'
Click Next to go to the next page.
Click Back in the menu, which takes me to the previous page, not back in the navigation.
Click Back again, which takes me forward again to the page I was at.

After that Back only toggles between the two recent pages.

Sal version 2.7.0.671

Filter out _mbsetupuser

As per convo in Slack chat, insert

if data.get('username') != '_mbsetupuser':

here - https://github.com/salopensource/sal/blob/master/server/views.py#L1382

Checkin results in IntegrityError

With the sal-scripts v2.0, I get these all day long:

[22/Dec/2016 08:52:57] ERROR [django.request:124] Internal Server Error: /checkin/
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/docker/sal/sal/decorators.py", line 119, in wrap
    return function(request, *args, **kwargs)
  File "/home/docker/sal/server/views.py", line 2022, in checkin
    UpdateHistoryItem.objects.bulk_create(update_history_item_to_save)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 449, in bulk_create
    self._batched_insert(objs_with_pk, fields, batch_size)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 1060, in _batched_insert
    inserted_id = self._insert(item, fields=fields, using=self.db, return_id=True)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 1043, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 1054, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "server_updatehistoryite_update_history_id_78bcc0ce2c3b29f0_uniq"
DETAIL:  Key (update_history_id, recorded, status)=(236856, 2016-12-22 13:52:57.393286+00, pending) already exists.

Edit business unit fails if there is no machine group

Reproduce:
Clean installation of Sal.
Create new business unit
Edit business unit.

Expected:
Should be able to correct name of business unit.

Result:



Request Method: GET
Request URL: http://REDACTED/business_unit/edit/1/

Django Version: 1.10
Python Version: 2.7.6
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'sal',
 'server',
 'api',
 'catalog',
 'inventory',
 'licenses',
 'bootstrap3',
 'watson',
 'datatableview',
 'search')
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'server.middleware.AddToBU.AddToBU',
 'search.current_user.CurrentUserMiddleware')

Template loader postmortem
Django tried loading these templates, in this order:

Using engine django:
    * django.template.loaders.filesystem.Loader: /home/docker/sal/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.filesystem.Loader: /home/docker/sal/server/plugins/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.filesystem.Loader: /home/docker/sal/plugins/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /usr/local/lib/python2.7/dist-packages/django/contrib/auth/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /usr/local/lib/python2.7/dist-packages/django/contrib/admin/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /usr/local/lib/python2.7/dist-packages/django/contrib/admindocs/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /home/docker/sal/server/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /home/docker/sal/inventory/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /home/docker/sal/licenses/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /usr/local/lib/python2.7/dist-packages/bootstrap3/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /home/docker/sal/watson/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /home/docker/sal/datatableview/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)
    * django.template.loaders.app_directories.Loader: /home/docker/sal/search/templates/{'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>} (Source does not exist)



Traceback:

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py" in inner
  39.             response = get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view
  23.                 return view_func(request, *args, **kwargs)

File "/home/docker/sal/server/views.py" in edit_business_unit
  827.     return render('forms/edit_business_unit.html', c)

File "/usr/local/lib/python2.7/dist-packages/django/shortcuts.py" in render
  30.     content = loader.render_to_string(template_name, context, request, using=using)

File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py" in render_to_string
  67.         template = get_template(template_name, using=using)

File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py" in get_template
  25.     raise TemplateDoesNotExist(template_name, chain=chain)

Exception Type: TemplateDoesNotExist at /business_unit/edit/1/
Exception Value: {'business_unit': <BusinessUnit: TestUnit>, 'form': <EditUserBusinessUnitForm bound=False, valid=Unknown, fields=(name;users)>}

Workaround.
Creare machine group before editing business unit.

Can't seem to get more than one machine detail plugin

As soon as I enable a second machine detail widget, the machine detail view gives:

Environment:


Request Method: GET
Request URL: http://sal.valhalla.tk/machine_detail/666/

Django Version: 1.10
Python Version: 2.7.6
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'sal',
 'server',
 'api',
 'catalog',
 'inventory',
 'licenses',
 'bootstrap3',
 'watson',
 'datatableview',
 'search',
 'axes')
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'server.middleware.AddToBU.AddToBU',
 'search.current_user.CurrentUserMiddleware',
 'axes.middleware.FailedLoginMiddleware')



Traceback:

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py" in inner
  39.             response = get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view
  23.                 return view_func(request, *args, **kwargs)

File "/home/docker/sal/server/views.py" in machine_detail
  1423.     output = utils.orderPluginOutput(output)

File "/home/docker/sal/server/utils.py" in orderPluginOutput
  474.             if int(item['width']) != 0:

Exception Type: KeyError at /machine_detail/666/
Exception Value: 'width'

Can't create Machine Group with API

Perhaps I'm doing this wrong...

import requests
import json

data = json.dumps({"business_unit": 1, "name": "PartyHaus"})
response = requests.post("https://sal.is.the.bomb.com/api/machine_groups", headers={"privatekey": "something", "publickey": "also_something"}, data=data)

results in response 500 and the following in the server logs:

[12/Dec/2016 15:46:36] ERROR [django.request:124] Internal Server Error: /api/machine_groups/
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/docker/sal/api/auth.py", line 14, in wrap
    return function(request, *args, **kwargs)
  File "/home/docker/sal/api/views.py", line 204, in machine_group_list
    serializer.save()
  File "/usr/local/lib/python2.7/dist-packages/rest_framework/serializers.py", line 192, in save
    self.instance = self.create(validated_data)
  File "/usr/local/lib/python2.7/dist-packages/rest_framework/serializers.py", line 890, in create
    raise TypeError(msg)
TypeError: Got a `TypeError` when calling `MachineGroup.objects.create()`. This may be because you have a writable field on the serializer class that is not a valid argument to `MachineGroup.objects.create()`. You may need to make the field read-only, or override the MachineGroupSerializer.create() method to handle this correctly.
Original exception was:
 Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/rest_framework/serializers.py", line 873, in create
    instance = ModelClass.objects.create(**validated_data)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 399, in create
    obj.save(force_insert=True, using=self.db)
TypeError: save() got an unexpected keyword argument 'force_insert'

Searchable osquery data

It would be great to have the integrated osquery data available through the search box!

Thanks!

Filter Errors and Warnings

An admin should be able to filter errors and warnings (e.g. softwareupdate error: 100).

Username field too short

In the auth_user database, the username is limited to 31 char. This is too short when we use UPN as ID.

Using UPN instead of SAM is mandatory when you've environment using full name and attribute for login.

For example, my customer use most of the time username scheme like this one:

And in this scenario, you can't ask end user to remember the SAM version who's limited in char.

So please, remove the username length limitation, it's not useful in 2016, and it's painful.

Update History tracking kinda sucks

Now the version installed is built in to Munki's report, we should switch to that from parsing the install log. It will be less computationally intensive. Apart from getting an install date from the log that we don't from munki (perhaps a PR to munki to get this?), we don't gain anything.

Machine serial is modified, becomes issue when dealing with multiple inventory sources and VMs

sal/server/views.py

Lines 1664 to 1669 in 6588d82

 serial = data.get('serial') 

 serial = serial.upper() 

 # Take out some of the weird junk VMware puts in. Keep an eye out in case Apple actually uses these: 

 serial = serial.replace('/', '') 

 serial = serial.replace('+', '')

Uppercase and deleting characters results in SAL reporting a very different machine serial compared to munki, osquery, santa and other tools used in the community. In case one is aggregating multiple inventory sources a virtual machine synced from SAL (via API) is very different - resulting in odd behaviour in systems like Zentral that people report to us.

The issue could be fixed on the client side by pretending an Apple like serial to VMware - but the above code looks like a short way to get things rolling from longer ago. In case you'll refactor code sometime in future I'd be more than happy if you'd consider using the reported serial 'as is' so it will match the ones reported by other tools.

Machine Model Bar Plugin uses expensive calls

Retrieving the whole machine model can take a long time - for large installs, the plugin never loads. We should just retrieve the values we need. As @sheagcraig has turned the python up to 11 on this one (I'd never even heard of Counter), this may take me a while.

@sheagcraig do you know how this works enough so that we can use .values() to only get the info we need rather than the whole model?

Password change view fails

Environment:


Request Method: GET
Request URL: http://sal.tacos.com/changepassword/

Django Version: 1.10
Python Version: 2.7.6
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'sal',
 'server',
 'api',
 'catalog',
 'inventory',
 'licenses',
 'bootstrap3',
 'watson',
 'datatableview',
 'search',
 'axes')
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'server.middleware.AddToBU.AddToBU',
 'search.current_user.CurrentUserMiddleware',
 'axes.middleware.FailedLoginMiddleware')



Traceback:

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py" in inner
  39.             response = get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
  217.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
  215.                 response = response.render()

File "/usr/local/lib/python2.7/dist-packages/django/template/response.py" in render
  109.             self.content = self.rendered_content

File "/usr/local/lib/python2.7/dist-packages/django/template/response.py" in rendered_content
  84.         template = self.resolve_template(self.template_name)

File "/usr/local/lib/python2.7/dist-packages/django/template/response.py" in resolve_template
  68.             return get_template(template, using=self.using)

File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py" in get_template
  21.             return engine.get_template(template_name)

File "/usr/local/lib/python2.7/dist-packages/django/template/backends/django.py" in get_template
  39.             return Template(self.engine.get_template(template_name), self)

File "/usr/local/lib/python2.7/dist-packages/django/template/engine.py" in get_template
  160.         template, origin = self.find_template(template_name)

File "/usr/local/lib/python2.7/dist-packages/django/template/engine.py" in find_template
  134.                         name, template_dirs=dirs, skip=skip,

File "/usr/local/lib/python2.7/dist-packages/django/template/loaders/base.py" in get_template
  44.                     contents, origin, origin.template_name, self.engine,

File "/usr/local/lib/python2.7/dist-packages/django/template/base.py" in __init__
  191.         self.nodelist = self.compile_nodelist()

File "/usr/local/lib/python2.7/dist-packages/django/template/base.py" in compile_nodelist
  233.             return parser.parse()

File "/usr/local/lib/python2.7/dist-packages/django/template/base.py" in parse
  518.                     raise self.error(token, e)

Exception Type: TemplateSyntaxError at /changepassword/
Exception Value: 'future' is not a registered tag library. Must be one of:
admin_list
admin_modify
admin_static
admin_urls
bootstrap3
cache
dashboard_extras
i18n
l10n
log
static
staticfiles
tz
watson

TemplateSyntaxError at /changepassword/

TemplateSyntaxError at /changepassword/
'future' is not a registered tag library

Currently running 3.0.2.839. First noticed in 2.8 or 2.9 (not sure), upgraded to 3.0.1 per recommendation in Slack which didn't remedy the error. Just upgraded to 3.0.2.839 which presents the error.

http://dpaste.com/11W7TYJ

Installation failure on CentOS 7

I had this happily running for a few weeks using the Docker install instructions on a CentOS 7 machine - but then a yum update broke it.
I updated docker-engine from 1.10.0-1.el7.centos.x86_64 to 1.10.3-1.el7.centos.x86_64 (amongst other things)
Now even if I try to do a fresh install on the same box I get no joy, here is the error:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 351, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 343, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 93, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 19, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/loader.py", line 47, in __init__
    self.build_graph()
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/loader.py", line 182, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/recorder.py", line 59, in applied_migrations
    self.ensure_schema()
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/recorder.py", line 49, in ensure_schema
    if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/base.py", line 164, in cursor
    cursor = self.make_cursor(self._cursor())
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/base.py", line 135, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/base.py", line 130, in ensure_connection
    self.connect()
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/base.py", line 130, in ensure_connection
    self.connect()
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/base.py", line 119, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 204, in get_new_connection
    conn = Database.connect(**conn_params)
django.db.utils.OperationalError: unable to open database file

Issues Loading MunkiInstalls Plugin

MunkiInstalls Plugin just spins. Never loads.

Environment:

Request Method: GET
Request URL: http://sal.mac-bytes.com/pluginload/MunkiInstalls/

Django Version: 1.10
Python Version: 2.7.6
Installed Applications:
('django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin',
'django.contrib.admindocs',
'sal',
'server',
'api',
'catalog',
'inventory',
'licenses',
'bootstrap3',
'watson',
'datatableview',
'search')
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'server.middleware.AddToBU.AddToBU',
'search.current_user.CurrentUserMiddleware')

Traceback:

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py" in inner
39. response = get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _legacy_get_response
249. response = self._get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view
23. return view_func(request, *args, **kwargs)

File "/home/docker/sal/server/views.py" in plugin_load
574. html = plugin.plugin_object.widget_content(page, machines, theID)

File "/home/docker/sal/server/plugins/munkiinstalls/munkiinstalls.py" in widget_content
65. pending = UpdateHistoryItem.objects.filter(status='pending', update_history__machine=machines, recorded__range=(the_min, the_max), update_history__update_type='third_party').count()

File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py" in count
369. return self.query.get_count(using=self.db)

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py" in get_count
476. number = obj.get_aggregation(using, ['__count'])['__count']

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py" in get_aggregation
457. result = compiler.execute_sql(SINGLE)

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py" in execute_sql
835. cursor.execute(sql, params)

File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py" in execute
79. return super(CursorDebugWrapper, self).execute(sql, params)

File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)

File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py" in exit
94. six.reraise(dj_exc_type, dj_exc_value, traceback)

File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)

Exception Type: ProgrammingError at /pluginload/MunkiInstalls/
Exception Value: more than one row returned by a subquery used as an expression

add pkg receipts to stored inventory per client

There are installs of binaries and such that don't show in sys_profiler's apps output, so we'd want to collect the receipts pkgutil knows about. Would be useful for example in cases like the apple receipt of the ethernet-driver-blacklisted-kext fiasco .

migrate.py failing on 2.2.2

I'm trying to upgrade to 2.2.2 from 2.1.1 and am having issues running python manage.py migrate

Here is the error log:

Operations to perform:
  Synchronize unmigrated apps: sal, staticfiles, admindocs, messages, api, bootstrap3
  Apply all migrations: watson, sessions, admin, sites, auth, server, contenttypes, inventory, licenses
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying server.0015_auto_20150819_1501...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 351, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 343, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 148, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 201, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 484, in alter_field
    old_db_params, new_db_params, strict)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 636, in _alter_field
    params,
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 111, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: must be owner of relation server_salsetting

Can not move Computer from one Business Group to another.

delete com.salopensource.sal.plist from client
remove Computer from Business Group "A"
Write a com.salopensource.sal.plist with key from Business Group "B".

Expected result:

Computer should be member of Business Group "B" after submiting to Sal.

Actual result:

Computer is still member of Business Group "A".

Make no difference if i add Machine serial manually in Sal.

Fix Machine.machine_model_friendly = None

Getting the friendly name currently requires a request to Apple. Build a cached table or put in the database the machine_model to machine_model_friendly values and use that, only checking with Apple when required. Or we could just have a static list.

Also, contrive a way to either update missing friendly values on a regular basis (when certain views get Machines?) or during a migration (for the static table perhaps).

I'm planning on looking into this.

Search box

It would be nice to have a search box in the sidebar to quickly locate a machine.

The "All Machines" view showing wrong time zone

I've set the local timezone and the "Latest Run" shows the correct time unless you view "All Machines" globally or for a specific business unit. The machine record in the database also shows the correct time.

Also, if searching for a machine, the table view shows the correct time under "Latest Run"

Expose machine ID with API

Not sure why I hid it. Implement using machine ID with API

Security machine widget not reporting Gatekeeper or Filevault reliably

For some reason, the security widget on the machine detail page isn't always showing the gatekeeper or filevault status correctly.

Application inventory

We should be able to ship application inventory (like you can with MWA).

Gatekeeper status on security machine detail widget is wrong

502 Proxy Error - Red Hat

When attempting to click Application Inventory from the homepage, the page returns an error

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /inventory/.

Reason: Error reading from remote server

When clicking Application Inventory from the /dashboard/2 page, the Inventory comes up fine, with no errors.

The Proxy error also occurs when clicking from the /dashboard/1 page.

Also, here is our sal.conf file:

<VirtualHost *:80>
    ServerName servername.domain.edu
    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</VirtualHost>

And this is our ssl.conf:

Listen 443 https

SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog
SSLSessionCache         shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout  300
SSLRandomSeed startup file:/dev/urandom  256
SSLRandomSeed connect builtin
SSLCryptoDevice builtin

<VirtualHost [server_ip_address]:443>
    ErrorLog logs/ssl_error_log
    TransferLog logs/ssl_access_log
    LogLevel warn

    SSLEngine on
    SSLProtocol all -SSLv2
    SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5:!SEED:!IDEA
    SSLCertificateFile /path/to/certificate
    SSLCertificateKeyFile /path/to/key
    SSLCACertificateFile /path/to/cert

    <Files ~ "\.(cgi|shtml|phtml|php3?)$">
        SSLOptions +StdEnvVars
    </Files>

    <Directory "/var/www/cgi-bin">
        SSLOptions +StdEnvVars
    </Directory>

    BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0
    CustomLog logs/ssl_request_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
    ServerName servername.domain.edu
    Alias /static/ /srv/saluser/sal_env/sal/static/

    <Directory /srv/saluser/sal_env/sal/static/>
        Require all granted
    </Directory>

    ProxyRequests Off
    ProxyPreserveHost On

    <Location />
        ProxyPass http://localhost:8000/
        ProxyPassReverse http://localhost:8000/
        Require all granted
    </Location>

    <Location /static>
        ProxyPass "!"
    </Location>
</VirtualHost>

Machine Details:

Red Hat Enterprise Linux 7
Apache 2.4
Sal 2.7.3.679

Allow 'pinning' of specific sort columns, e.g. `Activity/1-hour` table

Currently computer groups are sorted by last checking for all tables views. For at least the Activity/1-hour view, I'd want it to always be sorted by computername. Some discussion in Slack hinted that this may be possible to implement, conditionally per-user or otherwise.

Redundant requirements in requirements.txt

Double requirement given: cryptography==0.8 (from -r requirements.txt (line 9)) (already in cryptography==0.9.1 (from -r setup/requirements.txt (line 4)), name='cryptography')

Should be only one.

Python Futures seems missing

seems latest quay.io docker image from today is missing a dependency:

quick fix within the docker container docker exec -it sal bash

then installing Futures via pip

/home/docker/sal/plugins# pip install Futures
Collecting Futures
  Downloading futures-3.0.3-py2-none-any.whl
Installing collected packages: Futures
Successfully installed Futures-3.0.3
root@xxxxxxx:/home/docker/sal/plugins# supervisorctl restart gunicorn
gunicorn: ERROR (not running)
gunicorn: started

Allow static files from CDN

There seems to be an issue hosting static files from a CDN. I started using a different storage driver for my static files like here. Some of Sal's templates are hard coded to use /static/ so the static files are not loading from the CDN.

Changing the templates from something like:

<link href="/static/css/bootstrap.min.css" rel="stylesheet">

to something like:

<link href="{% static 'css/bootstrap.min.css' %}" rel="stylesheet">

allows us to set the STATIC_URL to whatever we want in the settings.
If this seems like a reasonable change, I can submit a PR with the changes.

Thanks

Cannot delete machine groups

You can delete business units but not machine groups. This should not be the case.

Search failing if using `operating_system!='

Reproduce:
Search item: Machine
Search field: operating_system
Operator: !=
Search term: 10.12.1

Expected:
Sal should return a list of machine not running 10.12.1

Result

Environment:


Request Method: GET
Request URL: http://REDACTED/search/run_search/3/

Django Version: 1.10
Python Version: 2.7.6
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'sal',
 'server',
 'api',
 'catalog',
 'inventory',
 'licenses',
 'bootstrap3',
 'watson',
 'datatableview',
 'search')
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'server.middleware.AddToBU.AddToBU',
 'search.current_user.CurrentUserMiddleware')



Traceback:

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py" in inner
  39.             response = get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/home/docker/sal/search/views.py" in run_search
  200.     machines = search_machines(search_id, machines)

File "/home/docker/sal/search/views.py" in search_machines
  185.     machines = machines.filter(queries).distinct()

File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py" in filter
  794.         return self._filter_or_exclude(False, *args, **kwargs)

File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py" in _filter_or_exclude
  812.             clone.query.add_q(Q(*args, **kwargs))

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py" in add_q
  1227.         clause, _ = self._add_q(q_object, self.used_aliases)

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py" in _add_q
  1247.                     current_negated, allow_joins, split_subq)

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py" in _add_q
  1253.                     allow_joins=allow_joins, split_subq=split_subq,

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py" in build_filter
  1133.         lookups, parts, reffed_expression = self.solve_lookup_type(arg)

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py" in solve_lookup_type
  1019.         _, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())

File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py" in names_to_path
  1327.                                      "Choices are: %s" % (name, ", ".join(available)))

Exception Type: FieldError at /search/run_search/3/
Exception Value: Cannot resolve keyword u'operating_system!=' into field. Choices are: activity, conditions, console_user, cpu_speed, cpu_type, errors, facts, first_checkin, hd_percent, hd_space, hd_total, historical_facts, hostname, id, install_log, install_log_hash, installed_updates, inventory, inventoryitem, last_checkin, last_puppet_run, machine_group, machine_group_id, machine_model, machine_model_friendly, manifest, memory, memory_kb, munki_version, operating_system, os_family, pending_apple_updates, pending_updates, pluginscriptsubmission, puppet_errors, puppet_version, report, sal_version, serial, updatehistory, warnings

Workaround:
Use < or > instead.

ImportError when building on Heroku

I followed along with the instructions for how to deploy Sal on Heroku. https://github.com/salopensource/sal/blob/master/docs/Deploying_on_Heroku.md

When I run heroku run python manage.py syncdb I get an import error because the settings module cannot be found? Not sure if the sal directory needs to be in the system path. Any help or suggestions for a fix would be appreciated.

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/app/.heroku/python/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/app/.heroku/python/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named settings

Messed up my DB when upgrading

Hi Graham,

I´m stuck... tried to upgrade to the new sal repo and messed up the database:

(sal_env)saluser@macsvc:/usr/local/sal_env/sal$ python manage.py migrate
Operations to perform:
Synchronize unmigrated apps: sal, staticfiles, admindocs, messages, server, api, bootstrap3
Apply all migrations: admin, contenttypes, sites, auth, sessions
Synchronizing apps without migrations:
Creating tables...
Creating table server_salsetting
Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/core/management/init.py", line 338, in execute_from_command_line
utility.execute()
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/core/management/init.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
self.execute(_args, *_cmd_options)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
output = self.handle(_args, *_options)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 179, in handle
created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 309, in sync_apps
editor.create_model(model)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 286, in create_model
self.execute(sql, params or None)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 111, in execute
cursor.execute(sql, params)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/db/utils.py", line 97, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
return self.cursor.execute(sql)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/usr/local/sal_env/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1170, "BLOB/TEXT column 'name' used in key specification without a key length")

Bug in plugin script processing can result in missing foreign key

In the event that a sal-postflight run does not complete during a Munki run, multiple pending results can get appended to the results file, which normally would be deleted. Then during processing during a checkin on a server using Postgres, a logic bug results in deleting a PluginScriptSubmission before the PluginScriptRow items that reference it as a foreign key have been created, due to the unexpected duplication of any plugin's (with historical == False) results.

The proposed solution is to:

Group the plugin script execution, result reading, and result file deletion more closely in the sal-postflight script to minimize the window during which this error can occur.
Perform a check during the server/utils.py process_plugin_script function to handle multiple results correctly.

Pending updates showing older version number when multiple versions are pending.

As discussed on the mailing list, updates are showing in the pending section as an older version (14.5.7) when a newer version is available and approved, and there are clients pending installs of both versions. See attached screenshots. All clients are on version 0.6.1 of the Sal scripts.

	serial = data.get('serial')
	serial = serial.upper()

	# Take out some of the weird junk VMware puts in. Keep an eye out in case Apple actually uses these:
	serial = serial.replace('/', '')
	serial = serial.replace('+', '')

salopensource / sal Goto Github PK

sal's People

Contributors

Stargazers

Watchers

Forkers

sal's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs