Comments (19)
The good news is that periodic snapshots are already there:
https://arensb.github.io/truenas/pool_snapshot_task_module.html#ansible-collections-arensb-truenas-pool-snapshot-task-module
I'll see if I can add SMART and scrub stuff, but I don't know when I'll be able to get to this.
In the meantime, maybe you can help me out by thinking about what these modules should look like: how would you like to write an Ansible playbook for them? What should the parameters be called, so as to make sense to the people using them, and also fit into the Ansible ecosystem. Are there any results that it would make sense to return, besides "it succeeded" or "it failed"?
from ansible-truenas.
If I could configure those, I'd probably do something like this
As far as naming variables, I'd probably try to stay as close as possible to what the Web UI shows.
- https://www.truenas.com/docs/core/uireference/services/smartscreen/
- https://www.truenas.com/docs/core/uireference/tasks/smarttests/
- https://www.truenas.com/docs/core/uireference/tasks/scrubtasks/
---
- name: Configure SMART disk temperature checks
arensb.truenas.smart:
check_interval: 30
power_mode: never
difference: 0
informational_temp: 0
critical_temp: 45
# I'm not sure if the following 2 should be merged into a single array or kept separately
- name: Add SMART short disk periodic tests
arensb.truenas.pool_smart_task:
disks: all
type: short
description: "run SHORT tests on the 7th, 17th and 27th of each month at 1am"
schedule: "0 1 7,17,27 * *"
- name: Add SMART long disk periodic tests
arensb.truenas.pool_smart_task:
disks: da0,da1,da2,da3
type: long
description: "run LONG tests on the 1st and 15th of each month at 1am"
schedule: "0 1 1,15 * *"
# similarly for this one: there can be multiple pools that need scrubs, need to figure out if arrays is what we want
- name: Add monthly scrub tasks to the tank pool
arensb.truenas.pool_scrub_task:
enabled: true
pool: tank
threshold_days: 22
description: "run Scrub tasks on the 19th of each month at 1am"
schedule: "0 1 19 * *"
If those tasks were to allow arrays, here's what I think they could look like
# Here's what it would look like if these allowed arrays
- name: Add SMART disk periodic tests
arensb.truenas.pool_smart_task:
- disks: all
type: short
description: "run SHORT tests on the 7th, 17th and 27th of each month at 1am"
schedule: "0 1 7,17,27 * *"
- disks: da0,da1,da2,da3
type: long
description: "run LONG tests on the 1st and 15th of each month at 1am"
schedule: "0 1 1,15 * *"
lastly, we might need to add state: absent
or state: present
to some of these scheduled tasks
from ansible-truenas.
Thanks. One consideration here is: how can we tell whether a job on the TrueNAS system is the same one as the one defined in an Ansible play? That is, if the play says to run a short SMART test on disk da0
every Monday, and the TrueNAS system currently has one job: a short SMART test on disk da0
every Tuesday, does that mean that the existing job should be changed to run on Mondays instead of Tuesdays? Or that we should add a second job, one that runs on Mondays, and ignore the Tuesday job?
I'm pretty sure I had to solve this same problem for NFS mounts or something, but I'm open to other people's ideas.
from ansible-truenas.
I've added a smart
module on branch feature/pr7-smart
. Usage docstrings are at the top of the module.
Can you please try it out and see if it works for you?
from ansible-truenas.
One consideration here is: how can we tell whether a job on the TrueNAS system is the same one as the one defined in an Ansible play?
The way I see it, there's a couple of ways to go about this.
approach 1
I wouldn't mind if the module required users to add the object id
. The first time users run, if id
is undefined, then CREATE, but if id
is defined, then UPDATE.
If this approach was taken, it would be straightforward, but we'd need really good documentation about it, including how to find out the ID such that users can edit their playbooks accordingly.
Example:
- name: Add SMART disk periodic tests
arensb.truenas.pool_smart_task:
- id: null # <<<<< this will CREATE the object since we haven't defined which object to change.
# Prior to a subsequent run, the user is expected to update the id such that TrueNAS doesn't create duplicates
disks: all
type: short
description: "run SHORT tests on the 7th, 17th and 27th of each month at 1am"
schedule: "0 1 7,17,27 * *"
- id: 2 # <<<< this will update object with = 2, or return an error if it doesn't exist
disks: da0,da1,da2,da3
type: long
description: "run LONG tests on the 1st and 15th of each month at 1am"
schedule: "0 1 1,15 * *"
approach 2
Another approach without specifying ids would be for the module itself to try to create an "primary key" to find the right object. Example: maybe pool_smart_task could use a primary key of "description" (similar to what was done to the snapshots module). Or maybe the pool_scrub_task could have a primary key of "pool+description"
IMO approach 1 is the more straightforward - it may be more difficult for users since they'd need to specify the ids in their playbooks, but this has the least amount of gotchas or surprising behaviors
from ansible-truenas.
I've added a smart module on branch feature/pr7-smart. Usage docstrings are at the top of the module.
Can you please try it out and see if it works for you?
I'll try it out and let you know :)
EDIT
Ok, I've tested it a bit and it's able to set all values, except for "interval". It currently seems to ignore the "interval" setting that was passed in.
This is how I tested:
- name: Configure SMART disk temperature checks
arensb.truenas.smart:
interval: "50"
power_mode: "idle"
temp_difference: "51"
temp_info: "52"
temp_crit: "53"
from ansible-truenas.
Back to my first comment, when choosing between supporting arrays or not supporting arrays, I think it would be simpler if NO arrays were supported.
Looking into some examples how official ansible modules are done, I see this
- name: Add the user 'johnd' with a specific uid and a primary group of 'admin'
ansible.builtin.user:
name: johnd
comment: John Doe
uid: 1040
group: admin
- name: Add the user 'james' with a bash shell, appending the group 'admins' and 'developers' to the user's groups
ansible.builtin.user:
name: james
shell: /bin/bash
groups: admins,developers
append: yes
Notice how each user is modified separately instead of through a single call
from ansible-truenas.
I wouldn't mind if the module required users to add the object
id
.
From what I've seen, the TrueNAS middleware assigns IDs for you, and doesn't allow you to choose them. So that's not an option.
But it does look similar to something I had to do elsewhere, in the sharing_nfs
module, I think: there, you have to specify the path to export, and the export flags. So if the flags on the NAS don't match the ones in the Ansible play, what should the module do?
The way I solved it there was to require a description (similar to the "Description" field in S.M.A.R.T. test tasks). For state == present
, it would look for an export with a matching description.
Also, for scheduled tasks, I'd like to stick to the same parameters as builtin.cron
, for consistency.
from ansible-truenas.
From what I've seen, the TrueNAS middleware assigns IDs for you, and doesn't allow you to choose them. So that's not an option.
right. What I was thinking was when the user wanted to update an existing the object, the user would need to manually edit the playbook to refer to the ID stored in the database. Example, the user would need to run midclt call pool.scrub.query | jq
, take note of the IDs and then write the playbook using the IDs. Also when the user created the item through the playbook, the module should return the new ID.
But I understand that this approach can be complicated for users since they would need to know how to lookup the IDs and also modify their playbooks to differentiate between CREATE and UPDATE.
Either way, @arensb what you're doing is great! I could see it going either way. I'd suggest that once you add these and get a new release out, to definitely share them on reddit and discord. (I've found your post on reddit and that's how I ended up here)
from ansible-truenas.
I've added a new branch, feature/pr7-smart-test-task with a draft smart_test_task
module. Give it a shot and see if it works.
You'll have to figure out the playbook from the docstring at the top. I haven't finished the examples and output sections.
from ansible-truenas.
I've added a new branch, feature/pr7-smart-test-task with a draft smart_test_task module. Give it a shot and see if it works.
Nice! I'll check it out for sure, but it will be a few days until I get to it since I'll be out of town
from ansible-truenas.
done some testing
feature/pr7-smart
@arensb I don't know if you had a chance to see this comment, but this was one issue I had found
Ok, I've tested it a bit and it's able to set all values, except for "interval". It currently seems to ignore the "interval" setting that was passed in.
feature/pr7-smart-test-task
it seems this also has another issue. My config is this
- name: Add SMART short disk periodic tests
arensb.truenas.smart_test_task:
name: "run SHORT tests on the 7th, 17th and 27th of each month at 1am"
disks: ALL
test: short
minute: 0
hour: 1
day: "7,17,27"
month: "*"
weekday: "*"
- name: Add SMART long disk periodic tests
arensb.truenas.smart_test_task:
name: "run LONG tests on the 1st and 15th of each month at 1am"
disks: ada0,ada1
test: long
minute: 0
hour: 1
day: "1,15"
month: "*"
weekday: "*"
but when I run it returns this
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'schedule'
fatal: [[email protected]]: FAILED! => {"changed": false, "module_stderr": "Shared connection to 192.168.122.100 closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n File \"/root/.ansible/tmp/ansible-tmp-1693974349.8145535-3600-172060455326188/AnsiballZ_smart_test_task.py\", line 107, in <module>\r\n _ansiballz_main()\r\n File \"/root/.ansible/tmp/ansible-tmp-1693974349.8145535-3600-172060455326188/AnsiballZ_smart_test_task.py\", line 99, in _ansiballz_main\r\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n File \"/root/.ansible/tmp/ansible-tmp-1693974349.8145535-3600-172060455326188/AnsiballZ_smart_test_task.py\", line 47, in invoke_module\r\n runpy.run_module(mod_name='ansible_collections.arensb.truenas.plugins.modules.smart_test_task', init_globals=dict(_module_fqn='ansible_collections.arensb.truenas.plugins.modules.smart_test_task', _modlib_path=modlib_path),\r\n File \"/usr/local/lib/python3.9/runpy.py\", line 225, in run_module\r\n return _run_module_code(code, init_globals, run_name, mod_spec)\r\n File \"/usr/local/lib/python3.9/runpy.py\", line 97, in _run_module_code\r\n _run_code(code, mod_globals, init_globals,\r\n File \"/usr/local/lib/python3.9/runpy.py\", line 87, in _run_code\r\n exec(code, run_globals)\r\n File \"/tmp/ansible_arensb.truenas.smart_test_task_payload_r380z7wa/ansible_arensb.truenas.smart_test_task_payload.zip/ansible_collections/arensb/truenas/plugins/modules/smart_test_task.py\", line 382, in <module>\r\n File \"/tmp/ansible_arensb.truenas.smart_test_task_payload_r380z7wa/ansible_arensb.truenas.smart_test_task_payload.zip/ansible_collections/arensb/truenas/plugins/modules/smart_test_task.py\", line 237, in main\r\nKeyError: 'schedule'\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
this was fixed with adding this line
# Collect arguments to pass to resource.create()
arg = {
'desc': name,
+ 'schedule': {},
}
feedback
IMO the ID that ansible uses in smart_test_task.py
may not be best to use description
since the UI doesn't even require that parameter. I think it would be better if it was possible to make the ID based on DISKS+TYPE, but that has it's own cons
# currently this is used
smart_test_info = mw.call("smart.test.query",
[["desc", "=", name]])
# maybe could be something like this - the challenge with the following is that the disks might not be ordered the same? (I haven't checked if the order is always the same)
$ midclt call smart.test.query '[["type", "=", "LONG"], ["disks", "=", ["{serial}555666777", "{serial}111222333"]]]'
But there's pros/cons to each approach, for example
- id: description --> simpler, but when a user changes the description and runs the playbook, a new entry is created
- id: disks+type --> more complex, but when a user adds more disks and runs the playbook, a new entry is created
I suppose using the description as an id should be fine as long as documentation is top notch
feature/pr7-scrub-task
looks good ✔️
from ansible-truenas.
It currently seems to ignore the "interval" setting that was passed in.
D'oh! This is fixed now, on branch feature/pr7-smart.
from ansible-truenas.
I was able to reproduce the bug in feature/pr7-smart-test-task, and fix it. As you suspected, it had to do with a messed-up schedule
field. (Or, more precisely, I found a bug in the schedule-processing code while editing the "if the resource exists" section, but then didn't apply the fix correctly to the "if we need to create the resource" section.) Please give it another try.
IMO the ID that ansible uses in smart_test_task.py may not be best to use description since the UI doesn't even require that parameter. I think it would be better if it was possible to make the ID based on DISKS+TYPE, but that has it's own cons
It looks as though the required fields to create a S.M.A.R.T. Test task are the test type ("LONG", "SHORT", etc.) and either all_disks
or a list of disks. However, that does not uniquely identify a job: you can have several LONG
jobs on disk ada0
.
In fact, there may be valid reasons for doing this: maybe you have several TrueNAS machines, and you want to run a SHORT
test on ada0,ada1
on all machines on the 1st of each month. But some of them are backup servers that keep user data on those disks, and you want to test them on the 15th of each month as well:
- hosts: truenas-hosts
tasks:
- arensb.truenas.smart_test_task:
disks: [ ada0, ada1 ]
test: short
day: "1"
- hosts: backup-servers
tasks:
- arensb.truenas.smart_test_task:
disks: [ ada0, ada1 ]
test: short
day: "15"
Assuming that backup-servers
is a subset of truenas-hosts
, we now have some hosts with two jobs with the same disks and type.
The purpose of an identifier is to be able to point to something (a task, in this case) and say "there: that's the one I'm talking about". That seems easier to do with the description field than with any other field or combination of fields. Yes, it means that you can't change the description without deleting and recreating the task, but hopefully that won't happen too often.
A bigger problem, to my mind, is that this module doesn't allow you to delete all tasks and reset the machine to a known state where the only tasks are the one defined in your playbook.
from ansible-truenas.
That seems easier to do with the description field than with any other field or combination of fields. Yes, it means that you can't change the description without deleting and recreating the task, but hopefully that won't happen too often.
Agreed. that seems the most simple and straightforward way to handle this
[I fixed it,] please give it another try.
will do :)
DONE - looks good now
from ansible-truenas.
DONE - looks good now
Thanks. I think that just leaves the S.M.A.R.T. service itself. Have you had a chance to try that out?
from ansible-truenas.
I did. The updated change that fixes the interval works! ✔️
from ansible-truenas.
while I have your attention @arensb, I've opened another much smaller request #8
from ansible-truenas.
closed with v1.8.0
from ansible-truenas.
Related Issues (14)
- user set sudo don't work on truenas scale HOT 10
- Create a test suite
- truenas.service doesn't start nor enables services HOT 6
- nfs module - additional parameters HOT 2
- truenas.services names HOT 1
- Manage ssh host keys and certs HOT 5
- users module should allow smb parameter when password_disabled is defined HOT 13
- Exampe code for nfs module raises error when setting nfsv4 HOT 8
- Can't set an NFS Share to read only HOT 3
- Can't have the same NFS mount point with different permissions HOT 1
- Can't create password_disabled user if smb is enabled on the machine HOT 2
- Can't set owner of dataset using 'arensb.truenas.filesystem' HOT 4
- Incorrect attribute name in pool_scrub_task module HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ansible-truenas.