nornir-automation / nornir_napalm

NAPALM's plugins for nornir

nornir_napalm

Collection of simple plugins for nornir

Installation

pip install nornir_napalm

Documentation

You can access the plugin documentation at https://nornir.tech/nornir_napalm

Plugins

Connections

• napalm - Connect to network devices using napalm

Tasks

• napalm_cli - Call napalm's cli method
• napalm_configure - Call napalm's configure method. It also allows committing/discarding configurations.
• napalm_get - Call napalm's get_* methods
• napalm_ping - Call napalm's ping method`
• napalm_validate - Call napalm's validate method

nornir_napalm's Issues

Return "Request contains invalid special characters" while using commands with "|"

Hi guys, hope everything finds you good.
I'm trying to use nornir to run show running-config | grep -v ^! to get the running configs without annotations. My code is more like:

nr = InitNornir(runner=runner, inventory=inventory, logging=nornir_logging)
cmds = ['show running-config | grep -v ^!']
nr.run(
   name=f'Running commands of {cmds}.',
   task=napalm_cli,
   commands=cmds,
)

But got: "Request contains invalid special characters". I've checked and found this:

This situation occurs because the Sandbox parses each command in a JSON RPC request as individual items and assigns an ID to each. When using JSON RPC requests, you cannot use internal punctuation to separate multiple commands on the same line. Instead, enter each command on a separate line and the request completes sucessfully.

Would you please help me that how could I use json requests in nornir_napalm? Or is there any other way that I could get what show running-config | grep -v ^! get without using any invalid syntax?
Thanks a lot!

Display of changed for 'napalm_configure' is always True, even when forced to False

nornir==3.3.0
nornir-napalm==0.2.0
nornir-utils==0.2.0
Python 3.8.10

--
I'm new to nornir, so I'm not sure yet how the changed: True | False works exactly yet. Is this related to the dry_run variable? Maybe this is not relevant as an issue...

How to reproduce:

1. I run the following task, in dry_run, so I am not expecting any changes on the device.

task.run(
        task=napalm_configure, dry_run=True, configuration="interface loopback 1000"
    )

2. And in the output, I see:
   ---- napalm_configure ** changed : True ---------------------------------------- INFO
   I would expect to see False. Is this normal?

nornir_napalm with nxos_ssh platform requires invoke?

Ran into a weird one today. With the following config and inventory:

---
# config.yaml
inventory:
  plugin: SimpleInventory
runner:
  plugin: threaded
  options:
    num_workers: 4

---
# hosts.yaml
example.dal:
  hostname: example.dal.example.com
  port: 22
  platform: nxos_ssh

And the following Python:

import logging
import os

from nornir_napalm.plugins.tasks import napalm_get
from nornir import InitNornir

nr = InitNornir(config_file="config.yaml", inventory={
    "options": {
        "host_file": "hosts.yaml",
    },
})
nr.inventory.defaults.username = os.getenv("NORNIR_USERNAME")
nr.inventory.defaults.password = os.getenv("NORNIR_PASSWORD")
r = nr.run(task=napalm_get, getters=["config"], severity_level=logging.DEBUG)

I get a ModuleNotFound that invoke couldn't be found. Confused why invoke would be necessary, I tried with straight napalm, using the following code:

from napalm import get_network_driver

driver = get_network_driver('nxos_ssh')
device = driver('example.dal.example.com', os.getenv("NAPALM_USERNAME"), os.getenv("NAPALM_PASSWORD"))
device.open()
device.get_config()

The above code using napalm directly works flawlessly. Using the nxos platform in nornir_napalm works without issue. But, for some reason, using nxos_ssh platform in nornir_napalm requires invoke (at least in my testing).

After installing invoke package, nornir_napalm works without any problem. I see it in the Poetry.lock, but I can't find any other references to it in this repo.

Versions of packages are below:

$ pip freeze | grep -i "nornir\|napalm"
napalm==4.0.0
nornir==3.3.0
nornir-napalm==0.3.0

Very slow device (Cisco IOS) fails to return diff on a configuration dry run

Hello, Can someone please help to fix the below issue?

I am trying to push the configuration from the .txt file to the cisco devices using the napalm_configure plugin. However, I am getting the error message "Subtask: napalm_configure (failed)". I assume the config file, host file, defaults file, group files are good because I can run the other scripts.

The following plugins are installed on my machine.
nornir==3.0.0
nornir-jinja2==0.1.0
nornir-napalm==0.1.1
nornir-netmiko==0.1.1
nornir-scrapli==2020.11.1
nornir-utils==0.1.1

+++++++++++++++Script++++++++++

from nornir import InitNornir
from nornir_napalm.plugins.tasks import napalm_configure
from nornir_utils.plugins.functions import print_result

nr = InitNornir(config_file="config.yaml", dry_run=True)

def dryrun(task):
    task.run(task=napalm_configure, filename="configuration_file.txt")   

results = nr.run(task=dryrun)
print_result(results)

++++++++Error Traceback++++++++
Subtask: napalm_configure (failed)

---- napalm_configure ** changed : False --------------------------------------- ERROR
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/nornir/core/task.py", line 98, in start
r = self.task(self, **self.params)
File "/usr/local/lib/python3.8/dist-packages/nornir_napalm/plugins/tasks/napalm_configure.py", line 35, in napalm_configure
diff = device.compare_config()
File "/usr/local/lib/python3.8/dist-packages/napalm/ios/ios.py", line 429, in compare_config
diff = self.device.send_command_expect(cmd)
File "/usr/local/lib/python3.8/dist-packages/netmiko/base_connection.py", line 1576, in send_command_expect
return self.send_command(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/netmiko/utilities.py", line 430, in wrapper_decorator
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/netmiko/base_connection.py", line 1529, in send_command
raise IOError(
OSError: Search pattern never detected in send_command: SW2#

OSError: Search pattern never detected in send_command

Hi,

I am trying to replace an ACL configuration on Cisco IOS with Nornir and NAPALM but I'm getting this error :

Error:

Traceback (most recent call last):
File "/home/carl/.local/lib/python3.8/site-packages/nornir/core/task.py", line 99, in start
r = self.task(self, **self.params)
File "/home/carl/.local/lib/python3.8/site-packages/nornir_napalm/plugins/tasks/napalm_configure.py", line 39, in napalm_configure
device.commit_config()
File "/home/carl/.local/lib/python3.8/site-packages/napalm/ios/ios.py", line 531, in commit_config
output = self._commit_handler(cmd)
File "/home/carl/.local/lib/python3.8/site-packages/napalm/ios/ios.py", line 470, in wrapper
return f(self, *args, **kwargs)
File "/home/carl/.local/lib/python3.8/site-packages/napalm/ios/ios.py", line 487, in _commit_handler
output = self.device.send_command_expect(cmd, expect_string=patterns)
File "/home/carl/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 1582, in send_command_expect
return self.send_command(*args, **kwargs)
File "/home/carl/.local/lib/python3.8/site-packages/netmiko/utilities.py", line 500, in wrapper_decorator
return func(self, *args, *kwargs)
File "/home/carl/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 1535, in send_command
raise IOError(
OSError: Search pattern never detected in send_command: (?:[>##]\s$|.*all username.*confirm)

Script:

import logging
import re
from nornir import InitNornir
from nornir_napalm.plugins.tasks import napalm_get
from nornir_napalm.plugins.tasks import napalm_configure
from nornir_jinja2.plugins.tasks import template_file
from nornir_utils.plugins.tasks.data import load_yaml
from nornir_utils.plugins.functions import print_result

nr = InitNornir(config_file="config.yaml")

etx = chr(3)

def replace_acl(task):
data = task.run(
task=load_yaml,
file=f"./host_vars/{task.host}.yaml",
severity_level=logging.DEBUG,
)
task.host["facts"] = data.result
config = task.run(task=napalm_get, getters=["config"], severity_level=logging.DEBUG)
showrun = config.result["config"]["running"]
pattern = re.compile("^ip access-list([^!]+)", flags=re.I | re.M)
routing_template = task.run(
task=template_file,
name="Building ACL Configuration",
template="acl.j2",
path="./templates",
)
template_to_load = routing_template.result
newconfig = re.sub(pattern, template_to_load, showrun)
final_config = newconfig.replace("^C", etx)
return final_config

def replace_feature(task):
config = replace_acl(task)
task.run(task=napalm_configure, configuration=config, replace=True)

results = nr.run(task=replace_feature)
print_result(results)

acl.j2 :

ip access-list standard SSH_IN
permit 172.21.{{ host.facts.subnet }}.0 0.0.0.255
permit 172.24.11.0 0.0.0.255
permit 172.24.40.0 0.0.3.255
deny any

R1.yaml :

subnet: 101

Info

Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.8(3)M

Any ideas ?

Thanks

Bump ruamel.yaml to match latest nornir dependency version requirements

nornir_napalm/poetry.lock

Lines 1219 to 1225 in 5f22d3e

	[[package]]
	category = "main"
	description = "ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order"
	name = "ruamel.yaml"
	optional = false
	python-versions = "*"
	version = "0.16.10"

vs

https://github.com/nornir-automation/nornir/blob/5cca4a8a579eecbd33c88304da6e451dc92e0922/poetry.lock#L1332-L1337

Connection is never closed by napalm_configure

Dear maintaners,

I am developing a Nornir based application that pushes configuration to IOS-XR devices via napalm_configure and i've noticed that - from time to time - all of the VTYs on the devices in my testing environment get consumed.

While investigating, i have noticed that a connection to the device is open by napalm_configure, but it is never closed (at least not by this plugin)

So, If i am understanding it correctly, then it is up to the device to close connection. While it would be smarter it the client would do so.
It would be quite and easy change to make within napalm_configure. But i am not 100% sure if that's the right point or if a connection should be closed elsewhere.

Would you mind to explain?

Thank you

Regards

Connections to IOS XR via Netconf driver fail due to ssh_config_file argument

When I am running nornir_napalm with the iosxr_netconf driver the connection fails, because nornir_napalm adds a argument called ssh_config_file that is passed on to ncclient in NAPALM, which does not expect that argument.

Versions used:

napalm==4.1.0
ncclient==0.6.15
nornir==3.4.1
nornir-napalm==0.4.0
nornir-utils==0.2.0

Here is my code:

from nornir import InitNornir
from nornir_utils.plugins.functions import print_result
from nornir.core.plugins.connections import ConnectionPluginRegister
from nornir_napalm.plugins.connections import Napalm
from nornir_napalm.plugins.tasks import napalm_get

ConnectionPluginRegister.register("nornir-napalm", Napalm)

nr = InitNornir(config_file="config.yaml")

result = nr.run(task=napalm_get, getters=["get_config"])
print_result(result)

With this config:

inventory:
  plugin: SimpleInventory
  options:
    host_file: "inventory/hosts.yaml"
    group_file: "inventory/groups.yaml"
    defaults_file: "inventory/defaults.yaml"

runner:
  plugin: threaded
  options:
    num_workers: 20 

And this inventory:

p-1:
  hostname: clab-isis-p-1
  platform: iosxr_netconf

This results in this error:

SSHSession.connect() got an unexpected keyword argument 'ssh_config_file'
napalm_get**********************************************************************
* p-1 ** changed : False *******************************************************
vvvv napalm_get ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ERROR
Traceback (most recent call last):
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/napalm/iosxr_netconf/iosxr_netconf.py", line 83, in open
    self.device = manager.connect(
                  ^^^^^^^^^^^^^^^^
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/ncclient/manager.py", line 187, in connect
    return connect_ssh(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/ncclient/manager.py", line 139, in connect_ssh
    session.connect(*args, **kwds)
TypeError: SSHSession.connect() got an unexpected keyword argument 'ssh_config_file'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/nornir/core/task.py", line 99, in start
    r = self.task(self, **self.params)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/nornir_napalm/plugins/tasks/napalm_get.py", line 32, in napalm_get
    device = task.host.get_connection(CONNECTION_NAME, task.nornir.config)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/nornir/core/inventory.py", line 494, in get_connection
    self.open_connection(
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/nornir/core/inventory.py", line 546, in open_connection
    conn_obj.open(
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/nornir_napalm/plugins/connections/__init__.py", line 57, in open
    connection.open()
  File "/home/jo/projects/isis-tests/nornir/venv/lib/python3.11/site-packages/napalm/iosxr_netconf/iosxr_netconf.py", line 97, in open
    raise ConnectionException(conn_err.args[0])
napalm.base.exceptions.ConnectionException: SSHSession.connect() got an unexpected keyword argument 'ssh_config_file'

Change encoding for eapi responses to json

Is it possible to set the desired encoding type returned by eapi. It seems to default to 'text', however json would be much more desirable. In napalm I normally do this via cmd = device.cli(['show interfaces transceiver detail'], encoding='json')

Is there a way to pass encoding='json' to cli in nornir_napalm?

Connecting through a Jump Server (optional_args)

Hi everyone,
I'm trying to connect to my ios net devices using Nornir_napalm plugin, Nornir and napalm with linux jump server in the middle. I know that I need the "optional_args" dict to be passed to Napalm so as to set the ssh config for the underlying Netmiko lib.

Since I'm not using Napalm directly, how do I pass the "optional_args" to the Napalm via the plugin nornir_napalm ?

nornir using a napalm extended driver

Hello,

I extended a few napalm modules with a get_facts_custom and I am able to have nornir call it without any issue:

results = devices.run(task=napalm_get, getters=["facts_custom"])
print_result(results)

All is well in the world there.

I just created a new function on the napalm modules and wanted to have nornir use them as well. The function is called copy_run_start which works when using pure napalm.

I tried using napalm_cli from nornir_napalm.plugins but failed:

results = devices.run(task=napalm_cli, commands=copy_run_start)

with this error: NameError: name 'copy_run_start' is not defined

Here is the napalm extended driver for ios :

from napalm.ios.ios import IOSDriver

class CustomIOSDriver(IOSDriver):
    """Custom NAPALM Cisco IOS Handler."""

    # save config to startup
    def copy_run_start(self):
        command = 'copy running-config startup-config'
        output = self._send_command(command)

        return output

How can I have nornir call this custom napalm function?

Any input is much appreciated!

Versions : Python 3.9.6

nornir                       3.2.0
nornir-napalm                0.2.0
nornir-utils                 0.2.0
napalm                       3.4.1

Thank you,
Dave

Not pulling diff with existing open connection to Juniper

Hello,

I ran into an issue where if I open a connection with a task, and do not close that connection, if I run another task (specifically in this case a dry run of a configuration push), I do not receive a diff back, even though I see the configuration being pushed correctly and rolled back.

Snippet of what I am running:
facts = task.run(task=napalm_get, getters=["get_facts"])

This opens the napalm connection to my device, and it stays open when my code gets to this portion:

deploy_configuration = task.run(task=napalm_configure, dry_run=True, configuration=task.host["full_template"], replace=True)

Now when I am doing a debug, I see that the connection is still open under nornir, but never receive the diff back. Once I add a close connection after the "get_facts" line, it all works cleanly. Ideally I would like to use the same ssh connection because while this works on single devices, when I am running it against several devices, I will be opening a lot of unnecessary ssh sessions.

Line I am adding to correct:
facts = task.run(task=napalm_get, getters=["get_facts"])
task.host.close_connection("napalm")

Thanks.

Load napalm-panos in nornir

Hi, guys. I'm wondering if there's a way to load naplm-panos in nornir? From getter-support-matrix I've noticed that we don't support palo alto devices. So napalm-panos seems to be the only way for getting information from them. Or is there any way to work around?

Thanks!

NAPALM 5.0.0 support

Yesterday NAPALM v5.0.0 was released.

Can we support versions 4 and 5 in the next version? Users could pin the NAPALM version in the requirements.
Besides dropping Python 3.7, the changes in NAPALM depend on how the user uses the library/tasks, but the interfaces this plugin uses remain the same.

My suggestion would be to:

• Drop Python 3.7 support as well
• change the version-pin to napalm = "^4|^5"
• Update the version to 0.5.0

What do you think would be the best solution?

no confirm_commit in napalm_configure

Hallo!

i'm using napalm_configure with revert_in to merge config.

if the rever_in timer is set, in the IOSDriver.commit_config method there is no device.confirm_commit().

every time system will rollback to the backup config.

if revert_in_min is None:

there is nothing to do if the revert_in_min is True!

It needs to be in `connection_options` format so for `groups.yaml` it would be similar to the following:

          It needs to be in `connection_options` format so for `groups.yaml` it would be similar to the following:

my_group:
  platform: nxos
  connection_options:
    napalm:
      extras:
        optional_args:
          fast_cli: False
          secret: my_secret

Originally posted by @ktbyers in #17 (comment)

Passing enable secret to NAPALM IOS driver via inventory and directly via ConnectOptions

Dears,

I am using Napalm on the Nornir framework and the script works fine if I configure the privilege level 15 password on the Cisco Device. However, on the production devices, by default it goes to the User EXEC(Switch>) mode, then I have to manually type "enable" to go into the Privileged EXEC mode(Switch#).

Is there any alternate option, to pass the secret password similar to the get.pass method?

Since I use the Nornir framework, I pass the 'secret' argument in groups.yaml file, however, it doesn't work. Is it possible to define the "secret" argument in an encrypted format rather than defining it in cleartext?

cisco_group:
platform: "ios"
connection_options:
napalm:
extras:
optional_args:
fast_cli: False
secret: cisco

Thanks in Advance

Search pattern never detected in send_command_expect

I'm new to Nornir. I'm trying to grab "get_facts" from multiple telnet devices. Most of the devices return results as expected. I'm just facing issue for a couple of devices (C4510 - IOS-XE 03.02.05.SG & C2960 - 15.2(4)E6). I'm receiving below error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 204, in _send_command
    output = self.device.send_command(command)
  File "/usr/local/lib/python3.6/site-packages/netmiko/utilities.py", line 347, in wrapper_decorator
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/netmiko/base_connection.py", line 1431, in send_command
    search_pattern
OSError: Search pattern never detected in send_command_expect: LO\-CO\-EXT\-SW1\#

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/nornir/core/task.py", line 98, in start
    r = self.task(self, **self.params)
  File "/usr/local/lib/python3.6/site-packages/nornir_napalm/plugins/tasks/napalm_get.py", line 44, in napalm_get
    result[g] = method(**options)
  File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 970, in get_facts
    show_ver = self._send_command("show version")
  File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 207, in _send_command
    raise ConnectionClosedException(str(e))
napalm.base.exceptions.ConnectionClosedException: Search pattern never detected in send_command_expect: LO\-CO\-EXT\-SW1\#

I tried adding 'global_delay_factor' to the optional_args but anything greater than 3 changes the error to below:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 204, in _send_command
    output = self.device.send_command(command)
  File "/usr/local/lib/python3.6/site-packages/netmiko/utilities.py", line 347, in wrapper_decorator
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/netmiko/base_connection.py", line 1427, in send_command
    new_data = self.read_channel()
  File "/usr/local/lib/python3.6/site-packages/netmiko/base_connection.py", line 503, in read_channel
    output = self._read_channel()
  File "/usr/local/lib/python3.6/site-packages/netmiko/base_connection.py", line 485, in _read_channel
    output = self.remote_conn.read_very_eager().decode("utf-8", "ignore")
  File "/usr/lib64/python3.6/telnetlib.py", line 366, in read_very_eager
    return self.read_very_lazy()
  File "/usr/lib64/python3.6/telnetlib.py", line 403, in read_very_lazy
    raise EOFError('telnet connection closed')
EOFError: telnet connection closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/nornir/core/task.py", line 98, in start
    r = self.task(self, **self.params)
  File "/usr/local/lib/python3.6/site-packages/nornir_napalm/plugins/tasks/napalm_get.py", line 44, in napalm_get
    result[g] = method(**options)
  File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 970, in get_facts
    show_ver = self._send_command("show version")
  File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 207, in _send_command
    raise ConnectionClosedException(str(e))
napalm.base.exceptions.ConnectionClosedException: telnet connection closed

Below are the connection_options I'm using for NAPALM:

napalm:
  platform: ios
  extras:
    optional_args:
      transport: telnet
      secret: 'secretkey'
      global_delay_factor: 3