Multiple Getters Failing on IOS-XE (ASR1001) about napalm-ios

[dbarrosop]

Are you shitting me? Is Cisco that st%&$d? They don't know that an IP represented as a string can have a max_length of 15 counting the dots? Interesting a networking vendor doesn't know about that.

[mirceaulinic]

*192.168.123.100
                4        12345    8783    8788    13061    0    0 1d10h           1

Whut. the. f!? That's beyond my imagination!

[itdependsnetworks]

One way to "normalize" the data:

out = '''
*192.168.123.97 4        12345   20902   20895    13061    0    0 3d09h           1
*192.168.123.98 4        12345   967     971    13061    0    0 03:45:19        2
*192.168.123.99 4        12345   18277   18271    13061    0    0 2d23h           2
*192.168.123.100
                4        12345    8783    8788    13061    0    0 1d10h           1
*192.168.123.101
                4        12345   16235   16202    13061    0    0 2d15h           2
*192.168.123.102
                4        12345   21769   21796    13061    0    0 3d12h           2
*192.168.123.103
                4        12345    2284    2293    13061    0    0 08:53:20        2
'''
output = out.split('\n')

count = 1
regexp = re.compile(r'^(?:\*|\s+)(?:\d{3}\.){3}\d{3}')
tempoutput = []
  
for line in output:
    if len(line.split()) == 9:
        pass
    elif regexp.search(line) is not None and len(output[count].split()) == 9:
        tempoutput.append(line.strip() + output[count])
    else:
        tempoutput.append(line)
    count = count + 1

output = tempoutput

[itdependsnetworks]

Added this in slack, but thought I would comment here, should probably have: peer_id = peer_id.replace('*','') as well.

[ktbyers]

Can we just re.sub based on IP address pattern with a trailing newline and then eliminate the newline?

@itdependsnetworks Sounds good on your peer_id replacement as well?

[tyler-8]

@ktbyers That would actually work quite well if done before the linesplit. Replace the trailing newline with a space and good to go. Genius!

I still need to figure out why the other getters are failing.

[tyler-8]

get_facts() fails due to show ip interface brief on IOS-XE also returns load and time source stats. Fix is show ip interface brief | begin ^Interface

Here's the output:

#show ip interface brief
Load for five secs: 1%/0%; one minute: 1%; five minutes: 1%
Time source is NTP, *09:59:40.324 UTC Wed Nov 30 2016

Interface              IP-Address      OK? Method Status                Protocol

[itdependsnetworks]

@ktbyers

I'm not sure how that will work, in my mind it will still be two different elements in the same list, though I am sure I am missing something. If it works, then all the better. Was just offering up at potential solution being that it only took me 5-10 min, but again, whatever works :)

[tyler-8]

@itdependsnetworks: What @ktbyers is proposing is eliminating the newlines following the IP before the .split("\n") function is performed. That should make the linesplit piece execute without issue.

[ktbyers]

@itdependsnetworks It's all good. I will try to do a few tests.

[itdependsnetworks]

@tyler-8 @ktbyers brilliant :)

output = re.sub(r'((?:\*|\s+)(?:\d{3}\.){3}\d{3}\s*)(?:\n|\r)', r'\1', output)

[tyler-8]

get_arp_table() has the same issue as get_facts(). show ip arp output gives load and NTP stats.

Raw output:

#sh ip arp
Load for five secs: 1%/0%; one minute: 1%; five minutes: 0%
Time source is NTP, *11:57:42.689 UTC Wed Nov 30 2016

Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  207.11.25.233          12   0010.dbff.1001  ARPA   GigabitEthernet0/0/0

Here's the fix:
sh ip arp | begin ^Protocol

[ktbyers]

@tyler-8 Okay, I will try to fix these in the same PR request.

[tyler-8]

@ktbyers - Okay confirmed for sure. These functions are breaking due to the same lines in the output.

device.get_facts()
device.get_arp_table()
device.get_interfaces()
device.get_interfaces_ip()

Adding the output modifiers to | begin ^Protocol or | begin ^Interface will fix it for all IOS devices.

[tyler-8]

I just saw that the get_arp_table() is already using an output modifier.
command = 'show arp | exclude Incomplete'

So for that command it'll have to be something like sh ip arp | exclude Incomplete|Load for|Time source but that still leaves a blank line at the beginning of the output. I'm not sure if there's a way to filter out blank lines in the CLI so that may be for NAPALM to do.

router#sh ip arp | exclude Incomplete|Load for|Time source  

Protocol  Address          Age (min)  Hardware Addr   Type   Interface

[tyler-8]

Thanks to @ogenstad

Try typing terminal no exec prompt timestamp, before the command.
You probably have something in your config to set that.

Which turned out to be the case for all the getters with issues except device.get_bgp_neighbors() - which still breaks due to IOS wrapping the neighbor lines.

Performing >>> device.cli(["terminal no exec prompt timestamp"]) prior to executing the other getters solves the problems.

[ktbyers]

@tyler-8 Yes, I figured it was probably a CLI setting for the ones excluding BGP.

It is probably worth fixing, however, so NAPALM just filters those lines out.

[tyler-8]

Found some additional silliness, though netmiko may already be handling this by doing terminal length 0.

When encountering -- More --, IOS will insert additional header rows, which can split up the peer-id from its BGP data.

*192.168.123.100
                4        12345    1941    1943    13847    0    0 07:32:54        1
*192.168.123.101
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
                4        12345    3768    3769    13847    0    0 14:39:49        2
*192.168.123.102
                4        12345    1098    1101    13847    0    0 04:16:13        1

[tyler-8]

Here's my regex stab at removing the line breaks/feeds after the peer-id:

line_merge = re.compile(r"((?:\d{1,3}\.){3}\d{1,3}\s*)(?:\n|\r)(.+)")
output = line_merge.sub(r"\1\2", bgp_output.strip())

[ktbyers]

Will try to fix next week: 12/5 - 12/9

[dbarrosop]

Would be nice to migrate to the new testing framework and include this test cases before fixing. Will double check with @ogenstad as I think he was working on it and help him out to have it ready ASAP.

[ktbyers]

I submitted a PR on this including tests here:

#78

[ktbyers]

@tyler-8 Can you test this fix works with your BGP line wrapping issue.

[ktbyers]

I also put in a fix that should filter these lines:

Load for five secs: 1%/0%; one minute: 1%; five minutes: 1%
Time source is NTP, *09:59:40.324 UTC Wed Nov 30 2016