dainok / netdoc Goto Github PK
View Code? Open in Web Editor NEWAutomatic Network Documentation plugin for NetBox
License: GNU General Public License v3.0
Automatic Network Documentation plugin for NetBox
License: GNU General Public License v3.0
Had to make a couple adjustments to account for HMM routes on NXOS and figured it might be good to bring these into the next release:
IP Route Table for VRF "overlay1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
0.0.0.0/0, ubest/mbest: 2/0
*via 10.13.0.1%default, [200/0], 9w5d, bgp-63001, internal, tag 63001, segid: 30001 tunnelid: 0xa0d0001 encap: VXLAN
*via 10.13.0.9%default, [200/0], 9w5d, bgp-63001, internal, tag 63001, segid: 30001 tunnelid: 0xa0d0009 encap: VXLAN
11.101.0.0/16, ubest/mbest: 1/0, attached
*via 11.101.0.1, Vlan101, [0/0], 10w0d, direct, tag 12345
11.101.0.1/32, ubest/mbest: 1/0, attached
*via 11.101.0.1, Vlan101, [0/0], 10w0d, local, tag 12345
11.101.0.10/32, ubest/mbest: 1/0, attached
*via 11.101.0.10, Vlan101, [190/0], 10w0d, hmm
{'distance': '190',
'encap': '',
'mask': '32',
'metric': '0',
'network': '11.101.0.10',
'nexthop_if': 'Vlan101',
'nexthop_ip': '11.101.0.10',
'nexthop_vrf': '',
'protocol': 'hmm',
'segid': '',
'tag': '',
'tunnelid': '',
'type': '',
'uptime': '10w0d',
'vrf': 'overlay1'}]
/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py:
def normalize_route_type(route_type):
"""Return route type protocol."""
route_type = route_type.lower()
**if route_type in ["hmm"]:
# Nexus Host Mobility Manager
return "hmm"**
/opt/netbox/venv/lib/python3.10/site-packages/netdoc/models.py:
class RouteTypeChoices(ChoiceSet):
"""Route type."""
CHOICES = [
("u", "Unknown"),
("b", "BGP"),
("c", "Connected"),
("s", "Static"),
("u", "User-space"),
("r", "RIP"),
("e", "EIGRP"),
("ex", "EIGRP external"),
("oia", "OSPF inter area"),
("on1", "OSPF NSSA external type 1"),
("on2", "OSPF NSSA external type 2"),
("oe1", "OSPF external type 1"),
("oe2", "OSPF external type 2"),
("i", "IS-IS"),
("is", "IS-IS summary"),
("i1", "IS-IS level-1"),
("i2", "IS-IS level-2"),
**("hmm", "Host Mobility Manager"),**
An exception occurred: UnboundLocalError: cannot access local variable 'parent_name' where it is not associated with a value
Traceback (most recent call last):
File "/opt/netbox-3.5.9/netbox/extras/scripts.py", line 504, in _run_script
script.output = script.run(data=data, commit=commit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox-3.5.9/netbox/scripts/netdoc_scripts.py", line 353, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.11/site-packages/netdoc/utils.py", line 578, in log_ingest
module.ingest(log)
File "/opt/netbox/venv/lib/python3.11/site-packages/netdoc/ingestors/xml_panw_ngfw_show_interface.py", line 108, in ingest
if parent_name:
^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'parent_name' where it is not associated with a value
Ingest FHRP details.
An exception occurred: ValueError: Invalid interface speed 10gigfd000
netdoc/utils.py
Please add:
speed = speed.replace("gigfd", "000") # HP Procurve
When using the script to add and discover an object, the following failure is logged:
`An exception occurred: DoesNotExist: ScriptModule matching query does not exist.
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 504, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 180, in run
output = discovery(
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/tasks.py", line 124, in discovery
utils.spawn_script("Ingest")
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/utils.py", line 1160, in spawn_script
module = ScriptModule.objects.get(data_path="netdoc_scripts.py")
File "/opt/netbox/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/netbox/venv/lib/python3.9/site-packages/django/db/models/query.py", line 637, in get
raise self.model.DoesNotExist(
extras.models.scripts.ScriptModule.DoesNotExist: ScriptModule matching query does not exist.`
From the log I saw pyVmomi able to discover vm ip address with key "guest_address". Is it able to ingest to Netbox with vm ip address?
Initially opened by @a084ed22
Hello,
I'm following netbox's recommendation on modeling switch stacks by creating a separate device for each stack member and a virtual chassis as a container. Netbox correctly shows on the device selected as master in the virtual chassis the interfaces from the other stack members.
Would it be possible to extend netdoc to use the same interfaces? In my test environment, I'm using the stack's name as the name for the virtual chassis, while the stack members' names have a colon and their stack ID appended. In this environment, netdoc ends up creating a new device named as the stack with all the available interfaces linked to it.
When running discover with admin account, I am getting error as below, Palo Alto FW model 220
Task parameters
{'order': 0, 'enable': False, 'command': '', 'platform': 'panw_ngfw', 'protocol': 'default', 'template': 'show system info', 'framework': 'xml', 'supported': True}
Details
{'command': '',
'enable': False,
'framework': 'xml',
'order': 0,
'platform': 'panw_ngfw',
'protocol': 'default',
'supported': True,
'template': 'show system info'}
Raw output
ERROR CODE 403
Invalid Credential
After upgrading netdoc the library directory keeps getting removed and I need to add it back manually. Should it not be downloaded and extracted with the installer?
An exception occurred: FileNotFoundError: [Errno 2] No such file or directory: '/opt/netbox/venv/lib/python3.10/site-packages/netdoc/library/Cisco.yml'
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 476, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/NetDoc.py", line 290, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 384, in log_ingest
module.ingest(log)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/ingestors/netmiko_cisco_nxos_show_inventory.py", line 30, in ingest
device.update(
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/schemas/device.py", line 119, in update
model_o = create_manufacturer_and_model(
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/schemas/device.py", line 168, in create_manufacturer_and_model
netbox_model = utils.find_model(
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 326, in find_model
with open(library_file, "r", encoding="utf-8") as vendor_fh:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/netbox/venv/lib/python3.10/site-packages/netdoc/library/Cisco.yml'
cat /opt/netbox/venv/lib/python3.10/site-packages/netdoc/
api/ ingestors/ navigation.py schemas/ tables.py tests/ views.py
discoverers/ __init__.py nornir_inventory.py scripts/ tasks.py topologies.py
filtersets.py migrations/ __pycache__/ search.py templates/ urls.py
forms.py models.py reports/ static/ templatetags/ utils.py
As you can see the entire directory is missing.
User static routes:
U - per-user static route
Example on a Cisco ASR1000:
U 192.192.55.132/30 [1/0] via 192.168.1.1
(Typically these are from radius/dynamic routing protocols injecting per-user routes)
Just needs the normalize_route_type adding.
PR to follow.
When create diagram l2, with dynamic option, its keep floating, if choose static option, the connections between same devices stacked to each other. Any idea?
Another thing is whether netdoc supported connection label showing on the diagram?
Hey
I would like to reopen this issue we still have about installing netdoc in a netbox hosted on a docker.
I encountered this error:
[+] Building 10.2s (10/10) FINISHED
=> [internal] load build definition from Dockerfile-Plugins 0.1s
=> => transferring dockerfile: 564B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 159B 0.0s
=> [internal] load metadata for docker.io/netboxcommunity/netbox:latest 0.6s
=> [internal] load build context 0.0s
=> => transferring context: 172B 0.0s
=> CACHED [1/6] FROM docker.io/netboxcommunity/netbox:latest@sha256:f4b0d71cf4f518d4ab087b45c706e2ec82b9619b4071c6c3ccd880048d6a6be9 0.0s
=> [2/6] COPY ./plugin_requirements.txt / 0.1s
=> [3/6] RUN /opt/netbox/venv/bin/pip install --no-warn-script-location -r /plugin_requirements.txt 7.8s
=> [4/6] COPY configuration/configuration.py /etc/netbox/config/configuration.py 0.1s
=> [5/6] COPY configuration/plugins.py /etc/netbox/config/plugins.py 0.1s
=> ERROR [6/6] RUN SECRET_KEY="dummydummydummydummydummydummydummydummydummydummy" /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py collectstatic --no-input 1.3s
------
> [6/6] RUN SECRET_KEY="dummydummydummydummydummydummydummydummydummydummy" /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py collectstatic --no-input:
#0 0.883 🧬 loaded config '/etc/netbox/config/configuration.py'
#0 0.883 🧬 loaded config '/etc/netbox/config/extra.py'
#0 0.883 🧬 loaded config '/etc/netbox/config/logging.py'
#0 0.884 🧬 loaded config '/etc/netbox/config/plugins.py'
#0 1.137 Traceback (most recent call last):
#0 1.137 File "/opt/netbox/netbox/manage.py", line 10, in <module>
#0 1.137 execute_from_command_line(sys.argv)
#0 1.138 File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
#0 1.138 utility.execute()
#0 1.138 File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 386, in execute
#0 1.138 settings.INSTALLED_APPS
#0 1.138 File "/opt/netbox/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 92, in __getattr__
#0 1.138 self._setup(name)
#0 1.138 File "/opt/netbox/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 79, in _setup
#0 1.138 self._wrapped = Settings(settings_module)
#0 1.138 File "/opt/netbox/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 190, in __init__
#0 1.138 mod = importlib.import_module(self.SETTINGS_MODULE)
#0 1.138 File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
#0 1.138 return _bootstrap._gcd_import(name[level:], package, level)
#0 1.138 File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
#0 1.138 File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
#0 1.138 File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
#0 1.138 File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
#0 1.138 File "<frozen importlib._bootstrap_external>", line 883, in exec_module
#0 1.139 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
#0 1.139 File "/opt/netbox/netbox/netbox/settings.py", line 711, in <module>
#0 1.139 plugin = importlib.import_module(plugin_name)
#0 1.139 File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
#0 1.139 return _bootstrap._gcd_import(name[level:], package, level)
#0 1.139 File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
#0 1.139 File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
#0 1.139 File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
#0 1.139 File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
#0 1.139 File "<frozen importlib._bootstrap_external>", line 883, in exec_module
#0 1.139 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
#0 1.139 File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/__init__.py", line 61, in <module>
#0 1.139 shutil.copy(src_file, dst_file)
#0 1.139 File "/usr/lib/python3.10/shutil.py", line 417, in copy
#0 1.139 copyfile(src, dst, follow_symlinks=follow_symlinks)
#0 1.139 File "/usr/lib/python3.10/shutil.py", line 256, in copyfile
#0 1.139 with open(dst, 'wb') as fdst:
#0 1.139 FileNotFoundError: [Errno 2] No such file or directory: '/etc/netbox/scripts/NetDoc.py'
------
failed to solve: process "/bin/sh -c SECRET_KEY=\"dummydummydummydummydummydummydummydummydummydummy\" /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py collectstatic --no-input" did not complete successfully: exit code: 1
While running the command : docker compose build --no-cache
To deploy the plugin, I've followed this
Here is my Dockerfile-plugin
FROM netboxcommunity/netbox:latest
COPY ./plugin_requirements.txt /
RUN /opt/netbox/venv/bin/pip install --no-warn-script-location -r /plugin_requirements.txt
# These lines are only required if your plugin has its own static files.
COPY configuration/configuration.py /etc/netbox/config/configuration.py
COPY configuration/plugins.py /etc/netbox/config/plugins.py
RUN SECRET_KEY="dummydummydummydummydummydummydummydummydummydummy" /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py collectstatic --no-input
Here is the plugin_requirements.txt:
netdoc
ntc-templates
netbox_topology_views
And I added this lines in the configuration.py:
PLUGINS = ['netdoc', 'ntc-templates']
PLUGINS_CONFIG = {
'netdoc': {
# 'MAX_INGESTED_LOGS': 50,
'NTC_TEMPLATES_DIR': '/opt/ntc-templates/ntc_templates/templates',
# 'NORNIR_LOG': f'{settings.BASE_DIR}/nornir.log',
# 'NORNIR_TIMEOUT': 300,
# 'NORNIR_SKIP_LIST': [],
# 'RAISE_ON_CDP_FAIL': True,
# 'RAISE_ON_LLDP_FAIL': True,
},
}
If you have any clue...
Initial requested by @PieterL75
Hey,
I like the idea of this a lot. For me there are 2 things that I would consider in new releases
Use SuzieQ for the retrieval of information. It parses a lot of vendors and put them into a standardized database. They do an awful good job at it ! https://github.com/netenglabs/suzieq
SuzieQ already has an integration with NetBox to decide what devices are and what credentials to use
Give an option to review the changes that will be made to NetBox. I like to know what the real-world is in comparison to the documented world. Updating the documented world requires a manual approval, as the real-world could be wrongly implemented.
Keep up the work ! really good
Pieter
Limited support by Netmiko
Templates present in NTC-Templates
I can set up a lab with several different models if need be.
I am trying to send event to telegram via netbox webhook whenever netdoc script detect any change
But for netmiko script, netdoc always update to device/interface even no any change, anyway to ignore if no any change?
Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: An exception occurred: TypeError: Device() got unexpected keyword arguments: 'device_role_id'
Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: Traceback (most recent call last): Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/netbox/extras/scripts.py", line 504, in _run_script Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: script.output = script.run(data=data, commit=commit) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 353, in run Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: log_ingest(log) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 578, in log_ingest Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: module.ingest(log) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/ingestors/netmiko_cisco_ios_hostname.py", line 36, in ingest Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: device_o = device.create(**data) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/schemas/device.py", line 108, in create Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: obj = utils.object_create(Device, **kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 1055, in object_create Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: return model_o.objects.create(**kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: return getattr(self.get_queryset(), name)(*args, **kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 656, in create Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: obj = self.model(**kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/netbox/utilities/tracking.py", line 41, in __init__ Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: super().__init__(*args, **kwargs) Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 567, in __init__ Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: raise TypeError( Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: TypeError: Device() got unexpected keyword arguments: 'device_role_id' Sep 28 13:42:54 dc-netbox-srv01 python3[13861]:
Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: Exception raised during script execution: Device() got unexpected keyword arguments: 'device_role_id'
Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: 13:42:54 default: Job OK (1b80535e-28ed-4600-8627-bd82c3eb29aa)
Sep 28 13:42:54 dc-netbox-srv01 python3[13861]: 13:42:54 Result is kept for 500 seconds
An exception occurred: AttributeError: 'NoneType' object has no attribute 'lower'
Traceback (most recent call last):
File "/opt/netbox-3.5.9/netbox/extras/scripts.py", line 504, in _run_script
script.output = script.run(data=data, commit=commit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox-3.5.9/netbox/scripts/netdoc_scripts.py", line 353, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.11/site-packages/netdoc/utils.py", line 578, in log_ingest
module.ingest(log)
File "/opt/netbox/venv/lib/python3.11/site-packages/netdoc/ingestors/xml_panw_ngfw_show_routing_route.py", line 78, in ingest
nexthop_if_label = utils.normalize_interface_label(nexthop_if_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib/python3.11/site-packages/netdoc/utils.py", line 636, in normalize_interface_label
name = name.lower().strip()
^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'lower'
Hello, I saw that there was an issue to add Huawei support, but it didn't follow through.
Is this all you need? How can I help?
To add a new supported device, we need, at least:
Originally posted by @dainok in #2 (comment)
When running discovery vcenter/esxi-host with admin credential, I am facing issue with netdoc 3.5.3 or 0.10.33, any idea?
An exception occurred: TypeError: Object of type vim.dvs.PortConnection is not JSON serializable
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 504, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 262, in run
output = discovery(
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/tasks.py", line 111, in discovery
module.discovery(filtered_devices, filters=filters, filter_type=filter_type)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/discoverers/json_vmware_vsphere.py", line 272, in discovery
raw_output=json.dumps(result.result),
File "/usr/lib/python3.10/json/init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python3.10/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type vim.dvs.PortConnection is not JSON serializable
Originally opened by @andyb2000
Spotted this one throwing up an error. In this case it's in a vrf but don't think that's relevant to the bug. The routing table has multiple default gateways via different paths (learnt via OSPF) and so it throws an exception due to duplicate key failure:
May 03 10:03:44 srvr-0040b606 NetBox[50802]: level=ERROR,logger=netbox.scripts.NetDoc.Ingest,msg=An exception occurred: `IntegrityError: duplicate key value violates unique constraint "netdoc_routetableentry_device_id_destination_di_7aa5e45e_uniq"
DETAIL: Key (device_id, destination, distance, metric, protocol, vrf_id)=(1026, 0.0.0.0/0, 110, 1, oia, 143) already exists.
Output from routing table (Cisco IOS):
Gateway of last resort is 192.168.1.251 to network 0.0.0.0
O*E2 0.0.0.0/0 [110/1] via 192.168.1.251, 7w0d, GigabitEthernet0/0/0.513
[110/1] via 192.168.1.250, 7w0d, GigabitEthernet0/0/0.513
10.0.0.0/8 is variably subnetted, 23 subnets, 3 masks
O E2 10.0.0.0/22
[110/20] via 192.168.1.251, 7w0d, GigabitEthernet0/0/0.513
[110/20] via 192.168.1.250, 7w0d, GigabitEthernet0/0/0.513
O E2 10.0.8.0/22
In this case because there are multiple paths (2 edge routers running an hsrp default gateway into 2 switches) which then ospf talk to a router providing the default gateway (default originate) that means each router has multiple paths to the same gateway.
router1 --> switch1 --> customer device
router2 --> switch2 --> customer device (same customer device)
Customer device announces a default route back to router1 and router2.
Actually, now I think about it, I think this quirk is only possible in OSPF where there are two equal cost paths to the two gateways.
When attempting to access the following portion of the NetDoc Tables I get the below error:
netbox-url.com/plugins/netdoc/routingtable/
<class 'django.urls.exceptions.NoReverseMatch'>
Reverse for 'routingtable' not found. 'routingtable' is not a valid view function or pattern name.
Python version: 3.10.6
NetBox version: 3.4.10
I didn't see anything else around with a similar error and not exactly sure what might have triggered it.
Initial requested by @alexandreseitz
Do you support juniper junos?
Ingesting log 3819 with command show cdp neighbors detail on device 10.254.2.161 via netmiko_cisco_nxos
An exception occurred: IntegrityError: Multiple neighbors on CNBJBCDR-BEIYAN-WUHAN-N93180-1:Ethernet1/8 (e1/8) or INAGGSW-N93180-CNWUH01-1326-01:Ethernet1/5 (e1/5)
Traceback (most recent call last):
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/schemas/cable.py", line 87, in link
cable_o = Cable.objects.filter(terminations__interface=left_interface_o).get(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 650, in get
raise self.model.DoesNotExist(
dcim.models.cables.Cable.DoesNotExist: Cable matching query does not exist.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "dcim_cabletermination_unique_termination"
DETAIL: Key (termination_type_id, termination_id)=(32, 43285) already exists.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/schemas/cable.py", line 96, in link
CableTermination.objects.create(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 671, in create
obj.save(force_insert=True, using=self.db)
File "/opt/netbox/netbox/dcim/models/cables.py", line 318, in save
super().save(*args, **kwargs)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 812, in save
self.save_base(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 863, in save_base
updated = self._save_table(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1006, in _save_table
results = self._do_insert(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1047, in _do_insert
return manager._insert(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1791, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1660, in execute_sql
cursor.execute(sql, params)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "dcim_cabletermination_unique_termination"
DETAIL: Key (termination_type_id, termination_id)=(32, 43285) already exists.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 476, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/NetDoc.py", line 290, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 381, in log_ingest
module.ingest(log)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/ingestors/netmiko_cisco_nxos_show_cdp_neighbors_detail.py", line 87, in ingest
cable.link(
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/schemas/cable.py", line 111, in link
raise IntegrityError(
django.db.utils.IntegrityError: Multiple neighbors on CNBJBCDR-BEIYAN-WUHAN-N93180-1:Ethernet1/8 (e1/8) or INAGGSW-N93180-CNWUH01-1326-01:Ethernet1/5 (e1/5)
I'm seeing on my bulk run some devices seem to get timeouts/unable to connect first time round. If I search for them and run a single discover they succeed (suspect too many concurrent connections to device, etc).
On those devices, netdoc shows:
ID Created Discoverable Device Command Order Configuration Supported Success Parsed Ingested
135576 2023-06-13 02:55 194.x.x.x via netmiko_cisco_ios — show running-config | include hostname 0 ✘ ✔ ✘ ✘ ✘
I'm going to look at creating a script/option to detect a discoverable with only 1 failed hostname discovery job so that I can do a bulk re-run (unless any better suggestions?)
After uprade netdoc 3.5.3, after connection has been discovered, create diagram having below issue, no issue on netdoc 0.10.33
<class 'AttributeError'>
module 'netdoc.topologies' has no attribute 'get__topology_data'
Python version: 3.10.12
NetBox version: 3.5.8
Plugins:
netdoc: 3.5.3
It would be nice to have support for ASA devices in addition to the IOS, XE, XR, NXOS devices. Netmiko has an ASA module:
https://ktbyers.github.io/netmiko/docs/netmiko/cisco/cisco_asa_ssh.html
I am facing issue when run script update IPAM from ARP tables, interface has no IP address due to HA standby state, but it still able having arp table, causing script error. Appreciated if we can ignore it.
1 Failure
IP address not found on interface xxx, maybe some ingestion script has failed
2 Failure
An exception occurred: AttributeError: 'NoneType' object has no attribute 'address'
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 504, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 443, in run
.address.prefixlen
AttributeError: 'NoneType' object has no attribute 'address'
3 Info
Database changes have been reverted due to error.
I am having issue when trying to discover device via netdoc
Server Error
There was a problem with your request. Please contact an administrator.
The complete exception is provided below:
<class 'extras.models.scripts.ScriptModule.DoesNotExist'>
ScriptModule matching query does not exist.
Python version: 3.10.12
NetBox version: 3.5.9
Plugins:
netdoc: 3.5.1
Ingest time is too long, more than 10 hours have not been completed, timed out. I would like to know that there are configuration items to delete some found objects for the data necessary to generate l2 and l3 diagrams.
Hello - trying to implement Netdoc into an existing NetBox-Installation and I followed your guidelines, I just excluded the steps which I've already done with the official documentation part.
My issue occurs when running the "sudo -u netbox /opt/netbox/upgrade.sh" - it runs through until I get the error, mentioned in the title: "ImportError: cannot import name 'get_scripts' from 'extras.scripts' (/opt/netbox/netbox/extras/scripts.py)"
Any suggestion on what it could be? it's an Ubuntu 22.04 with only NetBox installed (no docker or anything) - only plugin I want or try to install is your NetDoc plugin.
Initially requested by @amdei
What I have:
What I need:
How I would like to do it:
What I wan as a result:
Why:
What I can:
What I'm not sure about:
Would it be possible for wiser guys to give a piece of advice here on these uncertainties?
The hardest part is to resolve ideological issues.
Implementation should not take long, and promised to be relatively straightforward.
Hi,
have you thought about possibility of adapting Netdoc to be compatible with Nautobot API also? https://github.com/nautobot/nautobot
Ingest script errors out while ingesting vlans:
An exception occurred: `ValidationError: 'name' is a required property
Failed validating 'required' in schema: {'properties': {'name': {'type': 'string'}, 'status': {'enum': ['active', 'reserved', 'deprecated'], 'type': 'string'}, 'vid': {'type': 'integer'}}, 'required': ['vid', 'name', 'status'], 'type': 'object'}
On instance: {'status': 'active', 'vid': 1}`
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 476, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/NetDoc.py", line 290, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/utils.py", line 504, in log_ingest
module.ingest(log)
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/ingestors/netmiko_cisco_ios_show_vlan.py", line 26, in ingest
vlan_o = vlan.create(**data)
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/schemas/vlan.py", line 52, in create
validate(data, get_schema_create(), format_checker=FormatChecker())
File "/opt/netbox/venv/lib/python3.9/site-packages/jsonschema/validators.py", line 934, in validate
raise error
jsonschema.exceptions.ValidationError: 'name' is a required property
Failed validating 'required' in schema:
{'properties': {'name': {'type': 'string'},
'status': {'enum': ['active', 'reserved', 'deprecated'],
'type': 'string'},
'vid': {'type': 'integer'}},
'required': ['vid', 'name', 'status'],
'type': 'object'}
On instance:
{'status': 'active', 'vid': 1}
Raw log file attached:
environment:Install using docker
comment: After clicking discoverable, the business does not change the data. Execute the command docker-compose logs netbox-worker。
netbox-docker-netbox-worker-1 | 08:38:03 default: extras.scripts.run_script(data={'discoverables': [<Discoverable: 10.62.192.6 via netmiko_cisco_ios>]}, job_result=<JobResult: 05b95547-5148-415e-a8b7-67cc9914b09c>, request=<utilities.utils.NetBoxFakeRequest object at 0x7f94956c3d60>) (05b95547-5148-415e-a8b7-67cc9914b09c)
netbox-docker-netbox-worker-1 | ['10.62.192.6']
netbox-docker-netbox-worker-1 | dict_keys([])
netbox-docker-netbox-worker-1 | Nornir inventory is empty
netbox-docker-netbox-worker-1 | 08:38:03 default: Job OK (05b95547-5148-415e-a8b7-67cc9914b09c)
netbox-docker-netbox-worker-1 | 08:38:03 Result is kept for 500 seconds
what to do in this situation?
Is it possible to add functionality to add optional arguments to the connection dictionary such as
"ssh_config_file": "./ssh_config".
"global_delay_factor": 5
etc?
I have fixed all prefix duplicate issue after remove those duplicated prefix in mgmt vrf, after re-discovery, Netdoc auto re-create those prefix
Initially requested by @hawk-nsk
Do you support Huwei devices?
Maybe we can work together with that stuff?
Thanks in advance.
I tried to customizing script to ingest vm ip address but having issue.
Discover script:
Investor script:
When trying to run discovery/ingest, there is an issue "unexpected keyword arguments: 'address'".
Seem address of vm interface need to ingest under IPAM or something
Hi!
I'm running a brand new installation of Netbox with Netdoc (I have followed the steps described on https://github.com/dainok/netdoc/wiki/NetBox-with-NetDoc-installation) and I tried to run "Add and discover" script without creating anything but Site.
I'm getting the following error:
An exception occurred: ValueError: The discoverable 143.54.0.5 does not have an attached device thus logs cannot be ingested. Check if logs with priority 0 are ingested or if Device is attached to a different Discoverable.
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 504, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 353, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 572, in log_ingest
raise ValueError(
ValueError: The discoverable 143.54.0.5 does not have an attached device thus logs cannot be ingested. Check if logs with priority 0 are ingested or if Device is attached to a different Discoverable.
So, after I had this error, I've created the Manufacturer Cisco, Device Type WS-C4506-E, Device Router with a virtual interface that has the same IP address as the Discoverable, but still got the same error.
Any thoughts on how can I get it to work?
Thank you!
mode:netmiko_cisco_nxos
4 | Info | Ingesting log 1455 with command show lldp neighbors detail on device 10.254.23.3 via netmiko_cisco_nxos
-- | -- | --
An exception occurred: AttributeError: 'NoneType' object has no attribute 'lower'
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 476, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/NetDoc.py", line 290, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 373, in log_ingest
module.ingest(log)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/ingestors/netmiko_cisco_nxos_show_lldp_neighbors_detail.py", line 26, in ingest
remote_interface_label = utils.get_remote_lldp_interface_label(
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 303, in get_remote_lldp_interface_label
return normalize_interface_label(port_description)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 415, in normalize_interface_label
name = name.lower()
AttributeError: 'NoneType' object has no attribute 'lower'
When attempting to ingest a NXOS device with BGP routes it seems to keep erroring on the route type, Originally it was erroring because BGP route type was showing as type bgp-(ASN) external so I tried tweaking the textfms to account for that but that didn't seem to solve it since it still errors because it's grabbing the external portion of the show ip route as well.
An exception occurred: ValueError: Invalid route type bgp external Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 476, in _run_script
script.output = script.run(data=data, commit=commit) File "/opt/netbox/netbox/scripts/NetDoc.py", line 290, in run log_ingest(log) File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 384, in log_ingest module.ingest(log)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/ingestors/netmiko_cisco_nxos_show_ip_route_vrf_all.py", line 29, in ingest protocol = utils.normalize_route_type(protocol)
File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/utils.py", line 693, in normalize_route_type raise ValueError(f"Invalid route type {route_type}") ValueError: Invalid route type bgp external
I just recently ran through the netbox and netdoc install documents and afterwards I noticed two things:
Netdoc doesn't seem to make or add any files to the following directory causing issues:
/opt/netbox/venv/lib/python3.10/site-packages/netdoc/library/
(depending on the install it might be /opt/netbox-3.4.10/... instead)
However even after manually pulling down the library files and adding them all to that location netdoc still doesn't' ingest anything. I see it run when I trigger a discovery and then look at the logs and nothing is being ingested, everything shows x
While trying to ingest vrf routing table, ingest script errors out when encountering a static route pointing to Null0 device.
An exception occurred: ValueError: Even one of nexthop_ip or nexthop_if_id should have a value.
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 476, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/NetDoc.py", line 290, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/utils.py", line 381, in log_ingest
module.ingest(log)
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/ingestors/netmiko_cisco_ios_show_ip_route.py", line 72, in ingest
routetableentry.create(**data)
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/schemas/routetableentry.py", line 70, in create
raise ValueError("Even one of nexthop_ip or nexthop_if_id should have a value.")
ValueError: Even one of nexthop_ip or nexthop_if_id should have a value.
Route entry looks like this:
S 111.222.3.0/27 is directly connected
Raw log file attached.
Supported by Netmiko,
Templates already in NTC-templates project
As for the lab, I can set up something with different Mikrotik models if needed.
Platform supports CDP and LLDP for neighbor discovery.
Hi,
after latest upgrade from github, issue with ingesting IP addresses is popping up everywhere:
An exception occurred: TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):
File "/opt/netbox/netbox/extras/scripts.py", line 476, in _run_script
script.output = script.run(data=data, commit=commit)
File "/opt/netbox/netbox/scripts/NetDoc.py", line 290, in run
log_ingest(log)
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/utils.py", line 505, in log_ingest
module.ingest(log)
File "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/ingestors/netmiko_cisco_ios_show_ip_interface.py", line 26, in ingest
f"{ipaddr}/{mask_list[index]}" for index, ipaddr in enumerate(ip_list)
TypeError: 'NoneType' object is not iterable
Log file attached:
20582.json.txt
Same issue affects several IOS platforms:
I have a problem installing. Netbox 3.5.3
Applying database migrations (python3 netbox/manage.py migrate)...
Traceback (most recent call last):
File "netbox/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/management/init.py", line 446, in execute_from_command_line
utility.execute()
File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/management/init.py", line 420, in execute
django.setup()
File "/opt/netbox/venv/lib/python3.8/site-packages/django/init.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/opt/netbox/venv/lib/python3.8/site-packages/django/apps/registry.py", line 116, in populate
app_config.import_models()
File "/opt/netbox/venv/lib/python3.8/site-packages/django/apps/config.py", line 269, in import_models
self.models_module = import_module(models_module_name)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/opt/netbox/venv/lib/python3.8/site-packages/netdoc/models.py", line 24, in
from netdoc.utils import parse_netmiko_output, CONFIG_COMMANDS, FAILURE_OUTPUT
File "/opt/netbox/venv/lib/python3.8/site-packages/netdoc/utils.py", line 29, in
from extras.scripts import get_scripts, run_script
ImportError: cannot import name 'get_scripts' from 'extras.scripts' (/opt/netbox/netbox/extras/scripts.py)
Our Cisco XR NCS-5501-SE unable to ingest after discovered.
I have changed https://github.com/dainok/netdoc/blob/master/netdoc/ingestors/netmiko_cisco_xr_admin_show_inventory.py line 22 from item.get("name") to item.get("descr"), then it able to ingested.
Not sure if that is a typo.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.