GithubHelp home page GithubHelp logo

bl4ko / netbox-ssot Goto Github PK

View Code? Open in Web Editor NEW
23.0 3.0 4.0 9.05 MB

Microservice that synces objects from multiple sources into netbox and makes it automatic SSOT.

License: MIT License

Go 99.87% Dockerfile 0.10% Makefile 0.03%
dnac go netbox netbox-automation ovirt ssot olvm vcenter cisco-dna-center proxmox

netbox-ssot's Introduction

Netbox-SSOT

Go GitHub last commit GitHub Tag codecov GitHub License

Netbox-ssot is a small but powerful microservice designed to keep your Netbox instance in sync with external data sources.

It is designed to be run as a cronjob, and will periodically update Netbox with the latest data from the external sources. It syncs each source in parallel to speed up the process of syncing.

Currently, the supported external data sources types are:

Warning

This project is still under heavy development, use with caution.

  • v1.x.x works with netbox>=4.0.0

  • v0.x.x works with netbox>=3.7.0

Configuration

Netbox-ssot is configured via a single yaml file. The configuration file is divided into three sections:

  • logger: Logger configuration
  • netbox: Netbox configuration
  • source: Array of configuration for each data source

Example configuration can be found here.

Logger

Parameter Description Type Possible values Default Required
logger.level Log level int 0-3 1 Yes
logger.dest Log output filename. Default "" representing stdout. str Any valid path "" No

Netbox

Parameter Description Type Possible values Default Required
netbox.apiToken Netbox API token. str Any valid token "" Yes
netbox.hostname Hostname of your netbox instance (e.g netbox.example.com). str Valid hostname "" Yes
netbox.port Port of your netbox instance. int 0-65536 443 No
netbox.httpScheme HTTP scheme of your netbox instance. str [http, https] https No
netbox.validateCert Validate the TLS certificate of your netbox instance. bool [true, false] false No
netbox.timeout Max timeout for api call of your netbox instance. int >=0 30 No
netbox.removeOrphans Automatically remove all objects tagged with netbox-ssot which, were not found on the sources, during this iteration. bool [true, false] true No
netbox.tag Tag to be applied to all objects managed by netbox-ssot. string any "netbox-ssot" No
netbox.tagColor TagColor for the netbox-ssot tag. string any "07426b" No
netbox.sourcePriority Array of source names in order of priority. If an object (e.g. Vlan) is found in multiple sources, the first source in the list will be used. []string any [] No
netbox.arpDataLifeSpan Lifespan of each arp data entry in seconds (if entry is not found in the following interations). int >0 172800 No
netbox.caFile Path to a self signed certificate for netbox. string Valid path "" No

Source

Parameter Description Source Type Type Possible values Default Required
source.name Name of the data source. all str any "" Yes
source.type Type of the data source. all str [ovirt, vmware, dnac, proxmox, paloalto] "" Yes
source.httpScheme Http scheme for the source all str [ http,https] https No
source.hostname Hostname of the data source. all str any "" Yes
source.port Port of the data source. all int 0-65536 443 No
source.username Username of the data source account. all str any "" Yes
source.password Password of the data source account. all str any "" Yes
source.apiToken Password of the data source account. [fortigate] str any "" Yes
source.validateCert Enforce TLS certificate validation. all bool [true, false] false No
source.tagColor TagColor for the source tag. all string any Predefined No
source.ignoredSubnets List of subnets, which will be ignored (e.g. IPs won't be synced). all []string any [] No
source.interfaceFilter Regex representation of interface names to be ignored (e.g. (cali|vxlan|flannel|[a-f0-9]{15})) all string any [] No
source.collectArpData Collect data from the arp table of the device. [paloalto, ios-xe] bool [true, false] false No
source.datacenterClusterGroupRelations Regex relations in format regex = clusterGroupName, that map each datacenter that satisfies regex to clusterGroupname (see #130). [vmware, ovirt] []string any [] No
source.hostSiteRelations Regex relations in format regex = siteName, that map each host that satisfies regex to site. all []string any [] No
source.clusterSiteRelations Regex relations in format regex = siteName, that map each cluster that satisfies regex to site. all []string any [] No
source.clusterTenantRelations Regex relations in format regex = tenantName, that map each cluster that satisfies regex to tenant. all []string any [] No
source.hostTenantRelations Regex relations in format regex = tenantName, that map each host that satisfies regex to tenant. all []string any [] No
source.vmTenantRelations Regex relations in format regex = tenantName, that map each vm that satisfies regex to tenant. all []string any [] No
source.vlanGroupRelations Regex relations in format regex = vlanGroup, that map each vlan that satisfies regex to vlanGroup. all []string any [] No
source.vlanTenantRelations Regex relations in format regex = tenantName, that map each vlan that satisfies regex to tenant. all []string any [] No
source.customFieldMappings Mappings of format customFieldName = option. Currently, supported options are contact, owner, description. [vmware] []string any [] No
source.caFile Path to a self signed certificate for the source. any string Valid path "" No

Example config

logger:
  level: 1
  dest: ""

netbox:
  apiToken: "el1aof2azu6n50ks5zcenp3..."
  hostname: "netbox.example.com"
  httpScheme: http
  port: 443
  timeout: 30
  sourcePriority: ["olvm", "prodvmware", "prodprox", "dnacenter", "testvmware", "pa-uk", "fmc-lab"] # Not required, but recommended

source:
  - name: olvm
    type: ovirt
    hostname: ovirt.example.com
    port: 443
    username: "admin"
    password: "topsecret"
    interfaceFilter: (cali|vxlan|flannel|docker|[a-f0-9]{15})

  - name: prodvmware
    type: vmware
    hostname: vcenter.example.com
    username: user
    password: "top_secret"
    clusterSiteRelations:
      - .* = ExampleSite
    hostSiteRelations:
      - .*_NYC = New York
      - nyc.* = New York
    customFieldMappings: # Here we define map of our custom field names, to 3 option [email, owner, description]
      - Mail = email
      - Creator = owner
      - Description = description

  - name: prodprox
    type: proxmox
    username: svc@pve
    password: changeme
    hostname: 192.168.1.254
    port: 8006
    validateCert: false
    clusterSiteRelations:
     - .* = Site

  - name: forti
    type: fortigate
    hostname: forti.example.com
    apiToken: "apitokenhere"
    validateCert: False
    hostTenantRelations:
      - .* = MyTenant
    hostSiteRelations:
      - .* = MyTenant
    hostTenantRelations:
      - .* = MyTenant

  - name: pa-uk
    type: paloalto
    hostname: 192.168.1.52
    username: user
    password: passw0rd
    hostTenantRelations:
      - .* = MyTenant
    hostSiteRelations:
      - .* = MySite
    vlanTenantRelations:
      - .* = MyTenant
    collectArpData: true

  - name: dnacenter
    type: dnac
    hostname: dnac.example.com
    username: user
    password: "pa$$w0rd"
    vlanTenantRelations:
      - .* = MyTenant

  - name: fmc-lab
    type: fmc
    hostname: 172.16.1.30
    username: user
    password: password
    validateCert: False
    hostTenantRelations:
      - .* = MyTenant
    hostSiteRelations:
      - .* = MySite
    vlanTenantRelations:
      - .* = MyTenant

  - name: cs1
    type: ios-xe
    hostname: 10.10.1.1
    username: user
    password: password
    validateCert: False
    hostTenantRelations:
      - .* = MyTenant
    hostSiteRelations:
      - .* = MySite
    vlanTenantRelations:
      - .* = MyTenant
    collectArpData:
      true

Deployment

Via docker

docker run -v /path/to/config.yaml:/app/config.yaml ghcr.io/bl4ko/netbox-ssot

Via k8s

Create k8s secret from self defined config.yaml:

kubectl create secret generic netbox-ssot-secret --from-file=config.yaml

Apply cronjob with custom settings:

kubectl apply -f cronjob.yaml

Using self signed certificate

Create self signed certificate e.g.:

kubectl create secret generic netbox-ssot-cert --from-file=sub.pem=./sub.pem

Use cronjob with cert mounted:

kubectl apply -f cronjob_with_cert.yaml

netbox-ssot's People

Contributors

bl4ko avatar renovate[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

netbox-ssot's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

dockerfile
Dockerfile
  • golang 1.22.5@sha256:829eff99a4b2abffe68f6a3847337bf6455d69d17e49ec1a97dac78834754bd6
  • alpine 3.20.1@sha256:b89d9c93e9ed3597455c90a0b88a8bbb5cb7188438f70953fede212a0c4394e0
github-actions
.github/workflows/cd.yml
  • docker/login-action v3@0d4c9c5ea7693da7b068278f7b52bda2a190a446
  • docker/login-action v3@0d4c9c5ea7693da7b068278f7b52bda2a190a446
  • docker/setup-qemu-action v3@5927c834f5b4fdf503fca6f4c7eccda82949e1ee
  • docker/setup-buildx-action v3@4fd812986e6c8c2a69e18311145f9371337f27d4
  • docker/build-push-action v6.4.0@a254f8ca60a858f3136a2f1f23a60969f2c402dd
.github/workflows/check_links.yml
  • actions/checkout v4@692973e3d937129bcbf40652eb9f2f61becf3332
  • lycheeverse/lychee-action v1.10.0@2b973e86fc7b1f6b36a93795fe2c9c6ae1118621
  • peter-evans/create-issue-from-file v5@24452a72d85239eacf1468b0f1982a9f3fec4c94
.github/workflows/ci.yml
  • actions/checkout v4@692973e3d937129bcbf40652eb9f2f61becf3332
  • tj-actions/changed-files v44@cc733854b1f224978ef800d29e4709d5ee2883e4
  • actions/setup-go v5@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32
  • golangci/golangci-lint-action v6@a4f60bb28d35aeee14e6880718e0c85ff1882e64
  • codecov/codecov-action v4.5.0@e28ff129e5465c2c0dcc6f003fc735cb6ae0c673
  • actions/checkout v4@692973e3d937129bcbf40652eb9f2f61becf3332
  • tj-actions/changed-files v44@cc733854b1f224978ef800d29e4709d5ee2883e4
  • aquasecurity/trivy-action master
.github/workflows/gitleaks.yml
  • actions/checkout v4@692973e3d937129bcbf40652eb9f2f61becf3332
  • gitleaks/gitleaks-action v2@44c470ffc35caa8b1eb3e8012ca53c2f9bea4eb5
.github/workflows/release.yml
  • actions/checkout v4@692973e3d937129bcbf40652eb9f2f61becf3332
  • actions/setup-node v4@1e60f620b9541d16bece96c5465dc8ee9832be0b
.github/workflows/spellcheck.yml
  • actions/checkout v4@692973e3d937129bcbf40652eb9f2f61becf3332
  • streetsidesoftware/cspell-action v6@245201e3f58019204d99920deeb78aade6724230
gomod
go.mod
  • go 1.21
  • github.com/PaloAltoNetworks/pango v0.10.2
  • github.com/cisco-en-programmability/dnacenter-go-sdk/v5 v5.0.30
  • github.com/luthermonson/go-proxmox v0.1.0
  • github.com/ovirt/go-ovirt v4.3.4+incompatible
  • github.com/scrapli/scrapligo v1.3.1
  • github.com/vmware/govmomi v0.38.0
  • golang.org/x/text v0.16.0
  • gopkg.in/yaml.v3 v3.0.1
pre-commit
.pre-commit-config.yaml
  • pre-commit/pre-commit-hooks v4.6.0
  • dnephin/pre-commit-golang v0.5.1
  • gitleaks/gitleaks v8.18.2
  • streetsidesoftware/cspell-cli v8.7.0
regex
.github/workflows/release.yml
  • semantic-release 24.0.0

  • Check this box to trigger a request for Renovate to run again on this repository

ovirt - wrong vCPU count

Ovirt source reports wring count of vCPUs, it reads vCPU per socket.

For example:
I have VMs that are 2 virtual sockets, 1 core per socket, they are created with just 1 vCPU, thus should be 2.

Link Checker Report

Summary

Status Count
πŸ” Total 2
βœ… Successful 1
⏳ Timeouts 0
πŸ”€ Redirected 0
πŸ‘» Excluded 0
❓ Unknown 0
🚫 Errors 1

Errors per input

Errors in README.md

v1.0.3 null pointer on ovirt

Running latest version againts Netbox 4 and Ovirt results in a nullpointer:

2024/05/13 12:58:38 ovirt.go:123 INFO (ovirt-test): Successfully synced syncDatacenters-fm in 0.000088 seconds
panic: reflect: slice index out of range

goroutine 23 [running]:
reflect.Value.Index({0xb2bfc0?, 0xc0039920c8?, 0x4?}, 0xdfb0a1?)
/usr/local/Cellar/go/1.22.2/libexec/src/reflect/value.go:1430 +0x167
github.com/bl4ko/netbox-ssot/internal/utils.convertSliceToComparableSlice({0xb2bfc0?, 0xc0039920c8?, 0xa9a0af?})
/Users/jurikononov/src/tmp/netbox-ssot/internal/utils/diff_map.go:240 +0x46
github.com/bl4ko/netbox-ssot/internal/utils.addSliceDiff({0xb2bfc0?, 0xc003992088?, 0x1?}, {0xb2bfc0?, 0xc0039920c8?, 0xa9a0a9?}, {0xa9a0af, 0x4}, 0x38?, 0xc002b3f6e0)
/Users/jurikononov/src/tmp/netbox-ssot/internal/utils/diff_map.go:219 +0x1fb
github.com/bl4ko/netbox-ssot/internal/utils.JSONDiffMapExceptID({0xd57e80, 0xc003992080}, {0xd57e80, 0xc0039920c0}, 0x0, 0xc00015ff20)
/Users/jurikononov/src/tmp/netbox-ssot/internal/utils/diff_map.go:166 +0x985
github.com/bl4ko/netbox-ssot/internal/utils.JSONDiffMapExceptID({0xb8e540, 0xc003994000}, {0xb8e540, 0xc00037c808}, 0x0, 0xc00015ff20)
/Users/jurikononov/src/tmp/netbox-ssot/internal/utils/diff_map.go:121 +0x510
github.com/bl4ko/netbox-ssot/internal/netbox/inventory.(*NetboxInventory).AddClusterType(0xc0000de1e0, {0xf3c518, 0xc00085e360}, 0xc003994000)
/Users/jurikononov/src/tmp/netbox-ssot/internal/netbox/inventory/add_items.go:335 +0x255
github.com/bl4ko/netbox-ssot/internal/source/ovirt.(*OVirtSource).syncClusters(0xc0007ff500, 0xc0000de1e0)
/Users/jurikononov/src/tmp/netbox-ssot/internal/source/ovirt/ovirt_sync.go:102 +0x156
github.com/bl4ko/netbox-ssot/internal/source/ovirt.(*OVirtSource).Sync(0xc0007ff500, 0xc0000de1e0)
/Users/jurikononov/src/tmp/netbox-ssot/internal/source/ovirt/ovirt.go:118 +0x2fb
main.main.func1({0xf3c518, 0xc00085e360}, {0xf3a7a8, 0xc0007ff500})
/Users/jurikononov/src/tmp/netbox-ssot/cmd/netbox-ssot/main.go:98 +0x2f1
created by main.main in goroutine 1
/Users/jurikononov/src/tmp/netbox-ssot/cmd/netbox-ssot/main.go:78 +0x67e

Link Checker Report

Summary

Status Count
πŸ” Total 26
βœ… Successful 25
⏳ Timeouts 0
πŸ”€ Redirected 0
πŸ‘» Excluded 0
❓ Unknown 0
🚫 Errors 1

Errors per input

Errors in internal/devices/README.md

  • [ERR] file:///home/runner/work/netbox-ssot/netbox-ssot/internal/devices/data.go | Failed: Cannot find file
    Full Github Actions output

vmware serialnumber input sanitation

Currently virtual appliances are being created as devices, in fact there is no physical devices and thing runs on a VM instead of physical device. This creates two problems:

  • model is inaccurate and virtual appliances should be modeled as VM, not as physical device. The question remains how to detect if appliance is virtual or physical from the APIs
  • virtual appliances often have weird and long generated serial numbers, that are not accepted by netbox due to char limit

ERROR (vc-test): failed to add vmware host virtual-vsan-host.local with error: unexpected status code: 400: {"serial":["Ensure this field has no more than 50 characters."]}

I'd suggest there are 2 things:

  1. Ignore malformed serial numbers (if sn is longer than 50 characters, that in real world rarely happens, ignore it in case device reports garbage)
  2. Investigate and remodel virtual appliances as part of compute cluster. (affects all currently supported hypervisors)

I will think of the second point for some time in detils

ovirt - cluster device error

Hello,

On repetitive runs ovirt source integrations somehow want to create cluster devices again. I think it should edit existing ones?

The Error:
ERROR (ovirt-test): failed to add oVirt host my-ovird-node01.local with error: unexpected status code: 400: {"asset_tag":["device with this asset tag already exists."]}

Exclude NSX segments

Hello,

I've tested this against a vCenter bound to NSX and with some NSX-managed overlay segments. They seem to be read as regular portgroups/VLANs, however attempts to create or modify them on Netbox fails as their VLAN ID is unset.

rest.go:132         DEBUG   (vcenter-test): Creating objects.Vlan with path /api/ipam/vlans/ with data: Vlan{ID: 0, Name: NSX-TEST, Vid: 0, Status: active}
main.go:101         ERROR   (vcenter-test): unexpected status code: 400: {"vid":["This field is required."]}
main.go:120         INFO    (main): Skipping removing orphaned objects...
main.go:130         INFO    (main): ⚠ syncing of source cor-vvc-10-p-ar failed

I'm not sure what additional information I can provide.

vmware unclustered VM

If VM is running on a Host that is not a member of any cluster, it is failind to be added to the Netbox:

ERROR (vcenter-test): failed to sync vmware VM some-vm-in-single-node.local: unexpected status code: 400: {"device":["Must specify a cluster when assigning a host device."]}

Either a single node Cluster should be created for this use-case or host device should not be specified for that VM.

Link Checker Report

Summary

Status Count
πŸ” Total 2
βœ… Successful 1
⏳ Timeouts 0
πŸ”€ Redirected 0
πŸ‘» Excluded 0
❓ Unknown 0
🚫 Errors 1

Errors per input

Errors in README.md

vmware initialisation null pointer

I found a case where host.Config is nil:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x170 pc=0xa53c88]

goroutine 32 [running]:
github.com/bl4ko/netbox-ssot/internal/source/vmware.(*VmwareSource).InitHosts(0xc00128c000, {0xf3b910, 0xc0011d3d60}, 0xc001130f50)
/Users/jurikononov/src/tmp/netbox-ssot/internal/source/vmware/vmware_init.go:140 +0x448
github.com/bl4ko/netbox-ssot/internal/source/vmware.(*VmwareSource).Init(0xc00128c000)
/Users/jurikononov/src/tmp/netbox-ssot/internal/source/vmware/vmware.go:175 +0x1439
main.main.func1({0xf3b8d8, 0xc00126f7d0}, {0xf39b90, 0xc00128c000})
/Users/jurikononov/src/tmp/netbox-ssot/cmd/netbox-ssot/main.go:87 +0x154
created by main.main in goroutine 1
/Users/jurikononov/src/tmp/netbox-ssot/cmd/netbox-ssot/main.go:78 +0x67e

I suggest that vmware_init.go:140 should be wither split into two ifs or second condition should be removed (a bit redundant).
Got nullpointer for host.Config.Network when host.Config is nil.

vmware_init.go:140
if host.Config != nil && host.Config.Network != nil

Asset tag vs source-id

I see that currently solution uses asset tag as field to store UID/GUID from the reference system. The Asset tag field is essential for inventory numbers / ERP related tag for the DCIM functionality of the Netbox, therefore I would suggest to migrate this to the source-id custom field. I think source-id is already being used for some objects, so it just need unification?

vcenter null pointer due to irresponsive node

If ESXi node is not available, I get this nullpointer:

[signal SIGSEGV: segmentation violation code=0x1 addr=0x170 pc=0xa56b73]

goroutine 13 [running]:
github.com/bl4ko/netbox-ssot/internal/source/vmware.(*VmwareSource).syncHostPhysicalNics(_, , {{{{{...}, {...}, {...}}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}}, ...}, ...}, ...)
/Users/jurikononov/src/netbox-ssot/internal/source/vmware/vmware_sync.go:285 +0x93
github.com/bl4ko/netbox-ssot/internal/source/vmware.(*VmwareSource).syncHostNics(
, _, {{{{{...}, {...}, {...}}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}}, ...}, ...}, ...)
/Users/jurikononov/src/netbox-ssot/internal/source/vmware/vmware_sync.go:261 +0x65
github.com/bl4ko/netbox-ssot/internal/source/vmware.(*VmwareSource).syncHosts(0xc000dae000, 0xc0000da000)
/Users/jurikononov/src/netbox-ssot/internal/source/vmware/vmware_sync.go:246 +0xb6a
github.com/bl4ko/netbox-ssot/internal/source/vmware.(*VmwareSource).Sync(0xc000dae000, 0xc0000da000)
/Users/jurikononov/src/netbox-ssot/internal/source/vmware/vmware.go:209 +0x2fb
main.main.func1({0xf3bbb8, 0xc000da0570}, {0xf39e70, 0xc000dae000})
/Users/jurikononov/src/netbox-ssot/cmd/netbox-ssot/main.go:98 +0x2f1
created by main.main in goroutine 1
/Users/jurikononov/src/netbox-ssot/cmd/netbox-ssot/main.go:78 +0x67e

linked vcenter null pointer

I get a nullpointer exception on linked vcenter import. Looks like code cannot handle clusters with no vswitches :)

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x170 pc=0xa53c88]

goroutine 24 [running]:
github.com/bl4ko/netbox-ssot/internal/source/vmware.(*VmwareSource).InitHosts(0xc000b3c0f0, {0xf3b910, 0xc0009d9720}, 0xc000ab5a90)
/Users/jurikononov/src/netbox-ssot/internal/source/vmware/vmware_init.go:140 +0x448
github.com/bl4ko/netbox-ssot/internal/source/vmware.(*VmwareSource).Init(0xc000b3c0f0)
/Users/jurikononov/src/netbox-ssot/internal/source/vmware/vmware.go:175 +0x1439
main.main.func1({0xf3b8d8, 0xc000b01890}, {0xf39b90, 0xc000b3c0f0})
/Users/jurikononov/src/netbox-ssot/cmd/netbox-ssot/main.go:87 +0x154
created by main.main in goroutine 1
/Users/jurikononov/src/netbox-ssot/cmd/netbox-ssot/main.go:78 +0x67e

Ovirt - orphaned cleanup null pointer

I get this error

2024/05/03 05:41:10 INFO (main): Cleaning up orphaned objects...
panic: assignment to entry in nil map

goroutine 19 [running]:
github.com/bl4ko/netbox-ssot/internal/netbox/inventory.(*NetboxInventory).AddVM(0xc0000de1e0, {0xf3b8b8, 0xc0008be120}, 0xc0013d0540)
C:/BIT/New folder/netbox-ssot/internal/netbox/inventory/add_items.go:761 +0x6dd
github.com/bl4ko/netbox-ssot/internal/source/ovirt.(*OVirtSource).syncVms(0xc00008c820, 0xc0000de1e0)
C:/BIT/New folder/netbox-ssot/internal/source/ovirt/ovirt_sync.go:638 +0xe5
github.com/bl4ko/netbox-ssot/internal/source/ovirt.(*OVirtSource).Sync(0xc00008c820, 0xc0000de1e0)
C:/BIT/New folder/netbox-ssot/internal/source/ovirt/ovirt.go:107 +0x2fb
main.main.func1({0xf3b8b8, 0xc0008be120}, {0xf39b48, 0xc00008c820})
C:/BIT/New folder/netbox-ssot/cmd/netbox-ssot/main.go:98 +0x2f1
created by main.main in goroutine 1
C:/BIT/New folder/netbox-ssot/cmd/netbox-ssot/main.go:78 +0x67e

1.0 breaks v3 support and panics

Running SSOT 1.0 against Netbox 3.7.8 panics:

panic: reflect: slice index out of range

goroutine 1 [running]:
reflect.Value.Index({0xb2c040?, 0xc00037a438?, 0x4a2275?}, 0xad2bd8?)
/usr/local/Cellar/go/1.22.2/libexec/src/reflect/value.go:1430 +0x167
github.com/bl4ko/netbox-ssot/internal/utils.convertSliceToComparableSlice({0xb2c040?, 0xc00037a438?, 0x197?})
/Users/jurikononov/src/tmp/netbox-ssot/internal/utils/diff_map.go:243 +0x46
github.com/bl4ko/netbox-ssot/internal/utils.addSliceDiff({0xb2c040?, 0xc00036d498?, 0x1?}, {0xb2c040?, 0xc00037a438?, 0xad2bd2?}, {0xad2bd8, 0xc}, 0x58?, 0xc000324b70)
/Users/jurikononov/src/tmp/netbox-ssot/internal/utils/diff_map.go:219 +0x22a
github.com/bl4ko/netbox-ssot/internal/utils.JSONDiffMapExceptID({0xb8f1e0, 0xc00036d450}, {0xb8f1e0, 0xc00037a3f0}, 0x0, 0xc00015fef0)
/Users/jurikononov/src/tmp/netbox-ssot/internal/utils/diff_map.go:166 +0x985
github.com/bl4ko/netbox-ssot/internal/netbox/inventory.(*NetboxInventory).AddCustomField(0xc0000dc1e0, {0xf3c378, 0xc00015fec0}, 0xc00036d450)
/Users/jurikononov/src/tmp/netbox-ssot/internal/netbox/inventory/add_items.go:259 +0x145
github.com/bl4ko/netbox-ssot/internal/netbox/inventory.(*NetboxInventory).InitSsotCustomFields(0xc0000dc1e0, {0xf3c378, 0xc00015fec0})
/Users/jurikononov/src/tmp/netbox-ssot/internal/netbox/inventory/init_items.go:343 +0x199
github.com/bl4ko/netbox-ssot/internal/netbox/inventory.(*NetboxInventory).Init(0xc0000dc1e0)
/Users/jurikononov/src/tmp/netbox-ssot/internal/netbox/inventory/inventory.go:217 +0xb9d
main.main()
/Users/jurikononov/src/tmp/netbox-ssot/cmd/netbox-ssot/main.go:51 +0x532

Ideally it should check API version and return error that API is unsupported or so :)

Soft delete and changelog

In the CMDB/IPAN/DCIM one of the essential roles is to have historical information on what was the object state and why at the point of time. Therefore I think there should be a soft delete option for objects that are either currently not observed of have been deleted, otherwise the whole history (changelog) is lost together with the record.

Feature Request: renaming filteres

Support with search/replace for vc/ovirt Datacenter renames. The case if where in different systems admins used different names, aga:

ovirt: TestDC
vcenter: TESTDC

Currently it creates slug collision, would be good to being able to merge them by rewriting it for one of the sources.

Fortinet

Is it planned to have Fortinet under this project too?

HTTPScheme is not being read from config.yaml.

If you set HTTPScheme: "http" in the config.yaml it is not getting picked up by the parser and is being set to "https" by default.

netbox:
apiToken: "170397d82a14d3b6d50a74d1b7241fc90b0ef567"
hostname: "127.0.0.1"
port: 8080
timeout: 30
sourcePriority: ["olvm"] # Not required, but recommended
HTTPScheme: "http"
validateCert: false

Debug output:

❯ go run main.go
Netbox-SSOT has started at 2024-03-12T11:13:19-04:00
2024/03/12 11:13:19 main.go:38 DEBUG (main): Parsed Logger config: LoggerConfig{Level: 0, Dest: stdout}
2024/03/12 11:13:19 main.go:39 DEBUG (main): Parsed Netbox config: NetboxConfig{ApiToken: 170397d82a14d3b6d50a74d1b7241fc90b0ef567, Hostname: 127.0.0.1, Port: 8080, HTTPScheme: https, ValidateCert: false, Timeout: 30, Tag: netbox-ssot, TagColor: 00add8, RemoveOrphans: true}
2024/03/12 11:13:19 main.go:40 DEBUG (main): Parsed Source config: [SourceConfig{Name: olvm, Type: ovirt, HTTPScheme: https, Hostname: he-prod.phx.ipengine360.net, Port: 443, Username: mark.hudson@internal, Password: xyzpok-mAfzyq-xekpy1, PermittedSubnets: [], ValidateCert: false, Tag: Source: olvm, TagColor: aa1409, HostSiteRelations: [], ClusterSiteRelations: [], clusterTenantRelations: [], HostTenantRelations: [], VmTenantRelations [], VlanGroupRelations: [], VlanTenantRelations: []}]
2024/03/12 11:13:19 main.go:48 DEBUG (main): Netbox inventory: NetBoxInventory{Logger: &{Logger:0xc00033e690 level:0}, NetboxConfig: NetboxConfig{ApiToken: 170397d82a14d3b6d50a74d1b7241fc90b0ef567, Hostname: 127.0.0.1, Port: 8080, HTTPScheme: https, ValidateCert: false, Timeout: 30, Tag: netbox-ssot, TagColor: 00add8, RemoveOrphans: true}...}
2024/03/12 11:13:19 main.go:50 INFO (main): Starting initializing netbox inventory
2024/03/12 11:13:19 inventory.go:171 DEBUG (inventory): Initializing Netbox API with baseURL: https://127.0.0.1:8080
2024/03/12 11:13:19 api.go:44 WARNING (inventory): TLS certificate validation is disabled
2024/03/12 11:13:19 rest.go:61 DEBUG (inventory): Getting all objects.CustomField from Netbox
2024/03/12 11:13:19 rest.go:64 DEBUG (inventory): Getting objects.CustomField with limit=100 and offset=0
2024/03/12 11:13:19 main.go:53 ERROR (main): Get "https://127.0.0.1:8080/api/extras/custom-fields/?limit=100&offset=0": tls: first record does not look like a TLS handshake: InitCustomFields-fm

The automated release is failing 🚨

🚨 The automated release from the main branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you can benefit from your bug fixes and new features again.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can fix this πŸ’ͺ.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the main branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here are some links that can help you:

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.


Cannot push to the Git repository.

semantic-release cannot push the version tag to the branch main on the remote Git repository with URL https://x-access-token:[secure]@github.com/bl4ko/netbox-ssot.

This can be caused by:


Good luck with your project ✨

Your semantic-release bot πŸ“¦πŸš€

Link Checker Report

Summary

Status Count
πŸ” Total 26
βœ… Successful 25
⏳ Timeouts 0
πŸ”€ Redirected 0
πŸ‘» Excluded 0
❓ Unknown 0
🚫 Errors 1

Errors per input

Errors in internal/devices/README.md

  • [ERR] file:///home/runner/work/netbox-ssot/netbox-ssot/internal/devices/data.go | Failed: Cannot find file
    Full Github Actions output

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.