How to allocate next available free subnet about ipaddress-go

[seancfoley]

According to your code:

for net in available.iter_cidrs():
  if net.prefixlen <= needed_prefix:
    print(next(net.subnet(needed_prefix, 1)))

So it finds the first in the list with a prefix less than or equal to 25. So the one with /22 is chosn, since 22 < 25.

[seancfoley]

This library does not have a type equivalent to IPSet. In fact, neither does Python's ipaddress module, I was unaware that Python's netaddr had this additional functionality. However, this library does have the required API to do this. I can provide a code sample shortly.

[seancfoley]

The code in this comment is equivalent to this. The CIDRSet type has the functionality required from Python's IPSet:

package main

import (
	"fmt"
	"github.com/seancfoley/ipaddress-go/ipaddr"
)

func main() {
	cidr := "10.0.0.0/16"

	assigned := []string{
		//"10.0.0.7/27", not a CIDR block
		"10.0.0.7/32",
		"10.0.0.64/26",
		"10.0.0.128/25",
		"10.0.1.0/24",
		"10.0.2.0/25",
	}

	needed_prefix := 25
	available := CIDRSet{[]*ipaddr.IPAddress{convertStr(cidr)}}
	for _, str := range assigned {
		available.Subtract(convertStr(str))
	}

	fmt.Println("available:", available.Cidrs)
	for _, addr := range available.Cidrs {
		if addr.GetPrefixLen().Len() <= needed_prefix {
			fmt.Println("selected:", addr)
			break
		}
	}
}

func convertStr(addrStr string) *ipaddr.IPAddress {
	return ipaddr.NewIPAddressString(addrStr).GetAddress()
}

type CIDRSet struct {
	Cidrs []*ipaddr.IPAddress
}

// Add computes the set union
func (set *CIDRSet) Add(cidr *ipaddr.IPAddress) {
	set.Cidrs = cidr.MergeToPrefixBlocks(set.Cidrs...)
}

// computes the set difference
// removes from this set all addresses in the given cidr
func (set *CIDRSet) Subtract(cidr *ipaddr.IPAddress) {
	existingAddrs := set.Cidrs
	for i := 0; i < len(existingAddrs); i++ {
		existing := existingAddrs[i]
		if existing.Contains(cidr) {
			// remove the existing
			existingAddrs = append(existingAddrs[:i], existingAddrs[i+1:]...)

			// get the remainder
			remainderAddrs := existing.Subtract(cidr)

			// convert each remainder to CIDRs added to the set
			for _, remainder := range remainderAddrs {
				blocks := remainder.SpanWithPrefixBlocks()
				existingAddrs = append(existingAddrs, blocks...)
			}
			break
		}
		if cidr.Contains(existing) {
			existingAddrs = append(existingAddrs[:i], existingAddrs[i+1:]...)
			for ; i < len(existingAddrs); i++ {
				if cidr.Contains(existing) {
					existingAddrs = append(existingAddrs[:i], existingAddrs[i+1:]...)
				}
			}
			break
		}
	}
	set.Cidrs = existingAddrs
}

Output:

available: [10.0.4.0/22 10.0.8.0/21 10.0.16.0/20 10.0.32.0/19 10.0.64.0/18 10.0.128.0/17 10.0.0.0/30 10.0.0.4/31 10.0.0.6/32 10.0.0.8/29 10.0.0.16/28 10.0.0.32/27 10.0.3.0/24 10.0.2.128/25]
selected: 10.0.4.0/22

[micruzz82]

Thanks again Sean for this great work you have done. I'll test this out but this pretty much gives me the next steps to build on top of your library and use it for subnet calculations. Much appreciate all the great work you have done on this.

[seancfoley]

You're welcome.

I just simplified the CIDRSet code a bit, I realized the "Add" was not quite right, and also I did not need the check method.

[seancfoley]

I did not spent time trying to understand what you really wanted your code to do. You wrote some python code, you said you wanted the go equivalent, and so I provided the go equivalent.

As far as what it does, it seems you have a single block, and then you are subtracting from it a list of used blocks, the assigned blocks, and then you have a new list after that, and then you want to select one more from that new list.

Well, if you also remove the one that was previously selected, 10.0.2.128/25, it certainly will no longer be in the final list anymore.

So you'll have a different final list without it, exactly the same except that 10.0.2.128/25 is now gone. And that is exactly what your output shows.

That new list has no /25 subnet. Nor is there any expectation that it would. Previously in this comment you decided to change your code to find an exact prefix match. It will not match anything because there is no /25 subnet. So it is doing precisely what you intended, it is trying to find an exact prefix match, and there is none.

So I presume what you really want your code to do, is to find a block with the largest prefix x such that x <= 25.

So you can change that code to this:

// select block with largest prefix such that prefix still <= needed_prefix
var selected *ipaddr.IPAddress
for _, addr := range available.Cidrs {
	if addr.GetPrefixLen().Len() <= needed_prefix {
		if selected == nil || selected.GetPrefixLen().Len() < addr.GetPrefixLen().Len() {
			selected = addr
		}
	}
}
fmt.Println("selected:", selected)

Now, if you wanted to go a step further, you could use something similar to the partition method in this example.

You may have chosen a block with a prefix < 25, but you can still get a /25 block by partitioning the larger block that you picked. The code would look like this:

func selectBlock(cidr string, assigned []string, needed_prefix int) {
	available := CIDRSet{[]*ipaddr.IPAddress{convertStr(cidr)}}
	for _, str := range assigned {
		available.Subtract(convertStr(str))
	}

	fmt.Println("available:", available.Cidrs)

	// select block with largest prefix such that prefix still <= needed_prefix
	var selected *ipaddr.IPAddress
	for _, addr := range available.Cidrs {
		if addr.GetPrefixLen().Len() <= needed_prefix {
			if selected == nil || selected.GetPrefixLen().Len() < addr.GetPrefixLen().Len() {
				selected = addr
			}
		}
	}
	var unused []*ipaddr.IPAddress
	// partition to get a block that matches the prefix length exactly
	selected, unused = partitionSelected(selected, needed_prefix)
	available.Cidrs = append(available.Cidrs, unused...)
	fmt.Println("selected:", selected)
}

func partitionSelected(block *ipaddr.IPAddress, requiredPrefixLen ipaddr.BitCount) (
	allocated *ipaddr.IPAddress, unused []*ipaddr.IPAddress) {

	if block.GetPrefixLen().Len() < requiredPrefixLen {
		iterator := block.SetPrefixLen(requiredPrefixLen).PrefixBlockIterator()
		allocated = iterator.Next()
		for iterator.HasNext() {
			unused = append(unused, iterator.Next())
		}
	} else {
		allocated = block
	}
	return
}

func main() {
	cidr := "10.0.0.0/16"
	needed_prefix := 25
	assigned = []string{
		"10.0.0.7/32",
		"10.0.0.64/26",
		"10.0.0.128/25",
		"10.0.1.0/24",
		"10.0.2.0/25",
	}

	selectBlock(cidr, assigned, needed_prefix) // 10.0.2.128/25

	assigned = []string{
		"10.0.0.7/32",
		"10.0.0.64/26",
		"10.0.0.128/25",
		"10.0.1.0/24",
		"10.0.2.0/25",
		"10.0.2.128/25",
	}

	selectBlock(cidr, assigned, needed_prefix) // 10.0.3.0/25
}

[micruzz82]

I also had another thought.. if there is a large /16 and if the allocations were made contiguously for /24's but if there is a gap in between, you want to re-start allocation from the beginning of the gap. How could I do this please.?

For example.. lets say we allocated 10 x /24 subnets... for some reason, we had to return subnets from 4 to 8.

In this case, the next free subnet should not be from 11, but it should start from 4 till 8 and then resume from 11

If you would be able to help me with understanding how to scan to find a gap in allocation to mark them as free and reallocate them that would be much appreciated.

[micruzz82]

I have used this in python but I'm not sure how to do this with ipaddress-go. Any help would be much appreciated

from netaddr import *
import math

cidr = '10.0.0.0/16' # CIDR block

assigned = [ # Networks you've already used
  '10.0.0.7/27',
  '10.0.0.64/26',
  '10.0.0.128/25',
  '10.0.1.0/24',
  '10.0.2.0/25'
]

needed_prefix = 25
available = IPSet([cidr]) - IPSet(assigned)
for net in available.iter_cidrs():
  if net.prefixlen <= needed_prefix:
    print(next(net.subnet(needed_prefix, 1)))
    break

[micruzz82]

Thanks very much Sean. I'll raise another issue where this feature for finding available IP space would be useful.

[micruzz82]

I'll debug this as the needed prefix is set to 25 but returns /22

[micruzz82]

Thanks Sean. I think this works exactly perfect.

if addr.GetPrefixLen().Len() == needed_prefix

and it correctly allocated it..

I think the python library does it differently but for what I wanted to output, I changed to == and this is all working very well.

Thanks ever so much for all the help with this!!

[micruzz82]

thanks again.

[micruzz82]

Hi Sean

I just noticed something will running multiple runs of the code:

Run 1:

	assigned := []string{
		"10.0.0.7/32",
		"10.0.0.64/26",
		"10.0.0.128/25",
		"10.0.1.0/24",
		"10.0.2.0/25",
	}

         needed_prefix := 25

Output:
go run test20.go
available: [10.0.4.0/22 10.0.8.0/21 10.0.16.0/20 10.0.32.0/19 10.0.64.0/18 10.0.128.0/17 10.0.0.0/30 10.0.0.4/31 10.0.0.6/32 10.0.0.8/29 10.0.0.16/28 10.0.0.32/27 10.0.3.0/24 10.0.2.128/25]
selected: 10.0.2.128/25

After updating the above list with the selected subnet:

Run 2:

	assigned := []string{
		"10.0.0.7/32",
		"10.0.0.64/26",
		"10.0.0.128/25",
		"10.0.1.0/24",
		"10.0.2.0/25",
		"10.0.2.128/25",
	}
         needed_prefix := 25

Output:
 go run test20.go
available: [10.0.4.0/22 10.0.8.0/21 10.0.16.0/20 10.0.32.0/19 10.0.64.0/18 10.0.128.0/17 10.0.0.0/30 10.0.0.4/31 10.0.0.6/32 10.0.0.8/29 10.0.0.16/28 10.0.0.32/27 10.0.3.0/24]

I'm not quite sure why it didn't pick up a new subnet with the /25 prefix. Any thoughts?

[micruzz82]

It should have allocated the new /25 request from the 10.0.3.0/24 block

[micruzz82]

Thanks very much Sean for this. This is exactly what I was looking for and the code works great. Much appreciated for your help!

[seancfoley]

I've added a new type, PrefixBlockAllocator, which can be used to handle functionality like this.