oschwald / geoip2-golang Goto Github PK

View Code? Open in Web Editor NEW

1.8K 32.0 189.0 138 KB

Unofficial MaxMind GeoIP2 Reader for Go

License: ISC License

Go 100.00%

geoip2 maxmind-db geoip database go geolocation maxmind

geoip2-golang's Introduction

GeoIP2 Reader for Go

This library reads MaxMind GeoLite2 and GeoIP2 databases.

This library is built using the Go maxminddb reader. All data for the database record is decoded using this library. If you only need several fields, you may get superior performance by using maxminddb's Lookup directly with a result struct that only contains the required fields. (See example_test.go in the maxminddb repository for an example of this.)

Installation

go get github.com/oschwald/geoip2-golang

Usage

See GoDoc for documentation and examples.

Example

package main

import (
	"fmt"
	"log"
	"net"

	"github.com/oschwald/geoip2-golang"
)

func main() {
	db, err := geoip2.Open("GeoIP2-City.mmdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()
	// If you are using strings that may be invalid, check that ip is not nil
	ip := net.ParseIP("81.2.69.142")
	record, err := db.City(ip)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Portuguese (BR) city name: %v\n", record.City.Names["pt-BR"])
	if len(record.Subdivisions) > 0 {
		fmt.Printf("English subdivision name: %v\n", record.Subdivisions[0].Names["en"])
	}
	fmt.Printf("Russian country name: %v\n", record.Country.Names["ru"])
	fmt.Printf("ISO country code: %v\n", record.Country.IsoCode)
	fmt.Printf("Time zone: %v\n", record.Location.TimeZone)
	fmt.Printf("Coordinates: %v, %v\n", record.Location.Latitude, record.Location.Longitude)
	// Output:
	// Portuguese (BR) city name: Londres
	// English subdivision name: England
	// Russian country name: Великобритания
	// ISO country code: GB
	// Time zone: Europe/London
	// Coordinates: 51.5142, -0.0931
}

Testing

Make sure you checked out test data submodule:

git submodule init
git submodule update

Execute test suite:

go test

Contributing

Contributions welcome! Please fork the repository and open a pull request with your changes.

License

This is free software, licensed under the ISC license.

geoip2-golang's People

Contributors

Stargazers

Watchers

Forkers

hdczsf scritchley xtaci matveyco wheelcomplex nickbruun sosedoff moorage incontextsolutions spaiter vonwenm viki-org adjust crowdsurge algoadv archsaber opavader streamrail thebitmonk svagner chrisvarao todostreaming waldyrious pubnative harlow x140cc dseevr-archive ppiscuc mcshiwo amnotbusy ketutgunawan kamoljan mzimmerman xiaoyaosun markendev m3t1x i04n rjammala 0yangyang0 mehmetkose pikachule iaodiary atomicjets davidquan hswuhao renanmpimentel vinayselvaraj flameous fecshopsoft stanxii teammomentum gencer exfun qiuyulong gouki777 fypinglearnningzone priestd09 wxhzk 5up3rc saiyn 3dsinteractive c2code maxim-kuderko adbegon modasi epiqm rahulsoibam rowland kkkkk1530 chaimleib pmghong1991 9wfox zxfonline zuoyong8 guanghui1987 kk-kwok isgasho bdfnl farfeduc naughtyfox laolm backwardn jaw0 lspgn hotion oliverhuangchao fly304625 vmintam jackyin68 geminiwayne hulingling heyuanchao anti32 pierrebzl rbcervilla dimkus ailenli gaffaruh fake-cheater reiner030

geoip2-golang's Issues

Crash by: signal SIGBUS: bus error

The program crashed due to memory access violation.
Stacktrace:

fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x7fb9261ca2a3 pc=0x9fb6d9]

goroutine 861220198 [running]:
runtime.throw({0xc1d906?, 0x8?})
        /usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc0002cd738 sp=0xc0002cd708 pc=0x433f51
runtime.sigpanic()
        /usr/local/go/src/runtime/signal_unix.go:815 +0x125 fp=0xc0002cd788 sp=0xc0002cd738 pc=0x449385
github.com/oschwald/maxminddb-golang.nodeReader28.readLeft(...)
        /home/daimon/go/src/dfn/vendor/github.com/oschwald/maxminddb-golang/node.go:29
github.com/oschwald/maxminddb-golang.(*nodeReader28).readLeft(0x40d687?, 0x30?)
        <autogenerated>:1 +0x39 fp=0xc0002cd7a8 sp=0xc0002cd788 pc=0x9fb6d9
github.com/oschwald/maxminddb-golang.(*Reader).traverseTree(0xc000228b40, {0xc0002dbd9c, 0x4, 0x120?}, 0xa?, 0x20)
        /home/daimon/go/src/dfn/vendor/github.com/oschwald/maxminddb-golang/reader.go:286 +0xc3 fp=0xc0002cd7d8 sp=0xc0002cd7a8 pc=0x9f9083
github.com/oschwald/maxminddb-golang.(*Reader).lookupPointer(0xc000228b40, {0xc0002dbd90?, 0xbe0e00?, 0x7fb95132ca01?})
        /home/daimon/go/src/dfn/vendor/github.com/oschwald/maxminddb-golang/reader.go:264 +0x17a fp=0xc0002cd858 sp=0xc0002cd7d8 pc=0x9f8e7a
github.com/oschwald/maxminddb-golang.(*Reader).Lookup(0xc000228b40, {0xc0002dbd90?, 0xc0011ff7f0?, 0xc0002cd918?}, {0xaf2a00, 0xc0006dc5a0})
        /home/daimon/go/src/dfn/vendor/github.com/oschwald/maxminddb-golang/reader.go:137 +0x45 fp=0xc0002cd888 sp=0xc0002cd858 pc=0x9f8445
github.com/oschwald/geoip2-golang.(*Reader).City(0xc000964840, {0xc0002dbd90, 0x10, 0x10})
        /home/daimon/go/src/dfn/vendor/github.com/oschwald/geoip2-golang/reader.go:336 +0x127 fp=0xc0002cd950 sp=0xc0002cd888 pc=0x9fc1e7
github.com/digifront/DFN_backend/services/gateway/handler.getGeo({0xd33010?, 0xc000519bc0?}, {0xc0002dbd80, 0xc})
        /home/daimon/go/src/dfn/services/gateway/handler/handlers.go:520 +0x165 fp=0xc0002cda18 sp=0xc0002cd950 pc=0xaa4e85
...Other code

My code:

func getGeo(ctx context.Context, ip string) (*types.Geo, error) {

	netIP := net.ParseIP(ip)
	var geo types.Geo
	Geo.RLock()
	city, err := Geo.Db.City(netIP)
	Geo.RUnlock()
	if err != nil {
		...
		return &geo, err
	}
	...
	return &geo, nil
}

I have about 500 RPS, this crash happens 1-2 times a month on the production server. The DB file is updated 1 time per week. I tried to specifically reproduce this problem, but I did not succeed.

Reload the database in thread safe manner

The mmdb is updated frequently. I would like to reload the mmdb when the db has changed. How do I safely reload the db while many go routines share the same reader? I understand the reader is thread safe but I have no idea how I can Close and Open the db safely.

Update Procedure

Hi,

what are the implications of using geoipupdate using cron while having an open reference to the database? Is this problematic? How can an update be performed? We are using this library extensively for StriveCDN and would like to know how this situation can have negative impacts.

Best,
Chris

GeoIP2-City.mmdb

"go get" errors out because of "test-data" submodule

Hi I was using your library previously without issues. However when I tried to update my libraries for a fresh install I got the following error.

git --git-dir=/Users/omerkirk/Projects/go/src/github.com/oschwald/geoip2-golang/.git submodule update --init --recursive
No submodule mapping found in .gitmodules for path 'test-data'
package github.com/oschwald/geoip2-golang: exit status 1

This is probably simple to fix with manually cloning, still it would be a good idea to fix it here.

"no such device" error

Hello,

It's not strictly speaking an error in this library, but you can add a check.
If you call geoip2.Open with a path that is a directory instead of a "normal" file,
the error "no such device" is returned.
I think you should check first the file's type.

Why did it occur in my case ?
Because my OS (Linux+Gnome), uncompresses the GeoLite2-Country.mmdb.gz when I double-click it, to a directory named GeoLite2-Country.mmdb which contains a file named data.

Error returned for unknown DBIP-ASN-Lite mmdb file

$ mmdblookup --version

  mmdblookup version 1.4.2

$ mmdblookup --file dbip-asn.mmdb --ip 1.1.1.1

  {
    "autonomous_system_number": 
      13335 <uint32>
    "autonomous_system_organization": 
      "Cloudflare, Inc." <utf8_string>
  }

mmdb file:

https://db-ip.com/db/download/ip-to-asn-lite

Failed on my code:

package main

import (
	"fmt"
	"net"

	mmdb "github.com/oschwald/geoip2-golang"
)

func main() {

	mmdbASN, err := mmdb.Open("dbip-asn.mmdb")
	if err != nil {
		panic(err) // HERE the panic
	}
	ipParsed := net.ParseIP("1.1.1.1")
	asnData, asnErr := mmdbASN.ASN(ipParsed)
	if asnErr != nil {
		fmt.Println(asnData)
	}

	return
}

$ panic: geoip2: reader does not support the "DBIP-ASN-Lite (compat=GeoLite2-ASN)" database type

goroutine 1 [running]:
main.main()
        main.go:14 +0x11b

why some ip can't Parse?

why some ip can't Parse? and another is ok.

e.g. 139.9.204.130

thx your lib.

 ip := net.ParseIP("139.9.204.130")
 record, err := db.City(ip)

Portuguese (BR) city name: 
panic: runtime error: index out of range

goroutine 1 [running]:
main.main()
        geo-ip.go:25 +0x5a6
exit status 2

Switch tests to Testify

The maxminddb module is already using Testify and Go Check doesn't seem to be actively developed anymore.

IPv6 support?

I found MaxMind has an IPv6 database. Does this library support IPv6 queries? Thanks.

try to install geoip2 but got below message

go get error:
go get github.com/oschwald/geoip2-golang

github.com/oschwald/geoip2-golang

../github.com/oschwald/geoip2-golang/reader.go:320: r.mmdbReader.Close() used as value

how can i use it with a pool like mysql

func getIpInfo(ipStr string) (string, string, string, string){
    ipDb, err := geoip2.Open("/www/test/ip/GeoLite2-City_20180327/GeoLite2-City.mmdb")
    if err != nil {
            log.Fatal(err)
    }
    defer ipDb.Close()
    // If you are using strings that may be invalid, check that ip is not nil
    ip := net.ParseIP(ipStr)
    record, err := ipDb.City(ip)
    if err != nil {
        log.Fatal(err)
    }
    return record.Country.IsoCode, record.Country.Names["en"], record.Subdivisions[0].Names["en"], record.City.Names["en-US"]
}

every time use it , it will reopen db connect, them close connect, how can i use it with a pool ip db?

Thread safety

Is the Reader object thread safe? Can multiple threads call City(), Country() etc without having to create multiple Readers? This would be helpful to document.

when will it become official?

For GeoLite2-ASN.mmdb: query organization name with known ASN

Currenty I only retrieve the AutonomousSystemNumber uint at one point in a longer date pipeline which aggregates over multiple IPs. Is there a "decent" way to query the database with the above number and return the corresponding AutonomousSystemOrganization string?

I would be fine with just iterating over the whole file, but so far I only could identify the Networks-oriented traverse functionality.

Is there an elegant way to accomplish this?

Exporting getDBType, databaseType and isTYPE constants

I was thinking that would be handy to export getDBType, databaseType and the isTYPE constants to help users discovering databases schema without having to query the databases and looking for InvalidMethodError errors, Would such change be accepted?

Segmentation violation when running Reader.City

Hello.

When trying to use this library, I am getting a fatal error when caling db.City(ip)

There is the related code :

package processors

import (
	"github.com/lodgify/ngx-lp/format"
	log "github.com/sirupsen/logrus"
	geo "github.com/oschwald/geoip2-golang"
	"net"
	"fmt"
	"time"
	"github.com/lodgify/ngx-lp/configuration"
)

// SetProxyTypeProcessor detects the type of proxy from the LogSource field
func SetGeoIPProcessor(processor Processor, config configuration.Configuration) Processor {
	var componentName = "SetGeoIPProcessor"
	contextLogger := log.WithFields(log.Fields{
		"component": componentName,
	})

	defer initTimeTrack(time.Now(), componentName, contextLogger)

	contextLogger.Debugf("Loading GeoIP database from %s", config.GeoIPDatabaseFile)
	db, err := geo.Open(config.GeoIPDatabaseFile)
	defer db.Close()

	if err != nil {
		contextLogger.Fatal("Failed to load GEOIP database ", err)
	}

	return ProcessorFunc(func(entry format.NginxRequestLogEntry) {
		defer timeTrack(time.Now(), componentName, contextLogger)

		if entry.DeducedIP  == "" {
			contextLogger.Error("Missing deduced IP, cannot process GEOIP analysis")
			processor.Process(entry)
		} else {
			ip := net.ParseIP(entry.DeducedIP)
			data, err := db.City(ip)
			if err != nil {
				contextLogger.Error("Failed to get data from IP", err)
			}

			entry.GeoIP.CityName = data.City.Names["en"]
			entry.GeoIP.PostalCode = data.Postal.Code
			entry.GeoIP.ContinentCode = data.Continent.Code
			entry.GeoIP.ContinentName = data.Continent.Names["en"]
			entry.GeoIP.Coordinates = fmt.Sprintf("%f, %f", data.Location.Latitude, data.Location.Longitude)
			entry.GeoIP.Latitude = data.Location.Latitude
			entry.GeoIP.Longitude = data.Location.Longitude
			entry.GeoIP.CountryName = data.Country.Names["en"]
			entry.GeoIP.CountryCode = data.Country.IsoCode
			entry.GeoIP.Location.Longitude = data.Location.Longitude
			entry.GeoIP.Location.Latitude = data.Location.Latitude

			processor.Process(entry)
		}
	})
}

The init phase works fine, but when the ProcessorFunc run, it's crashing the whole Go runtime :

unexpected fault address 0x7fa662d282a3
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x7fa662d282a3 pc=0x631ab3]

goroutine 7 [running]:
runtime.throw(0x6c778b, 0x5)
	/usr/local/Cellar/go/1.9.2/libexec/src/runtime/panic.go:605 +0x95 fp=0xc42012a880 sp=0xc42012a860 pc=0x429c65
runtime.sigpanic()
	/usr/local/Cellar/go/1.9.2/libexec/src/runtime/signal_unix.go:374 +0x227 fp=0xc42012a8d0 sp=0xc42012a880 pc=0x43f917
github.com/oschwald/maxminddb-golang.(*Reader).readNode(0xc42006f040, 0x60, 0x0, 0x1, 0x0, 0xc42012a9e0)
	/Users/mathieu.poussin/Go/src/github.com/oschwald/maxminddb-golang/reader.go:214 +0xf3 fp=0xc42012a940 sp=0xc42012a8d0 pc=0x631ab3
github.com/oschwald/maxminddb-golang.(*Reader).findAddressInTree(0xc42006f040, 0xc42010e45c, 0x4, 0x4, 0x4, 0x4, 0x0)
	/Users/mathieu.poussin/Go/src/github.com/oschwald/maxminddb-golang/reader.go:187 +0xdd fp=0xc42012a9b0 sp=0xc42012a940 pc=0x63186d
github.com/oschwald/maxminddb-golang.(*Reader).lookupPointer(0xc42006f040, 0xc42010e450, 0x10, 0x10, 0x6b3420, 0xc42012aa01, 0xc4200f4100)
	/Users/mathieu.poussin/Go/src/github.com/oschwald/maxminddb-golang/reader.go:169 +0x9b fp=0xc42012aa20 sp=0xc42012a9b0 pc=0x6315db
github.com/oschwald/maxminddb-golang.(*Reader).Lookup(0xc42006f040, 0xc42010e450, 0x10, 0x10, 0x6573e0, 0xc4200f4100, 0xc, 0xc42010e450)
	/Users/mathieu.poussin/Go/src/github.com/oschwald/maxminddb-golang/reader.go:110 +0x4d fp=0xc42012aa68 sp=0xc42012aa20 pc=0x63135d
github.com/oschwald/geoip2-golang.(*Reader).City(0xc42000efd0, 0xc42010e450, 0x10, 0x10, 0x10, 0x6ca56d, 0x11)
	/Users/mathieu.poussin/Go/src/github.com/oschwald/geoip2-golang/reader.go:239 +0x185 fp=0xc42012ab40 sp=0xc42012aa68 pc=0x633395
github.com/lodgify/ngx-lp/processors.SetGeoIPProcessor.func1(0xc420132000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42010e382, 0xa, ...)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/processors/geoip.go:38 +0x1ff fp=0xc42012adb8 sp=0xc42012ab40 pc=0x63542f
github.com/lodgify/ngx-lp/processors.ProcessorFunc.Process(0xc42004c2c0, 0xc420132000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42010e382, ...)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/processors/processor.go:18 +0x59 fp=0xc42012af98 sp=0xc42012adb8 pc=0x6345e9
github.com/lodgify/ngx-lp/processors.SetDeducedIPProcessor.func1(0xc420132000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42010e382, 0xa, ...)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/processors/deduce_ip.go:32 +0x1f1 fp=0xc42012b1d8 sp=0xc42012af98 pc=0x634d61
github.com/lodgify/ngx-lp/processors.ProcessorFunc.Process(0xc42005d410, 0xc420132000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42010e382, ...)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/processors/processor.go:18 +0x59 fp=0xc42012b3b8 sp=0xc42012b1d8 pc=0x6345e9
github.com/lodgify/ngx-lp/processors.SetEnvironmentProcessor.func1(0xc420132000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42010e382, 0xa, ...)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/processors/detect_environment.go:32 +0x1ad fp=0xc42012b5c8 sp=0xc42012b3b8 pc=0x6350ed
github.com/lodgify/ngx-lp/processors.ProcessorFunc.Process(0xc42005d4a0, 0xc420132000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42010e382, ...)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/processors/processor.go:18 +0x59 fp=0xc42012b7a8 sp=0xc42012b5c8 pc=0x6345e9
github.com/lodgify/ngx-lp/processors.SetProxyTypeProcessor.func1(0xc420132000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42010e382, 0xa, ...)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/processors/proxy_type.go:31 +0x1ad fp=0xc42012b9b8 sp=0xc42012b7a8 pc=0x635a4d
github.com/lodgify/ngx-lp/processors.ProcessorFunc.Process(0xc42005d530, 0xc420132000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42010e382, ...)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/processors/processor.go:18 +0x59 fp=0xc42012bb98 sp=0xc42012b9b8 pc=0x6345e9
main.main.func1(0xc420062460, 0x807a00, 0xc42005d530, 0xc42005a1e0)
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/main.go:60 +0x85 fp=0xc42012bfc0 sp=0xc42012bb98 pc=0x636a25
runtime.goexit()
	/usr/local/Cellar/go/1.9.2/libexec/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc42012bfc8 sp=0xc42012bfc0 pc=0x457501
created by main.main
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/main.go:50 +0x72e

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc4200962fc)
	/usr/local/Cellar/go/1.9.2/libexec/src/runtime/sema.go:56 +0x39
sync.(*WaitGroup).Wait(0xc4200962f0)
	/usr/local/Cellar/go/1.9.2/libexec/src/sync/waitgroup.go:131 +0x72
gopkg.in/mcuadros/go-syslog%2ev2.(*Server).Wait(0xc4200962c0)
	/Users/mathieu.poussin/Go/src/gopkg.in/mcuadros/go-syslog.v2/server.go:305 +0x31
main.main()
	/Users/mathieu.poussin/Go/src/github.com/lodgify/ngx-lp/main.go:67 +0x73c

goroutine 5 [chan receive]:
gopkg.in/mcuadros/go-syslog%2ev2.(*Server).goParseDatagrams.func1(0xc4200962c0)
	/Users/mathieu.poussin/Go/src/gopkg.in/mcuadros/go-syslog.v2/server.go:363 +0x12a
created by gopkg.in/mcuadros/go-syslog%2ev2.(*Server).goParseDatagrams
	/Users/mathieu.poussin/Go/src/gopkg.in/mcuadros/go-syslog.v2/server.go:359 +0x90

goroutine 6 [IO wait]:
internal/poll.runtime_pollWait(0x7fa665edff70, 0x72, 0x0)
	/usr/local/Cellar/go/1.9.2/libexec/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc4200ee098, 0x72, 0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.9.2/libexec/src/internal/poll/fd_poll_runtime.go:85 +0xae
internal/poll.(*pollDesc).waitRead(0xc4200ee098, 0xc420110000, 0x10000, 0x10000)
	/usr/local/Cellar/go/1.9.2/libexec/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).ReadFrom(0xc4200ee080, 0xc420110000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.9.2/libexec/src/internal/poll/fd_unix.go:170 +0x19a
net.(*netFD).readFrom(0xc4200ee080, 0xc420110000, 0x10000, 0x10000, 0x451de0, 0xc42003cdf8, 0xc42003ce50, 0xc42003ce70, 0x40e30e)
	/usr/local/Cellar/go/1.9.2/libexec/src/net/fd_unix.go:208 +0x5e
net.(*UDPConn).readFrom(0xc42000c078, 0xc420110000, 0x10000, 0x10000, 0xc42003cf10, 0x46bccb, 0x6619c0, 0xc420106020)
	/usr/local/Cellar/go/1.9.2/libexec/src/net/udpsock_posix.go:47 +0x6a
net.(*UDPConn).ReadFrom(0xc42000c078, 0xc420110000, 0x10000, 0x10000, 0x264, 0x808f00, 0xc42010c000, 0x0, 0x0)
	/usr/local/Cellar/go/1.9.2/libexec/src/net/udpsock.go:118 +0x72
gopkg.in/mcuadros/go-syslog%2ev2.(*Server).goReceiveDatagrams.func1(0xc4200962c0, 0x80c0c0, 0xc42000c078)
	/Users/mathieu.poussin/Go/src/gopkg.in/mcuadros/go-syslog.v2/server.go:329 +0x103
created by gopkg.in/mcuadros/go-syslog%2ev2.(*Server).goReceiveDatagrams
	/Users/mathieu.poussin/Go/src/gopkg.in/mcuadros/go-syslog.v2/server.go:325 +0x70

Go: 1.9.2
DB: GeoLite2-City_20171107

Any idea of what can be the issue ? In this case the IP trying to be analysed is a private one but I suppose it should not panic for this.

Thanks

Issue with Go 1.18

Since upgrading to Go 1.18 I have an issue similar to https://stackoverflow.com/questions/71507321/go-1-18-build-error-on-mac-unix-syscall-darwin-1-13-go253-golinkname-mus

Can you publish a new version with the new deps update you did?

Thank you

metro_code works with GeoIP2/GeoLIte2 databases

Currently it is a string in the actual database, even though the docs and other APIs suggest that it should be an integer. This may be a bug with the database, but, if not, we should fix our definition of it in the struct.

Lookup ASN org from number

Is there a way to lookup an AS Org given the number? All the lookups seem to be IP based. Even looking at the parent library, it's unclear how/if this could be achieved.

Thanks!

New tag for 1.12 testing

can we get a patch bumped tag ?

CIDR Support

It seems that the current version doesn't support getting the CIDR of an IP. Do we have any plans?

*Reader.City returns the zero value if not found

Hello,

The methods City() and Country should return nil if it finds nothing.

The current behavior is that is returns the zero value: https://github.com/oschwald/geoip2-golang/blob/master/reader.go#L240.

In this case, we have to check if city.GeonameID == 0 to know that it is not found.

Wouldn't it be better to return nil instead ?

What's the difference with oschwald / maxminddb-golang？

Unexpected fault address opening MaxMind DB ISP file

Opening file described at https://www.maxmind.com/en/geoip2-isp-database results in unexpected fault address.

`unexpected fault address 0x7f38fa135003
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x7f38fa135003 pc=0x4e178c]

goroutine 1 [running]:
runtime.throw(0x5156fd, 0x5)
/usr/lib/go-1.16/src/runtime/panic.go:1117 +0x72 fp=0xc00014b860 sp=0xc00014b830 pc=0x435712
runtime.sigpanic()
/usr/lib/go-1.16/src/runtime/signal_unix.go:731 +0x2c8 fp=0xc00014b898 sp=0xc00014b860 pc=0x44a368
github.com/oschwald/maxminddb-golang.nodeReader28.readLeft(...)
/home/mezimm00/go/pkg/mod/github.com/oschwald/[email protected]/node.go:25
github.com/oschwald/maxminddb-golang.(*nodeReader28).readLeft(0xc0001280c0, 0x0, 0x50aea0)
:1 +0x4c fp=0xc00014b8b8 sp=0xc00014b898 pc=0x4e178c
github.com/oschwald/maxminddb-golang.(*Reader).setIPv4Start(0xc000152180)
/home/mezimm00/go/pkg/mod/github.com/oschwald/[email protected]/reader.go:121 +0x66 fp=0xc00014b8f0 sp=0xc00014b8b8 pc=0x4e0046
github.com/oschwald/maxminddb-golang.FromBytes(0x7f38fa135000, 0xaf4e18, 0xaf4e18, 0xaf4e18, 0xaf4e18, 0x0)
/home/mezimm00/go/pkg/mod/github.com/oschwald/[email protected]/reader.go:106 +0x3be fp=0xc00014b9d0 sp=0xc00014b8f0 pc=0x4dfcbe
github.com/oschwald/maxminddb-golang.Open(0x51c797, 0x27, 0x16, 0x1, 0x0)
/home/mezimm00/go/pkg/mod/github.com/oschwald/[email protected]/reader_other.go:40 +0x27c fp=0xc00014ba80 sp=0xc00014b9d0 pc=0x4e0e1c
github.com/oschwald/geoip2-golang.Open(0x51c797, 0x27, 0xc000118740, 0x0, 0x0)
/home/mezimm00/go/pkg/mod/github.com/oschwald/[email protected]/reader.go:252 +0x39 fp=0xc00014bae8 sp=0xc00014ba80 pc=0x4e1cb9`

how to see all cities and countries?

safe for access from multiple goroutines

I am developing an http api on top of this package. I am wondering if the db is safe to access from multiple goroutines.

https://github.com/mseshachalam/geoip/blob/master/main.go#L42 here for each api call i am opening the mmdb file. Is it okay if i open it in the outer scope i.e in the main function, something like this.

db, err := geoip2.Open("GeoLite2-City.mmdb")
defer db.Close()
router.GET("/ip/:ip", func(c *gin.Context) { })

fs.FS support?

Any chance of adding io/fs.FS support to the library (and the underlying libraries, I assume)? That would greatly simplify testing and just generally make the library more versatile.

Thanks.

Switch to ISC license

I would like to switch all the code in this library to the ISC license. The reason for this is to make this library compatible with code that is licensed under the GPL v2.

@oxtoacart, @andrewlook, @sosedoff, and @GregBowyer, as you all contributed code to this library, would you confirm in this issue that this is acceptable to you? Thanks!

suport ipv6？

Bug with certain ip

Hi,
I'm running the example with ip 95.173.163.146

	db, err := geoip2.Open("/geo/GeoLite2-City.mmdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()
	// If you are using strings that may be invalid, check that ip is not nil
	ip := net.ParseIP("95.173.163.146")
	record, err := db.City(ip)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Portuguese (BR) city name: %v\n", record.City.Names["pt-BR"])
	fmt.Printf("English subdivision name: %v\n", record.Subdivisions[0].Names["en"])
	fmt.Printf("Russian country name: %v\n", record.Country.Names["ru"])
	fmt.Printf("ISO country code: %v\n", record.Country.IsoCode)
	fmt.Printf("Time zone: %v\n", record.Location.TimeZone)
	fmt.Printf("Coordinates: %v, %v\n", record.Location.Latitude, record.Location.Longitude)

Db I'm getting from http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz
I have error:

Portuguese (BR) city name: 
2016/11/25 21:11:28 http: panic serving 127.0.0.1:49426: runtime error: index out of range
goroutine 6 [running]:
net/http.(*conn).serve.func1(0xc42007e380)
	/usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0x6796c0, 0xc42000c150)
	/usr/local/go/src/runtime/panic.go:458 +0x243
main.setGeo(0xc4200c00f0, 0xc420045b40, 0x0)
	/home/dodger/projects/go/src/redirector/geo_sevice.go:75 +0x5df
main.sayHello(0x7e2dc0, 0xc4200ea000, 0xc4200c00f0)
	/home/dodger/projects/go/src/redirector/redirector.go:82 +0x234
net/http.HandlerFunc.ServeHTTP(0x6e9ad8, 0x7e2dc0, 0xc4200ea000, 0xc4200c00f0)
	/usr/local/go/src/net/http/server.go:1726 +0x44
net/http.(*ServeMux).ServeHTTP(0x7fdd60, 0x7e2dc0, 0xc4200ea000, 0xc4200c00f0)
	/usr/local/go/src/net/http/server.go:2022 +0x7f
net/http.serverHandler.ServeHTTP(0xc42007e300, 0x7e2dc0, 0xc4200ea000, 0xc4200c00f0)
	/usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc42007e380, 0x7e3300, 0xc420012680)
	/usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2293 +0x44d

len(record.City.Names["pt-BR"] or len(record.City.Names["en"] retturn 0
There are no errors with other IP addresses.
I would appreciate your help.

New release?

I was wondering whether there will be a new release tagged anytime soon? There have been a few additions to the API and I would like to update the Debian packaged version, which currently tracks versioned releases.

mmdb export to csv or to dat

Hello, I have made changes at mmdb file with geoip2-golang, is ther any way to convert mmdb to csv or to export to csv or direct to dat file? Out F5 Load balancers understands only old dat fiiles

Ability to look up city name using GeonameID

It is recommended to store the GeonameID for a city when storing this information in a database as the names of the city can change.

There doesn't appear to be a way to convert this GeonameID back into the name of the city.

It would be nice if the API provided a method to do this.

Lookup?

How can I do Lookup?
maybe geoip2 should have func GetReader() for this?

no check for empty Subdivisions slice

There is no check for empty Subdivisions slice (in example) when printing results. Sometimes it leads to panic.

runtime error: invalid memory address or nil pointer dereference

I have a structure:

// ....

var db *geoip2.Reader

// ....

func GetLocation(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")
	// ....

	record, err := db.City(net.ParseIP(ip))
	if err != nil {
		log.Fatal(err)
	}

	// ....

}

func main() {
	// ....

	db, err := geoip2.Open("/usr/share/GeoIP/GeoIP2-City.mmdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	// ....
	r.HandleFunc("/get-location/{ip}", GetLocation).Methods("GET")
       // ....
	
}

And I create request:

curl -X GET http://localhost:8000/get-location/81.2.69.142

End response:

2019/03/06 12:16:13 http: panic serving 127.0.0.1:47474: runtime error: invalid memory address or nil pointer dereference
goroutine 21 [running]:
net/http.(*conn).serve.func1(0xc00009cfa0)
	/usr/local/go/src/net/http/server.go:1746 +0xd0
panic(0x6c2160, 0x94aae0)
	/usr/local/go/src/runtime/panic.go:513 +0x1b9
github.com/oschwald/geoip2-golang.(*Reader).City(0x0, 0xc00015bae0, 0x10, 0x10, 0x10, 0xc0000b5048, 0x0)
	/home/oleg/Documents/html/go/src/github.com/oschwald/geoip2-golang/reader.go:241 +0x37
main.GetLocation(0x76ad80, 0xc0003c4000, 0xc00044a800)
	/home/oleg/Documents/html/oblivki/localgeo/main.go:140 +0x1e7
net/http.HandlerFunc.ServeHTTP(0x72e9f0, 0x76ad80, 0xc0003c4000, 0xc00044a800)
	/usr/local/go/src/net/http/server.go:1964 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0001180c0, 0x76ad80, 0xc0003c4000, 0xc00044a500)
	/home/oleg/Documents/html/go/src/github.com/gorilla/mux/mux.go:212 +0xd0
net/http.serverHandler.ServeHTTP(0xc0000865b0, 0x76ad80, 0xc0003c4000, 0xc00044a500)
	/usr/local/go/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00009cfa0, 0x76b000, 0xc0000941c0)
	/usr/local/go/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2851 +0x2f5

Instantiate a *geoip2.Reader from an io.Reader

It would be handy if there was a way to create a new *geoip2.Reader from an io.Reader.

My rationale, is that to save a little bit of disk space, it would be nice to be able to store compressed versions of the databases, and load them by passing in a compress/*gzip.Reader, for example.

If you would like, I could code this up, and submit a pull request.

reader.ASN broken?

Tried to use free geolite2 ASN database, but it always return error:

the MaxMind DB file's data section contains bad data (float 64 size of 25)

Tried to re-download, tried different IP's - same error. For example, with IP 8.8.8.8.
No issues with city and country databases.

Memory leak with many sequential lookups to the same reader?

I'm seeing resident memory grow continuously with a high rate (~12k/sec) of sequential lookups to the same reader(multiple workers.. each have their own reader). It looks like certain IP blocks for one reason or another seem to exacerbate it more than others .. I can't put my finger on the why yet but here is as trivial as an example as I could do. Worth noting that closing the reader does in fact clean everything up but it's a serious performance hit. Perhaps it's my use?

import (
	"errors"
	"fmt"
	"log"
	"net"
	"os"
	"github.com/oschwald/geoip2-golang"
)


type Asn struct {
	Number uint   `json:"asnumber"`
	Org    string `json:"asorg"`
}

type AsnDB struct {
	DB *geoip2.Reader
}

func (a *AsnDB) AsnDBInit(path string) error {

	db, geoerr := geoip2.Open(path)
	if geoerr != nil {
		fmt.Printf("Error opening maxmind database:  %s\n", geoerr)
		return errors.New("Error opening maxmind database: " + geoerr.Error())
	}

	a.DB = db
	return nil
}

func (a *AsnDB) GetAsn(ip net.IP) (Asn, error) {
	var as Asn
	asn, serr := a.DB.ASN(ip)

	if serr != nil {
		fmt.Printf("Error looking up %s in maxmind ASN database:  %s\n", ip.String(), serr.Error())
		asn = nil
		return as, errors.New("ASN DB Error looking up ip: " + ip.String() + " error: " + serr.Error())
	}

	as.Number = asn.AutonomousSystemNumber
	as.Org = asn.AutonomousSystemOrganization

	return  as, nil
}

func main ()  {
	done := make(chan bool, 1)

	for i:=0; i < 1000; i++ {
	    go testgeo()
	}	
	<-done
}

func testgeo()  {
	cidrs := []string{"192.0.0.0/8", "10.0.0.0/24", "192.168.0.0/16"}
        var db AsnDB
        aserr := db.AsnDBInit("/path/to/db")
        if aserr != nil {
                fmt.Println("Could not open asn db: " + aserr.Error())
                os.Exit(1)
        }

        for {
            for i:=0; i < len(cidrs); i++ {
                ip, ipnet, err := net.ParseCIDR(cidrs[i])
                if err != nil {
                        log.Fatal(err)
                }
                for ip := ip.Mask(ipnet.Mask); ipnet.Contains(ip); inc(ip) {
                    org,_ := db.GetAsn(ip)
                    fmt.Println(ip.String() + " org: " + org.Org)
                }
            }
        }
}

func inc(ip net.IP) {
    for j := len(ip)-1; j>=0; j-- {
	ip[j]++
	if ip[j] > 0 {
	    break
	}
    }
}

Consider adding ConnectionType and ISP Examples?

Something like this:

//mmdb ConnectionType
func GetConnectionType(ipStr string) string {
ipObj := net.ParseIP(ipStr)
reader := dbReader("./assets/GeoIP2-Connection-Type.mmdb")
defer reader.Close()
record, err := reader.ConnectionType(ipObj)
if err != nil {
logBadIp(err)
return "unknown"
} else {
return record.ConnectionType
}
}

//mmdb ISP
func GetIsp(ipStr string) (string, string) {
ipObj := net.ParseIP(ipStr)
dbIsp := dbReader("./assets/GeoIP2-ISP.mmdb")
defer dbIsp.Close()
record, err := dbIsp.ISP(ipObj)
if err != nil {
logBadIp(err)
return "unknown", "unknown"
} else {
return record.ISP, record.Organization
}
}

func dbReader(mmdbPath string) *geoip2.Reader {
dbConn, err := geoip2.Open(mmdbPath)
if err != nil {
log.Println(err)
}
return dbConn
}
func logBadIp(err error) {
log.Println(err)
log.Println("bad IP")
}

Feature: fetch ASN by ASN number?

Hi, i'd like to query ASN info based on the ASN number.

the ASN method does not support the GeoLite2-City database

I am trying to get the ISP of an ip address. In my /usr/local/etc/GeoIP.conf file I have the ASN database:

  EditionIDs GeoLite2-City GeoLite2-Country GeoLite2-ASN

The code:

```
 m.DB = "/usr/share/GeoIP/GeoLite2-City.mmdb"
db, err := geoip2.Open(m.DB)
if err != nil {
	return nil, err
}

defer db.Close()

c, err := db.ASN(ip)
if err != nil {
	return nil, err
}


The error I get is:

    geoip2: the ASN method does not support the GeoLite2-City database

How do I get the ISP?

Add go module support

Large memory usage

Firstly I want to say thank you for the library.

Secondly, I have noticed that it is using 800-1000 megs of memory. Does that sound correct?

@alexclifford

Database specific methods should check database type

I can't get ISP infos, I always get an empty result:

{
        "AutonomousSystemNumber": 0,
        "AutonomousSystemOrganization": "",
        "ISP": "",
        "Organization": ""
}

Script:

package main

import (
    "fmt"
    "net"
    "encoding/json"
    "github.com/oschwald/geoip2-golang"
)

var db, err = geoip2.Open("GeoLite2-Country.mmdb")

func p(input interface{}) {
    var output []byte
    output, _ = json.MarshalIndent(input, "", " ")
    fmt.Println(string(output))
}

func main() {
    ip := "8.8.8.8"

    if err != nil {
        fmt.Println(err.Error())
    }

    recordCity, err := db.City(net.ParseIP(ip))
    if err != nil {
        fmt.Println(err.Error())
    }

    recordISP, err := db.ISP(net.ParseIP(ip))
    if err != nil {
        fmt.Println(err.Error())
    }

    p(recordCity)
    p(recordISP)
}

Output: https://gist.github.com/fnkr/68a6726ae544435a4061#file-output-json-L69-L74