GithubHelp home page GithubHelp logo

dnscache's Introduction

DNS Lookup Cache

license Go Report Card Build Status Coverage godoc

The dnscache package provides a DNS cache layer to Go's net.Resolver.

Install

Install using the "go get" command:

go get -u github.com/rs/dnscache

Usage

Create a new instance and use it in place of net.Resolver. New names will be cached. Call the Refresh method at regular interval to update cached entries and cleanup unused ones.

resolver := &dnscache.Resolver{}

// First call will cache the result
addrs, err := resolver.LookupHost(context.Background(), "example.com")

// Subsequent calls will use the cached result
addrs, err = resolver.LookupHost(context.Background(), "example.com")

// Call to refresh will refresh names in cache. If you pass true, it will also
// remove cached names not looked up since the last call to Refresh. It is a good idea
// to call this method on a regular interval.
go func() {
    t := time.NewTicker(5 * time.Minute)
    defer t.Stop()
    for range t.C {
        resolver.Refresh(true)
    }
}()

If you are using an http.Transport, you can use this cache by specifying a DialContext function:

r := &dnscache.Resolver{}
t := &http.Transport{
    DialContext: func(ctx context.Context, network string, addr string) (conn net.Conn, err error) {
        host, port, err := net.SplitHostPort(addr)
        if err != nil {
            return nil, err
        }
        ips, err := r.LookupHost(ctx, host)
        if err != nil {
            return nil, err
        }
        for _, ip := range ips {
            var dialer net.Dialer
            conn, err = dialer.DialContext(ctx, network, net.JoinHostPort(ip, port))
            if err == nil {
                break
            }
        }
        return
    },
}

In addition to the Refresh method, you can RefreshWithOptions. This method adds an option to persist resource records on failed lookups

r := &Resolver{}
options := dnscache.ResolverRefreshOptions{}
options.ClearUnused = true
options.PersistOnFailure = false
resolver.RefreshWithOptions(options)

dnscache's People

Contributors

101glover avatar am3o avatar daveoxley avatar deme0607 avatar florianloch avatar garionion avatar otoolec avatar rbeuque74 avatar rs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dnscache's Issues

Data race

Data race when calling Refresh. For my benchmark I was running Refresh every 5 seconds.

	go func() {
		t := time.NewTicker(5 * time.Second)
		defer t.Stop()
		for range t.C {
			r.Refresh(true)
		}
	}()
==================
WARNING: DATA RACE                                                                                                                                                  
Read at 0x00c42021c4a8 by goroutine 8:                                                                                                                              
  github.com/iflix-letsplay/playlist-service/vendor/github.com/rs/dnscache.(*Resolver).Refresh()                                                                    
      /go/src/github.com/iflix-letsplay/playlist-service/vendor/github.com/rs/dnscache/dnscache.go:55 +0x290                                                        
  github.com/iflix-letsplay/playlist-service/utils.GetHttpClient.func1()                                                                                            
      /go/src/github.com/iflix-letsplay/playlist-service/utils/client.go:20 +0xee                                                                                   

Previous write at 0x00c42021c4a8 by goroutine 47:                                                                                                                   
  github.com/iflix-letsplay/playlist-service/vendor/github.com/rs/dnscache.(*Resolver).lookup()                                                                     
      /go/src/github.com/iflix-letsplay/playlist-service/vendor/github.com/rs/dnscache/dnscache.go:167 +0x2a1                                                       
  github.com/iflix-letsplay/playlist-service/vendor/github.com/rs/dnscache.(*Resolver).LookupHost()                                                                 
      /go/src/github.com/iflix-letsplay/playlist-service/vendor/github.com/rs/dnscache/dnscache.go:44 +0x123                                                        
  github.com/iflix-letsplay/playlist-service/utils.glob..func1()                                                                                                    
      /go/src/github.com/iflix-letsplay/playlist-service/utils/client.go:30 +0xde                                                                                   
  net/http.(*Transport).dial()                                                                                                                                      
      /usr/local/go/src/net/http/transport.go:884 +0x33c                                                                                                            
  net/http.(*Transport).dialConn()                                                                                                                                  
      /usr/local/go/src/net/http/transport.go:1060 +0x3027                                                                                                          
  net/http.(*Transport).getConn.func4()                                                                                                                             
      /usr/local/go/src/net/http/transport.go:943 +0xa2                                                                                                             

Goroutine 8 (running) created at:                                                                                                                                   
  github.com/iflix-letsplay/playlist-service/utils.GetHttpClient()                                                                                                  
      /go/src/github.com/iflix-letsplay/playlist-service/utils/client.go:16 +0x42                                                                                   
  github.com/iflix-letsplay/playlist-service/api.init()                                                                                                             
      github.com/iflix-letsplay/playlist-service/api/_test/_obj_test/init.go:25 +0x195                                                                              
  main.init()                                                                                                                                                       
      <autogenerated>:1 +0xae

Goroutine 47 (finished) created at:
  net/http.(*Transport).getConn()
      /usr/local/go/src/net/http/transport.go:942 +0x5c8
  net/http.(*Transport).RoundTrip()
      /usr/local/go/src/net/http/transport.go:400 +0x973
  net/http.send()
      /usr/local/go/src/net/http/client.go:249 +0x371
  net/http.(*Client).send()
      /usr/local/go/src/net/http/client.go:173 +0x1b9
  net/http.(*Client).Do()
      /usr/local/go/src/net/http/client.go:602 +0x513
  github.com/iflix-letsplay/playlist-service/vendor/github.com/rubyist/circuitbreaker.(*HTTPClient).Do.func1()
      /go/src/github.com/iflix-letsplay/playlist-service/vendor/github.com/rubyist/circuitbreaker/client.go:98 +0x83
  github.com/iflix-letsplay/playlist-service/vendor/github.com/rubyist/circuitbreaker.(*Breaker).CallContext()
      /go/src/github.com/iflix-letsplay/playlist-service/vendor/github.com/rubyist/circuitbreaker/circuitbreaker.go:347 +0x7b
  github.com/iflix-letsplay/playlist-service/vendor/github.com/rubyist/circuitbreaker.(*Breaker).Call()
      /go/src/github.com/iflix-letsplay/playlist-service/vendor/github.com/rubyist/circuitbreaker/circuitbreaker.go:334 +0x78
  github.com/iflix-letsplay/playlist-service/vendor/github.com/rubyist/circuitbreaker.(*HTTPClient).Do()
      /go/src/github.com/iflix-letsplay/playlist-service/vendor/github.com/rubyist/circuitbreaker/client.go:97 +0x2a6
  github.com/iflix-letsplay/playlist-service/lib.doRequest()
      /go/src/github.com/iflix-letsplay/playlist-service/lib/playlist.go:122 +0x31d
  github.com/iflix-letsplay/playlist-service/lib.GetPlaylist()
      /go/src/github.com/iflix-letsplay/playlist-service/lib/playlist.go:27 +0x5b0
  github.com/iflix-letsplay/playlist-service/api.retrievePlaylistHandler()
      github.com/iflix-letsplay/playlist-service/api/_test/_obj_test/getplaylist.go:51 +0x848
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:1918 +0x51
  net/http.serverHandler.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2619 +0xbc
  net/http.(*conn).serve()
      /usr/local/go/src/net/http/server.go:1801 +0x83b
==================
--- FAIL: BenchmarkRetrievePlaylist
        benchmark.go:147: race detected during execution of benchmark
FAIL

Program Panic

log:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x41d6c8]
goroutine 24456420 [running]:
[project path]/vendor/github.com/rs/dnscache.(*Resolver).load(0xc00077d7c0, 0xc002218b40, 0x29, 0xc001c11968, 0x451937, 0x0, 0xc001c11980, 0x2, 0x2)
#011/.go/src/[project path]/vendor/github.com/rs/dnscache/dnscache.go:160 +0x78
[project path]/vendor/github.com/rs/dnscache.(*Resolver).lookup(0xc00077d7c0, 0x1232d60, 0xc0000b2010, 0xc002218b40, 0x29, 0xc002218b40, 0x29, 0x28, 0x0, 0xc0035b0000)
#011/.go/src/[project path]/vendor/github.com/rs/dnscache/dnscache.go:77 +0x49
[project path]/vendor/github.com/rs/dnscache.(*Resolver).LookupHost(0xc00077d7c0, 0x1232d60, 0xc0000b2010, 0xc002218b10, 0x28, 0xc001c11ab0, 0x41716f, 0xc001925320, 0x120, 0x110)
#011/.go/src/[project path]/vendor/github.com/rs/dnscache/dnscache.go:44 +0xcd
[project name]/lib/httpx.dnsCachingDialContext.func1(0x1232d60, 0xc0000b2010, 0x1059afd, 0x3, 0xc002218b10, 0x2b, 0x4102a3, 0x682099, 0xc0000b2010, 0xef9640)
#011/.go/src/[project name]/lib/httpx/client.go:44 +0xd2
net/http.(*Transport).dial(0xc000463680, 0x1232d60, 0xc0000b2010, 0x1059afd, 0x3, 0xc002218b10, 0x2b, 0xc002874c00, 0x8, 0x8, ...)
#011/usr/local/go/src/net/http/transport.go:916 +0x20f
net/http.(*Transport).dialConn(0xc000463680, 0x1232d60, 0xc0000b2010, 0x0, 0xc0035b0000, 0x4, 0xc002218b10, 0x2b, 0x0, 0x0, ...)
#011/usr/local/go/src/net/http/transport.go:1240 +0x313
net/http.(*Transport).getConn.func4(0xc000463680, 0x1232d60, 0xc0000b2010, 0xc0036c4060, 0xc000c04a20)
#011/usr/local/go/src/net/http/transport.go:999 +0x6e
created by net/http.(*Transport).getConn
#011/usr/local/go/src/net/http/transport.go:998 +0x3d7
[project name].service: Main process exited, code=exited, status=2/INVALIDARGUMENT
[project name].service: Unit entered failed state.
[project name].service: Failed with result 'exit-code'.```

dnsCachingDialContext func:

func dnsCachingDialContext(r *dnscache.Resolver, timeout time.Duration) func(ctx context.Context, network, addr string) (net.Conn, error) {
	return func(ctx context.Context, network string, addr string) (conn net.Conn, err error) {
		separator := strings.LastIndex(addr, ":")
		ips, err := r.LookupHost(ctx, addr[:separator])
		if err != nil {
			return nil, err
		}
		for _, ip := range ips {
			conn, err = net.DialTimeout(network, ip+addr[separator:], timeout)
			if err == nil {
				break
			}
		}
		return
	}
}

DialContext function

It would be great to add the code snippet from docs into library itself and cover with a test:

    DialContext: func(ctx context.Context, network string, addr string) (conn net.Conn, err error) {
        host, port, err := net.SplitHostPort(addr)
        if err != nil {
            return nil, err
        }
        ips, err := r.LookupHost(ctx, host)
        if err != nil {
            return nil, err
        }
        for _, ip := range ips {
            var dialer net.Dialer
            conn, err = dialer.DialContext(ctx, network, net.JoinHostPort(ip, port))
            if err == nil {
                break
            }
        }
        return
    },

So users may just use it direcly instead of copying from docs.

Docs on LookupHost don't match usage

This example from Readme does not work:

resolver := &dnscache.Resolver{}

// First call will cache the result
addrs, err := resolver.LookupHost("example.com")

This does (taken from tests, not sure if that's the correct context for me):

resolver := &dnscache.Resolver{}
addrs, err := resolver.LookupHost(context.Background(), "example.com")

It would be great, we'd have correct docs. I can submit the PR, but not sure about what context it should be.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.