GithubHelp home page GithubHelp logo

overture's Introduction

overture

Build status GoDoc Go Report Card codecov

Overture is a customized DNS relay server.

Overture means the orchestral piece at the beginning of a classical music composition, just like DNS which is nearly the first step of surfing the Internet.

Please note:

  • Read the entire README first is necessary if you want to use overture safely or create an issue for this project .
  • Production usage is not recommended and there is no guarantee or warranty of it.

Features

  • Multiple DNS upstream
    • Via UDP/TCP with custom port
    • Via SOCKS5 proxy (TCP only)
    • With EDNS Client Subnet (ECS) RFC7871
  • Dispatcher
    • Custom domain
    • Custom IP network
    • IPv6 record (AAAA) redirection
  • Full IPv6 support
  • Minimum TTL modification
  • Hosts (Both IPv4 and IPv6 are supported and IPs will be returned in a random order. If you want to use regex match hosts, please understand how regex works first)
  • Cache with ECS and Redis(Persistence) support
  • DNS over HTTP server support

Dispatch process

DNS queries with certain domain will be forced to use selected DNS when matched.

For the IP network dispatch, overture will send queries to primary DNS first. Then, If that answer is empty or not matched, the alternative DNS servers will be used instead.

Installation

The binary releases are available in releases.

Usages

Start with the default config file ./config.yml

Only file having a .json suffix will be considered as json format for compatibility and that support is deprecated.

$ ./overture

Or use your own config file:

$ ./overture -c /path/to/config.yml

Verbose mode:

$ ./overture -v

Log to file:

$ ./overture -l /path/to/overture.log

For other options, please check the helping menu:

$ ./overture -h

Tips:

  • Root privilege might be required if you want to let overture listen on port 53 or one of other system ports.

Configuration Syntax

Configuration file is "config.yml" by default:

bindAddress: :53
debugHTTPAddress: 127.0.0.1:5555
dohEnabled: false
primaryDNS:
  - name: DNSPod
    address: 119.29.29.29:53
    protocol: udp
    socks5Address:
    timeout: 6
    ednsClientSubnet:
      policy: disable
      externalIP:
      noCookie: true
alternativeDNS:
  - name: 114DNS
    address: 114.114.114.114:53
    protocol: udp
    socks5Address:
    timeout: 6
    ednsClientSubnet:
      policy: disable
      externalIP:
      noCookie: true
onlyPrimaryDNS: false
ipv6UseAlternativeDNS: false
alternativeDNSConcurrent: false
whenPrimaryDNSAnswerNoneUse: primaryDNS
ipNetworkFile:
  primary: ./ip_network_primary_sample
  alternative: ./ip_network_alternative_sample
domainFile:
  primary: ./domain_primary_sample
  alternative: ./domain_alternative_sample
  matcher: full-map
hostsFile:
  hostsFile: ./hosts_sample
  finder: full-map
minimumTTL: 0
domainTTLFile: ./domain_ttl_sample
cacheSize: 0
cacheRedisUrl: redis://localhost:6379/0
cacheRedisConnectionPoolSize: 10 
rejectQType:
  - 255

Tips:

  • bindAddress: Specifying any port (e.g. :53) will let overture listen on all available addresses (both IPv4 and IPv6). Overture will handle both TCP and UDP requests. Literal IPv6 addresses are enclosed in square brackets (e.g. [2001:4860:4860::8888]:53)

  • debugHTTPAddress: Specifying an HTTP port for debug (5555 is the default port despite it is also acknowledged as the android Wi-Fi adb listener port), currently used to dump DNS cache, and the request url is /cache, available query argument is nobody(boolean)

    • true(default): only get the cache size;

      $ curl 127.0.0.1:5555/cache | jq
      {
        "length": 1,
        "capacity": 100,
        "body": {}
      }
    • false: get cache size along with cache detail.

      $ curl 127.0.0.1:5555/cache?nobody=false | jq
      {
        "length": 1,
        "capacity": 100,
        "body": {
          "www.baidu.com. 1": [
            {
              "name": "www.baidu.com.",
              "ttl": 1140,
              "type": "CNAME",
              "rdata": "www.a.shifen.com."
            },
            {
              "name": "www.a.shifen.com.",
              "ttl": 300,
              "type": "CNAME",
              "rdata": "www.wshifen.com."
            },
            {
              "name": "www.wshifen.com.",
              "ttl": 300,
              "type": "A",
              "rdata": "104.193.88.123"
            },
            {
              "name": "www.wshifen.com.",
              "ttl": 300,
              "type": "A",
              "rdata": "104.193.88.77"
            }
          ]
        }
      }
  • dohEnabled: Enable DNS over HTTP server using DebugHTTPAddress above with url path /dns-query. DNS over HTTPS server can be easily achieved helping by another web server software like caddy or nginx.

  • primaryDNS/alternativeDNS:

    • name: This field is only used for logging.
    • address: Same rule as BindAddress.
    • protocol: tcp, udp, tcp-tls or https
    • socks5Address: Forward dns query to this SOCKS5 proxy, “” to disable.
    • ednsClientSubnet: Use this to improve DNS accuracy for many reasons. Please check RFC7871 for details.
      • policy
        • auto: If the client IP is not in the reserved IP network, use the client IP. Otherwise, use the external IP.
        • manual: Use the external IP if this field is not empty, otherwise use the client IP if it is not one of the reserved IPs.
        • disable: Disable this feature.
      • externalIP: If this field is empty, ECS will be disabled when the inbound IP is not an external IP.
      • noCookie: Disable cookie.
  • onlyPrimaryDNS: Disable dispatcher feature, use primary DNS only.

  • ipv6UseAlternativeDNS: For to redirect IPv6 DNS queries to alternative DNS servers.

  • alternativeDNSConcurrent: Query the primaryDNS and alternativeDNS at the same time.

  • whenPrimaryDNSAnswerNoneUse: If the response of primaryDNS exists and there is no ANSWER SECTION in it, the final chosen DNS upstream should be defined here. (There is no AAAA record for most domains right now)

  • *File: Both relative like ./file or absolute path like /path/to/file are supported. Especially, for Windows users, please use properly escaped path like C:\\path\\to\\file.txt in the configuration.

  • domainFile.Matcher: Matching policy and implementation, including "full-list", "full-map", "regex-list", "mix-list", "suffix-tree" and "final". Default value is "full-map".

  • hostsFile.Finder: Finder policy and implementation, including "full-map", "regex-list". Default value is "full-map".

  • domainTTLFile: Regex match only for now;

  • minimumTTL: Set the minimum TTL value (in seconds) in order to improve caching efficiency, use 0 to disable.

  • cacheSize: The number of query record to cache, use 0 to disable.

  • cacheRedisUrl, cacheRedisConnectionPoolSize: Use redis cache instead of local cache.

  • rejectQType: Reject query with specific DNS record types, check List of DNS record types for details.

Domain file example (full match)

example.com

Domain file example (regex match)

^xxx.xx

IP network file example (CIDR match)

1.0.1.0/24
::1/128

Domain TTL file example (regex match)

 example.com$ 100

Hosts file example (full match)

127.0.0.1 localhost
::1 localhost

Hosts file example (regex match)

10.8.0.1 example.com$

DNS servers with ECS support

  • DNSPod 119.29.29.29:53

For DNSPod, ECS might only work via udp, you can test it by patched dig to certify this argument by comparing answers.

The accuracy depends on the server side.

$ dig @119.29.29.29 www.qq.com +client=119.29.29.29

; <<>> DiG 9.9.3 <<>> @119.29.29.29 www.qq.com +client=119.29.29.29
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64995
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; CLIENT-SUBNET: 119.29.29.29/32/24
;; QUESTION SECTION:
;www.qq.com.            IN  A

;; ANSWER SECTION:
www.qq.com.     300 IN  A   101.226.103.106

;; Query time: 52 msec
;; SERVER: 119.29.29.29#53(119.29.29.29)
;; WHEN: Wed Mar 08 18:00:52 CST 2017
;; MSG SIZE  rcvd: 67
$ dig @119.29.29.29 www.qq.com +client=119.29.29.29 +tcp

; <<>> DiG 9.9.3 <<>> @119.29.29.29 www.qq.com +client=119.29.29.29 +tcp
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58331
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.qq.com.            IN  A

;; ANSWER SECTION:
www.qq.com.     43  IN  A   59.37.96.63
www.qq.com.     43  IN  A   14.17.32.211
www.qq.com.     43  IN  A   14.17.42.40

;; Query time: 81 msec
;; SERVER: 119.29.29.29#53(119.29.29.29)
;; WHEN: Wed Mar 08 18:01:32 CST 2017
;; MSG SIZE  rcvd: 87

Acknowledgements

License

This project is under the MIT license. See the LICENSE file for the full license text.

overture's People

Contributors

aiamadeus avatar comzyh avatar f-td5x avatar godla avatar hexchain avatar jemyzhang avatar jmvoid avatar jsvisa avatar lexuge avatar li3p avatar love4taylor avatar maddie avatar nyamisty avatar pcmid avatar qyb avatar rampagex avatar sgralpha avatar sh1r0 avatar sharermax avatar shawn1m avatar simonsmh avatar tuzi3040 avatar v-e-o avatar wen-long avatar wzv5 avatar zhanhb avatar zhmin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

overture's Issues

Domestic DNS resolution timeout with EDNSClientSubnetPolicy set to 'auto' and DNSPod as primary server

When I have primary DNS server set as DNSPod (119.29.29.29:53) and set EDNSClientSubnetPolicy to auto, I'm getting numerous error saying: (not the exact line but the same meaning):

DNS result returned nil, maybe timeout

... which results in extremely slow DNS resolution for almost all websites (including the domestic ones like baidu.com). As stated in README, DNSPod should support EDNSClientSubnetPolicy.

Setting EDNSClientSubnetPolicy to disable solves this problem.

主辅转发的逻辑问题

PrimaryDNS中有两个上游DNS,一个是企鹅家的,一个是本地DNS,现在需求是本地DNS服务器(信息量挺繁杂的,换用overture支持的hosts挺麻烦的)解析不了的使用DNSPOD进行智能解析。那么问题来了,因为本地DNS返回结果快于企鹅家的,所以本地解析不了的域名被直接分给了AlternativeDNS去解析,出于大家都懂得的原因,AlternativeDNS还是用你给的配置比较好,然而又不能利用openDNS实现智能解析,处境十分尴尬。所以,希望当PrimaryDNS返回结果为空的时候能够等待其他PrimaryDNS的响应(当然是在Timeout时间内),把解析权优先给PrimaryDNS,而不是PrimaryDNS中的某台服务器第一个返回了空值程序就直接把解析权给了AlternativeDNS,应该先让PrimaryDNS尝试解析,大家都解析不了的再让AlternativeDNS尝试解析。希望能优化一下,非常感谢!

解析某些域名得到空记录

解析 scontent-hkg3-1.cdninstagram.com 时候 A 记录空了。

DEBU[0026] Question: ;scontent-hkg3-1.cdninstagram.com. IN       A
DEBU[0026] Domain match fail, try to use primary DNS.
DEBU[0027] Finally use primary DNS.

附上 JSON 配置

{
  "BindAddress": ":5335",
  "PrimaryDNSAddress": "202.96.128.166:53",
  "PrimaryDNSMethod": "udp",
  "AlternativeDNSAddress": "203.80.96.10:53",
  "AlternativeDNSMethod": "udp",
  "Timeout": 3,
  "RedirectIPv6Record": true,
  "IPNetworkFilePath": "china_ip_list.txt",
  "DomainFilePath": "blacklist.txt",
  "DomainBase64Decode": false
}

已经开启了 TCP 和 UDP 转发,通过 dig 命令是没问题的。

➜  ~ dig @203.80.96.10 scontent-hkg3-1.cdninstagram.com

; <<>> DiG 9.8.3-P1 <<>> @203.80.96.10 scontent-hkg3-1.cdninstagram.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56916
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 4

;; QUESTION SECTION:
;scontent-hkg3-1.cdninstagram.com. IN	A

;; ANSWER SECTION:
scontent-hkg3-1.cdninstagram.com. 48 IN	A	31.13.95.48

;; AUTHORITY SECTION:
cdninstagram.com.	69291	IN	NS	a.ns.igcdn.com.
cdninstagram.com.	69291	IN	NS	b.ns.igcdn.com.

;; ADDITIONAL SECTION:
a.ns.igcdn.com.		638	IN	A	69.171.239.11
a.ns.igcdn.com.		638	IN	AAAA	2a03:2880:fffe:b:face:b00c::99
b.ns.igcdn.com.		638	IN	A	69.171.255.11
b.ns.igcdn.com.		638	IN	AAAA	2a03:2880:ffff:b:face:b00c::99

;; Query time: 12 msec
;; SERVER: 203.80.96.10#53(203.80.96.10)
;; WHEN: Sat Jan 14 12:12:23 2017
;; MSG SIZE  rcvd: 195

版本是 Latest release 的 1.1.0 版。

几个问题。

1:Domain 匹配失败的,又去匹配IP network,这样导致一些ip是非cn的亚洲域名解析明显变慢,如何单独关闭IP network匹配。
2,pc->dnsmasq->overtrue
哪怕"OnlyPrimaryDNS": true时,很多国内域名都解析很慢,换阿里dns和腾讯dns都这样。求解。
chrome里很多正在解析主机

关于hosts通配符的建议和求助

首先我发现*.a.com好像不包括a.com?于是我就得写两行。然后发现当网址是c.b.a.com的时候,*.a.com也不能将其包括在内。不知道有没能把a.com结尾的全部指到一个IP的方法?目前我在搞FQ用的DNS,google家的东西域名非常多,如果能把google.com结尾的都指向一个IP(SNI IP)能节省大量空间并且更方便。还有google.xx这的域名,*放在后面似乎是没用的啊。如果已能实现,麻烦赐教。

请求添加"默认服务器"字段

$ nslookup - 119.29.29.29
默认服务器: pdns.dnspod.cn
Address: 119.29.29.29

$ nslookup - 223.5.5.5
默认服务器: public1.alidns.com
Address: 223.5.5.5
使用nslookup命令时,一般的公网DNS都会返回'默认服务器'字段

$ nslookup - 192.168.10.251(overture的地址)
默认服务器: UnKnown
Address: 192.168.10.251
使用overture服务器时,此字段返回UnKnown
是否可以自己定义?

Goroutine null deference when ExchangeFromRemote

INFO[0000] Overture 1.3.5.1-3-gc2cc31f
INFO[0000] If you need any help, please visit the project repository: https://github.com/shawn1m/overture
INFO[0000] Load IP network file successful
INFO[0000] Load domain file successful
INFO[0000] Minimum TTL is disabled
INFO[0000] CacheSize is 10000
INFO[0000] Load hosts file successful
INFO[0000] Start overture on :53
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x596e20]

goroutine 320330 [running]:
panic(0x664360, 0xc4200140a0)
/usr/lib/go-1.7/src/runtime/panic.go:500 +0x1a1
github.com/shawn1m/overture/core/common.GetEDNSClientSubnetIP(0xc42059d6c8, 0xc420014e74, 0xb)
/root/go/src/github.com/shawn1m/overture/core/common/edns.go:63 +0xa0
github.com/shawn1m/overture/core/outbound.(*Client).ExchangeFromRemote(0xc42059d6c0, 0xc4203b0100)
/root/go/src/github.com/shawn1m/overture/core/outbound/client.go:53 +0x7a
github.com/shawn1m/overture/core/outbound.(*ClientBundle).ExchangeFromRemote.func1(0xc4200a8401, 0xc42059d6c0, 0xc4200a8420)
/root/go/src/github.com/shawn1m/overture/core/outbound/clientbundle.go:46 +0x39
created by github.com/shawn1m/overture/core/outbound.(*ClientBundle).ExchangeFromRemote
/root/go/src/github.com/shawn1m/overture/core/outbound/clientbundle.go:48 +0xa8

Maybe another use after free issue, that QuestionMessage was destroyed before used in running exchange goroutines.
Add reference count with this object?

三重错误

1.3.5-rc4版本
运行方式:c>b>a>dnspod (c为客户端,b为第2台overture a为第一台overture )
运行后不久,a报错:
level=warning msg="Dial DNS upstream failed: dial udp 119.29.29.29:53: socket: too many open files"
level=warning msg="DNSPod Fail: Maybe this server does not support EDNS Client Subnet"

b报错:
level=warning msg="DNSPod Fail: Send question message failed"

dig @b主机 imgcache.qq.com无响应。

EvictRandom

EvictRandom这个实现是不是目前没必要
应该只有InsertMessage可以写Cache entry,一次只能增加一条
然后call EvictRandom

clen := len(c.table)
	if clen < c.capacity {
		return
	}
	i := c.capacity - clen
	for k := range c.table {
		delete(c.table, k)
		i--
		if i == 0 {
			break
		}
	}

这个语义应该是把超出capacity的部分删掉,但上下文有读写锁,所以只能是没超或者超了一条

1.35.1有时解析很慢,dig返回mismatch的简单分析。

对于1.35.1和1.35-rc5
在不缓存的情况下,分别用dig 请求解析twitter.com和v2ex.com的日志。

1.35.1
time="2017-07-11T15:24:28+08:00" level=debug msg="Question: ;v2ex.com. IN A"
time="2017-07-11T15:24:28+08:00" level=debug msg="Matched: Custom domain v2ex.com v2ex.com"
time="2017-07-11T15:24:28+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1799 IN A 23.251.125.131"
time="2017-07-11T15:24:28+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1799 IN A 23.251.124.131"
time="2017-07-11T15:24:28+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1799 IN A 23.251.126.133"
time="2017-07-11T15:24:33+08:00" level=debug msg="Question: ;v2ex.com. IN A"
time="2017-07-11T15:24:33+08:00" level=debug msg="Matched: Custom domain v2ex.com v2ex.com"
time="2017-07-11T15:24:33+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1799 IN A 23.251.125.131"
time="2017-07-11T15:24:33+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1799 IN A 23.251.124.131"
time="2017-07-11T15:24:33+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1799 IN A 23.251.126.133"
time="2017-07-11T15:24:33+08:00" level=debug msg="Cached: v2ex.com. 1 "
我只dig一次返回,而日志里面却向上游服务器发送了两次请求,dig这边显示:
Warning: ID mismatch: expected ID 34457, got 39936
Warning: ID mismatch: expected ID 34457, got 6752
过了10秒钟后才返回ip。

1.35-rc5
time="2017-07-11T15:18:28+08:00" level=debug msg="Question: ;v2ex.com. IN A"
time="2017-07-11T15:18:28+08:00" level=debug msg="Matched: Custom domain v2ex.com v2ex.com"
time="2017-07-11T15:18:28+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1413 IN A 23.251.124.131"
time="2017-07-11T15:18:28+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1413 IN A 23.251.126.133"
time="2017-07-11T15:18:28+08:00" level=debug msg="DNSTunnel Answer: v2ex.com. 1413 IN A 23.251.125.131"
time="2017-07-11T15:18:28+08:00" level=debug msg="Cached: v2ex.com. 1 "
这种也是dig一次,日志也是请求一次,返回正常,dig迅速返回ip。

1.35.1
time="2017-07-11T15:18:38+08:00" level=debug msg="Question: ;twitter.com. IN A"
time="2017-07-11T15:18:38+08:00" level=debug msg="Matched: Custom domain twitter.com twitter.com"
time="2017-07-11T15:18:38+08:00" level=debug msg="DNSTunnel Answer: twitter.com. 1513 IN A 104.244.42.1"
time="2017-07-11T15:18:38+08:00" level=debug msg="DNSTunnel Answer: twitter.com. 1513 IN A 104.244.42.193"
time="2017-07-11T15:18:38+08:00" level=debug msg="Cached: twitter.com. 1 "
正常,dig迅速返回ip。

1.35-rc5
time="2017-07-11T15:24:22+08:00" level=debug msg="Question: ;twitter.com. IN A"
time="2017-07-11T15:24:22+08:00" level=debug msg="Matched: Custom domain twitter.com twitter.com"
time="2017-07-11T15:24:22+08:00" level=debug msg="DNSTunnel Answer: twitter.com. 1450 IN A 104.244.42.129"
time="2017-07-11T15:24:22+08:00" level=debug msg="DNSTunnel Answer: twitter.com. 1450 IN A 104.244.42.1"
time="2017-07-11T15:24:22+08:00" level=debug msg="Cached: twitter.com. 1 "
正常,dig迅速返回ip。

总结就是,rc5版只会发送一次请求,等待上游服务器返回请求值后再返回给下游。
而1.35.1不知道基于什么判断会有时(并不是所有情况下都这样)连续发送两次请求给上游服务器,而且两次都有返回,dig这边就观察到了这种warning

新版本多个 `PrimaryDNS` 时 DNS 策略不生效

config.json

{
    "BindAddress":":5335",
    "PrimaryDNS":[
        {
            "Name":"ChinaTelecom 1",
            "Address":"202.96.128.166:53",
            "Protocol":"udp",
            "Timeout":6,
            "EDNSClientSubnet":{
                "Policy":"disable",
                "ExternalIP":""
            }
        },
        {
            "Name":"ChinaTelecom 2",
            "Address":"202.96.134.133:53",
            "Protocol":"udp",
            "Timeout":6,
            "EDNSClientSubnet":{
                "Policy":"disable",
                "ExternalIP":""
            }
        }
    ],
    "AlternativeDNS":[
        {
            "Name":"PCCW HK",
            "Address":"203.80.96.10:53",
            "Protocol":"tcp",
            "Timeout":6,
            "EDNSClientSubnet":{
                "Policy":"disable",
                "ExternalIP":""
            }
        }
    ],
    "RedirectIPv6Record":true,
    "IPNetworkFile":"china_ip_list.txt",
    "DomainFile":"gfwlist.txt",
    "DomainBase64Decode":false,
    "HostsFile":"hosts",
    "MinimumTTL":0,
    "CacheSize":0
}
dig @127.0.0.1 -p5335 www.facebook.com

; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p5335 www.facebook.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42124
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.facebook.com.		IN	A

;; ANSWER SECTION:
www.facebook.com.	2896	IN	A	93.46.8.89

;; Query time: 14 msec
;; SERVER: 127.0.0.1#5335(127.0.0.1)
;; WHEN: Fri Feb 17 17:31:33 2017
;; MSG SIZE  rcvd: 66

DEBUG INFO

DEBU[0001] Question: ;www.facebook.com.	IN	 A
DEBU[0001] Domain match fail, try to use primary DNS
DEBU[0001] ChinaTelecom 1 Answer: www.facebook.com.	737	IN	A	93.46.8.89
DEBU[0001] Try to match response ip address with IP network
DEBU[0001] IP network match fail, finally use alternative DNS
DEBU[0001] ChinaTelecom 2 Answer: www.facebook.com.	2896	IN	A	93.46.8.89
DEBU[0001] ChinaTelecom 2 Answer: www.facebook.com.	2896	IN	A	93.46.8.89
DEBU[0001] PCCW HK Answer: www.facebook.com.	2856	IN	CNAME	star-mini.c10r.facebook.com.
DEBU[0001] PCCW HK Answer: star-mini.c10r.facebook.com.	54	IN	A	31.13.95.36

串联使用overture带不上客户端ip

之前在v2ex上提到过的问题:
c 是客户机, a 主机安装 overture , b 主机也安装 overture ,然后 b 收到c的查询请求递归到 a , a 再递归到 8.8.8.8或者119.29.29.29

此前您回复:如果开启 edns ,生效的应该都是 c 的公网出口 ip ;不开启的话,生效的是末端 a

今日测试发现,a和b都开启了edns,但客户端c接收到的解析结果是末端a主机ip递归到8.8.8.8或119.29.29.29的结果,在b向a的传递过程中,似乎将c客户端的ip丢失了。

Verbose mode 下有部分字符乱码

非常好的软件!

终端显示汉字没有问题。

在配置文件中开启缓存,启动后查询dns,吐出信息如下:

DEBU[0005] XXX Answer: xxxx.xxx.xxx.      3600    IN      A       xxx.xxx.xxx.xxx
DEBU[0005] Try to match response ip address with IP network
DEBU[0005] IP network match fail, finally use alternative DNS
2U��`����] Cache    xxx.xxx.xxx.�9��^kK

Listen udp failed: listen udp :1096: errno -9

I just download mipsel 1.3.5.1 prebuilt version, and use provided config.json in that zip.

Only changed listening port from 53 to 1096, run it with "./overture-linux-mipsle -c config.json -v".

Device is ASUS router AC66U.

请问一下这个怎么和shadowsocks配和使用呢,ss安卓内置overture吗?电脑端怎么实现呢

请问一下这个怎么和shadowsocks配和使用呢,我看本地和服务端都有人部署,,有人能说一下使用方法没谢谢

听说overture的cdn解析特牛逼

补充一下我的想法:
user访问请求——>overture dns查询(或路由缓存 )——>ip(国外)——>shadowsocks
............................................ ......................................................................... ip(国内)——>直连

运行问题

./overture 运行起来了 关闭了ssh连接 overture也就没了 要怎么在后台运行?

不能正常解析域名

gfwlist里的域名v4、v6解析正常,不在列表里的不能正常解析,报Get dns response failed: Response message is nil, maybe timeout 错误。
配置文件:
{
"BindAddress": ":5353",
"PrimaryDNSAddress": "119.29.29.29:53",
"PrimaryDNSProtocol": "udp",
"AlternativeDNSAddress": "127.0.0.1:1053",
"AlternativeDNSProtocol": "tcp",
"Timeout": 6,
"RedirectIPv6Record": true,
"IPNetworkFilePath": "/tmp/china_ip_list.txt",
"DomainFilePath": "/tmp/gfwlist.txt",
"DomainBase64Decode": true,
"MinimumTTL": 6000,
"EDNSClientSubnetPolicy": "auto",
"EDNSClientSubnetIP": ""
}。
还有,能不能设置多个上游服务器?哪个先返回用哪个。

不建议*全包括

之前由于通配符存在三级以上域名绕过hosts的问题,新版修正后将*.a.com中,.a之前的所有字符全包括了,由此就产生了一个问题。

当hosts写为*.ss0.bdstatic.com时,发现会把所有包含ss0字符的bdstatic.com域名全部指向到127.0.0.1,例如gss0.bdstatic.com

ss0.bdstatic.com是百度的广告域名,我要将其指向到127.0.0.1屏蔽掉,但gss0.bdstatic.com是百度知道的css一旦屏蔽导致百度知道无法正常访问。而这里面有不少是无需屏蔽的域名,此种方式误伤太大。

建议将*.ss0.bdstatic.com仅包括所有属于ss0.bdstatic.com这个域的子域,例如abc.ss0.bdstatic.com
而不扩展到包含ss0这个关键字的bdstatic.com的其他子域,例如abcss0.bdstatic.com

谢谢!

有关AAAA记录转发5S超时的bug

hi 你好
我在使用你的ot之后遇到了如下的问题;想咨询您一下;细节如下:

oveture(192.168.1.1)配置主DNS是内网的一台主DNS服务器(192.168.1.2);overture的备NDS 也是内网的一台备DNS(192.168.1.3);客户端DNS指向overture(192.168.1.1)之后;A记录解析没问题;AAAA记录5s超时; 客户端直接将DNS指向内网的主或备DNS;没有此问题。

下面是同过overture之后超时的现象
image

下面是直接指向内网DNS的现象
image

image


curl 默认会查询AAAA记录;curl 通过overture之后;全部出现5s超时的问题;求解答~~

配置文件如下

{
"BindAddress": ":53",
"PrimaryDNS": [
{
"Name": "DNSPod",
"Address": "192.168.1.2:53",
"Protocol": "udp",
"SOCKS5Address": "",
"Timeout": 6,
"EDNSClientSubnet": {
"Policy": "disable",
"ExternalIP": ""
}
}
],
"AlternativeDNS": [
{
"Name": "OpenDNS",
"Address": "192.168.1.3:53",
"Protocol": "udp",
"SOCKS5Address": "",
"Timeout": 2,
"EDNSClientSubnet": {
"Policy": "disable",
"ExternalIP": ""
}
}
],
"OnlyPrimaryDNS": false,
"RedirectIPv6Record": true,
"IPNetworkFile": "./ip_network_sample",
"DomainFile": "./domain_sample",
"DomainBase64Decode": true,
"HostsFile": "./hosts_sample",
"MinimumTTL": 0,
"CacheSize" : 0,
"RejectQtype": [255]
}


日志如下

`time="2017-07-31T15:41:49+08:00" level=info msg="Overture 1.3.5.2"
time="2017-07-31T15:41:49+08:00" level=info msg="If you need any help, please visit the project repository: https://github.com/shawn1m/overture"
time="2017-07-31T15:41:49+08:00" level=info msg="Load IP network file successful"
time="2017-07-31T15:41:49+08:00" level=info msg="Load domain file successful"
time="2017-07-31T15:41:49+08:00" level=info msg="Minimum TTL is disabled"
time="2017-07-31T15:41:49+08:00" level=info msg="Cache is disabled"
time="2017-07-31T15:41:49+08:00" level=debug msg="Load hosts took 287ns"
time="2017-07-31T15:41:49+08:00" level=info msg="Load hosts file successful"
time="2017-07-31T15:41:49+08:00" level=info msg="Start overture on :53"

time="2017-07-31T15:53:51+08:00" level=debug msg="Question: ;thirdapi.xxxxx.com.\tIN\t A"
time="2017-07-31T15:53:51+08:00" level=debug msg="Domain match fail, try to use primary DNS"
time="2017-07-31T15:53:51+08:00" level=debug msg="Question: ;thirdapi.xxxxx.com.\tIN\t AAAA"
time="2017-07-31T15:53:51+08:00" level=debug msg="Finally use alternative DNS"
time="2017-07-31T15:53:51+08:00" level=debug msg="DNSPod Answer: thirdapi.xxxxx.com.\t86400\tIN\tA\t10.199.1.11"
time="2017-07-31T15:53:51+08:00" level=debug msg="Primary DNS answer is empty, finally use alternative DNS"
time="2017-07-31T15:53:56+08:00" level=debug msg="Question: ;thirdapi.xxxxx.com.\tIN\t A"
time="2017-07-31T15:53:56+08:00" level=debug msg="Domain match fail, try to use primary DNS"
time="2017-07-31T15:53:56+08:00" level=debug msg="DNSPod Answer: thirdapi.xxxxx.com.\t86400\tIN\tA\t10.199.1.11"
time="2017-07-31T15:53:56+08:00" level=debug msg="Try to match response ip address with IP network"
time="2017-07-31T15:53:56+08:00" level=debug msg="Matched: IP network 10.199.1.11 0.0.0.0/0"
time="2017-07-31T15:53:56+08:00" level=debug msg="Finally use primary DNS"
time="2017-07-31T15:53:56+08:00" level=debug msg="Question: ;thirdapi.xxxxx.com.\tIN\t AAAA"
time="2017-07-31T15:53:56+08:00" level=debug msg="Finally use alternative DNS"`


Note: 由于dig和nslookup只会解析A记录;所以非常快;没有遇到此问题。但是CURL默认会解析4A记录;造成IPV6转发超时的现象。

优化 hosts 文件加载速度

首先感谢作者,终于有了一个方便配置的可以替换 pdnsd 和 dnsmasq 组合的工具。不过使用过程中发现一个问题,我用于 adblock 等用途的 hosts 文件在 5 万行左右,这个时候在 rMBP 15-inch, Mid 2014 (2.2 GHz Intel Core i7)环境下启动 overture 需要将近一分钟;在 高通骁龙 820/821 环境下启动需要 3 到 4 分钟。不知这个速度是否还有优化的空间呢?谢谢!

use udp protocol with socks5 proxy failed

Error:

WARN[0002] Dial DNS upstream with SOCKS5 proxy failed: proxy: no support for SOCKS5 proxy connections of type udp.

But I confirmed that the socks5 proxy is enabled with udp connections.

Configuration:

  "AlternativeDNS": [
    {
      "Name": "GoogleDNS",
      "Address": "8.8.8.8:53",
      "Protocol": "udp",
      "SOCKS5Address": "127.0.0.1:1080",
      "Timeout": 6,
      "EDNSClientSubnet": {
        "Policy": "auto",
        "ExternalIP": "xxx.xxx.xxx.xxx"
      }
    }
  ],

使用overture无法打开steam程序

最近steam主程序总是无法打开,无法加载主页,显示网络错误。
从overture更换为别的dns(如114或dnspod)后就正常了。
不知道是什么原因。
用overture dig steam的地址返回的是正常值

上级 DNS 不支持 ipv6 地址

某些运营商已向用户提供 ipv6 地址,测试发现网络质量相对较好,打算将 ipv6 的 google dns 添加过来

    {
      "Name": "google_6",
      "Address": "2001:4860:4860::8888",
      "Protocol": "tcp",
      "SOCKS5Address": "",
      "Timeout": 6,
      "EDNSClientSubnet": {
        "Policy": "disable",
        "ExternalIP": ""
      }
    }

运行时报错

WARN[0006] Dial DNS upstream failed: dial tcp: address 2001:4860:4860::8888:53: too many colons in address

不知是否有计划支持

hostfile seems not work

hostfile config

127.0.0.1 localhost
192.168.31.182	xunlei_com

overture in debug mode

DEBU[0149] Question: ;xunlei_com.	IN	 NS                
DEBU[0149] Domain match fail, try to use primary DNS    
DEBU[0149] Primary DNS answer is empty, finally use alternative DNS

nslook up result

nslookup -q=ns xunlei_com
Server:192.168.31.182
Address:192.168.31.182#53
** server can`t find xunlei_com:NXDOMAIN

i used this version
https://github.com/shawn1m/overture/releases/download/1.3.1/overture-linux-amd64.zip

warning msg="Maybe DNSPod Fail: does not support edns client subnet and it need to be replaced"

运行后过了几小时日志里出现错误:
time="2017-05-04T10:38:45+08:00" level=info msg="Start overture on :53"
time="2017-05-04T12:20:45+08:00" level=warning msg="Maybe DNSPod Fail: does not support edns client subnet and it need to be replaced"
time="2017-05-04T15:23:42+08:00" level=warning msg="Maybe DNSPod Fail: does not support edns client subnet and it need to be replaced"

然后又过了两小时,overture进程无故自己退出了,内存并没有占用完。

1.3.5-rc4 ARM 版本运行一会就会退出

软件版本:
下载的 1.3.5-rc4 或者在路由器上本地编译的版本:
go get -v -u github.com/shawn1m/overture/main

运行环境:
ASUS RT-AC68U / Tomato
Buffalo WZR-1750DHP / DD-WRT

问题:
运行一会程序就会退出,换回 rc3 则无问题

调试信息:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1e6100]

goroutine 139 [running]:
github.com/shawn1m/overture/core/cache.(*Cache).InsertMessage(0x1076bd00, 0x10889e60, 0x15, 0x0)
	/mnt/sda2/compile/go/src/github.com/shawn1m/overture/core/cache/cache.go:74 +0x34
github.com/shawn1m/overture/core/outbound.(*ClientBundle).CacheResults(0x10898090)
	/mnt/sda2/compile/go/src/github.com/shawn1m/overture/core/outbound/clientbundle.go:97 +0xa4
github.com/shawn1m/overture/core/outbound.(*Dispatcher).Exchange(0x1064e680)
	/mnt/sda2/compile/go/src/github.com/shawn1m/overture/core/outbound/dispatcher.go:42 +0xac
github.com/shawn1m/overture/core/inbound.(*Server).ServeDNS(0x10658540, 0x333950, 0x1087ef00, 0x1088a780)
	/mnt/sda2/compile/go/src/github.com/shawn1m/overture/core/inbound/server.go:65 +0x1c4
github.com/miekg/dns.(*ServeMux).ServeDNS(0x10650538, 0x333950, 0x1087ef00, 0x1088a780)
	/mnt/sda2/compile/go/src/github.com/miekg/dns/server.go:210 +0x4c
github.com/miekg/dns.(*Server).serve(0x10652360, 0x331798, 0x10889340, 0x3309b0, 0x10650538, 0x10890000, 0x21, 0x200, 0x10650598, 0x10845da0, ...)
	/mnt/sda2/compile/go/src/github.com/miekg/dns/server.go:579 +0x240
created by github.com/miekg/dns.(*Server).serveUDP
	/mnt/sda2/compile/go/src/github.com/miekg/dns/server.go:533 +0x218

1.3.4hosts文件有空行的话运行不了

windows版,386和amd64都这样。

如上面↑↑↑这样空行,就无法启动会一闪而过。
另外还有一个请求
请求就是之前的issue提到的,希望出一个以**.a.com包含全部a.com结尾域名(包括x.a.com;x.x.a.com;x.x.x.a.com......)的通配符。

Domain/IP/Hosts file支持多个文件

目前这3个file在配置文件中都只是一个文件。希望能将它们的配置项扩展为数组型的,可列出多个文件,合并内容。

比如domain file,默认的文件很大,我不想去修改,因为升级时可以直接覆盖。但是又想增加自己的,现有的配置就没办法了。如果可以支持多个文件并且自动合并内容,就可以开一个自己的domaon file,加入自己想额外加入的就行了。

base64 decode feature change

With commit aa57485, it seems that the base64 decode feature (for the base64 encoded file like gfwlist if I understand correctly) has been moved from runtime to compile time. Currently in my setup, I have a scheduled job that fetches the latest gfwlist daily and restart overture. Does that mean I will have to do the base64 decoding myself from now on?

If yes, it would be nice to take this to people's attention since they might have the same setup as mine.

"no such file or directory"

I've set these three variable properly "IPNetworkFilePath", "DomainFilePath", "HostsFile" in config file. But neither relative path nor absolute path is working.

ERRO[0000] Open IP network file failed: open : no such file or directory
ERRO[0000] Open Custom domain file failed: open : no such file or directory
...
INFO[0000] Load hosts file failed: 0xc420074f90

1.3.5-rc2运行时报错

time="2017-04-26T13:45:41+08:00" level=info msg="Overture 1.3.5-rc2"
time="2017-04-26T13:45:41+08:00" level=info msg="If you need any help, please visit the project repository: https://github.com/shawn1m/overture"
time="2017-04-26T13:45:41+08:00" level=fatal msg="Json syntex error: invalid character ':' after top-level value"
请问是什么原因?

可否考虑降低cpu与带宽占用?

目前使用1.3.5-rc2版,与bind9.10做了简单对比发现cpu占用比bind高50%左右,带宽占用高出bind一倍。
dig返回的MSG SIZE包大小,比bind平均大2倍左右。
OnlyPrimaryDNS已设置为true
请问目前还有没有办法进一步降低cpu与带宽消耗呢?谢谢。

一些建议

访问控制:IP白名单和IP黑名单,拒绝解析指定的域名(之前我的DNS一直有好多IP不停地查询一个.gov域名,宽带都被吃的死死的);
让hosts支持单域名多IP(原谅我这点还没做测试就提建议了);
支持循环复用或者随机排序;
Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.