spiderclub / haipproxy

:sparkling_heart: High available distributed ip proxy pool, powerd by Scrapy and Redis

高可用IP代理池

README　｜　中文文档

本项目所采集的IP资源都来自互联网，愿景是为大型爬虫项目提供一个高可用低延迟的高匿IP代理池。

项目亮点

• 代理来源丰富
• 代理抓取提取精准
• 代理校验严格合理
• 监控完备，鲁棒性强
• 架构灵活，便于扩展
• 各个组件分布式部署

快速开始

注意，代码请在release列表中下载，master分支的代码不保证能稳定运行

单机部署

服务端

• 安装Python3和Redis。有问题可以阅读这篇文章的相关部分。

• 根据Redis的实际配置修改项目配置文件config/settings.py中的REDIS_HOST、REDIS_PASSWORD等参数。

• 安装scrapy-splash，并修改配置文件config/settings.py中的SPLASH_URL

• 安装项目相关依赖

  pip install -r requirements.txt

• 启动scrapy worker，包括代理IP采集器和校验器

  python crawler_booter.py --usage crawler

  python crawler_booter.py --usage validator

• 启动调度器，包括代理IP定时调度和校验

  python scheduler_booter.py --usage crawler

  python scheduler_booter.py --usage validator

客户端

近日不断有同学问，如何获取该项目中可用的代理IP列表。haipproxy提供代理的方式并不是通过api api来提供，而是通过具体的客户端来提供。 目前支持的是Python客户端和语言无关的squid二级代理

python客户端调用示例

from client.py_cli import ProxyFetcher
args = dict(host='127.0.0.1', port=6379, password='123456', db=0)
＃　这里`zhihu`的意思是，去和`zhihu`相关的代理ip校验队列中获取ip
＃　这么做的原因是同一个代理IP对不同网站代理效果不同
fetcher = ProxyFetcher('zhihu', strategy='greedy', redis_args=args)
# 获取一个可用代理
print(fetcher.get_proxy())
# 获取可用代理列表
print(fetcher.get_proxies()) # or print(fetcher.pool)

更具体的示例见examples/zhihu

squid作为二级代理

• 安装squid，备份squid的配置文件并且启动squid，以ubuntu为例

  sudo apt-get install squid

  sudo sed -i 's/http_access deny all/http_access allow all/g' /etc/squid/squid.conf

  sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.backup

  sudo service squid start

• 根据操作系统修改项目配置文件config/settings.py中的SQUID_BIN_PATH、SQUID_CONF_PATH、SQUID_TEMPLATE_PATH等参数

• 启动squid conf的定时更新程序

  sudo python squid_update.py

• 使用squid作为代理中间层请求目标网站,默认代理URL为'http://squid_host:3128',用Python请求示例如下

  import requests
  proxies = {'https': 'http://127.0.0.1:3128'}
  resp = requests.get('https://httpbin.org/ip', proxies=proxies)
  print(resp.text)

Docker部署

• 安装Docker

• 安装docker-compose

  pip install -U docker-compose

• 修改settings.py中的SPLASH_URL和REDIS_HOST参数

  # 注意，如果您使用master分支下的代码，这步可被省略
  SPLASH_URL = 'http://splash:8050'
  REDIS_HOST = 'redis'

• 使用docker-compose启动各个应用组件

  docker-compose up

这种方式会一同部署squid，您可以通过squid调用代理IP池，也可以使用客户端调用，和单机部署调用方式一样

注意事项

• 本项目高度依赖Redis，除了消息通信和数据存储之外，IP校验和任务定时工具也使用了Redis中的多种数据结构。 如果需要替换Redis，请自行度量
• 由于GFW的原因，某些网站需要通过科学上网才能进行访问和采集，如果用户无法访问墙外的网站，请将rules.py task_queue为 SPIDER_GFW_TASK和SPIDER_AJAX_GFW_TASK的任务enable属性设置为0或者启动爬虫的时候指定爬虫类型为common和 ajax

  python crawler_booter.py --usage crawler common ajax

• 相同代理IP，对于不同网站的代理效果可能大不相同。如果通用代理无法满足您的需求，您可以为特定网站编写代理IP校验器

工作流程

效果测试

以单机模式部署haipproxy和测试代码，以知乎为目标请求站点，实测抓取效果如下

测试代码见examples/zhihu

项目监控(可选)

项目监控主要通过sentry和prometheus,通过在关键地方 进行业务埋点对项目各个维度进行监测，以提高项目的鲁棒性

项目使用Sentry作Bug Trace工具，通过Sentry可以很容易跟踪项目健康情况

使用Prometheus+Grafana做业务监控，了解项目当前状态

捐赠作者

开源不易，如果本项目对您有用，不妨进行小额捐赠，以支持项目的持续维护

同类项目

本项目参考了Github上开源的各个爬虫代理的实现，感谢他们的付出，下面是笔者参考的所有项目，排名不分先后。

dungproxy

proxyspider

ProxyPool

proxy_pool

ProxyPool

IPProxyTool

IPProxyPool

proxy_list

proxy_pool

ProxyPool

scylla

haipproxy's Issues

如何手动更换的代理IP

你好，
请问我用get_proxy()获取到一个代理IP之后，用着用着不能用了（被网站屏蔽了），这时在代码里面用delete_proxy()删掉这个IP，再用get_proxy()获取，还是原来到IP呢？
我用到是docker方式部署。

安装 依赖发生错误

启动

python crawler_booter.py --usage crawler

没有找到模块

[root@squid haipproxy-0.1]# python crawler_booter.py --usage crawler
Traceback (most recent call last):
File "crawler_booter.py", line 19, in
from scheduler import crawler_start
File "/root/haipproxy-0.1/scheduler/init.py", line 5, in
from .scheduler import (
File "/root/haipproxy-0.1/scheduler/scheduler.py", line 7, in
import click
ImportError: No module named 'click'
[root@squid haipproxy-0.1]#

安装都成功，但是一个IP都跑不到；有人碰见么

2018-09-05 10:29:07 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
Read 0 requests from haipproxy:spider:common
2018-09-05 10:29:07 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
Read 0 requests from haipproxy:spider:ajax
Read 0 requests from haipproxy:spider:common
Read 0 requests from haipproxy:spider:ajax
Read 0 requests from haipproxy:spider:common
Read 0 requests from haipproxy:spider:ajax
Read 0 requests from haipproxy:spider:common
Read 0 requests from haipproxy:spider:ajax

make配置redis时报错

之前gcc：command not found,在网上找了教程把gcc装好了，但现在make配置redis的时候，报了这个错误，能帮我看一下吗？谢啦

怎么更换httpbin.py

国内好像链接httpbin.org比较困难，今天用httpbin.org验证的时候就出现了

    <html>
      <head>
            <meta name="viewport" content="width=device-width, initial-scale=1">
            <meta charset="utf-8">
            <title>Application Error</title>
            <style media="screen">
              html,body,iframe {
                    margin: 0;
                    padding: 0;
              }
              html,body {
                    height: 100%;
                    overflow: hidden;
              }
              iframe {
                    width: 100%;
                    height: 100%;
                    border: 0;
              }
            </style>
      </head>
      <body>
            <iframe src="//www.herokucdn.com/error-pages/application-error.html"></iframe>
      </body>
    </html>

导致可用IP数量的下降，应该修改那些配置文件替换 httpbin.org成其他的网址

请问是否支持多线程爬取啊

想请教一下，当我多线程爬取的时候，一直是0 proxies have been fetched，这说明这个系统不支持多线程爬取吗

Generic package naming - perhaps use a top level package

Using generic package names (client, config, crawler, logger, scheduler, utils) is likely to cause confusion and possibly naming conflicts. When importing them, there's nothing that associates them as being part haipproxy, which can be confusing when out of context. An easy way to handle this would be moving them under a top-level haipproxy package, with the following layout:

README.md
...
haipproxy/
├── __init__.py
├── client
│   ├── __init__.py
│   └── ...
├── config
│   └── __init__.py
│...

求scrapy中间件

docker cannot perform

envir: ubuntu 16.04 64

when I run docker-compose up this command, It will stuck in the below:

how I solve this problem

by the way, I already set REDIS_HOST='redis' SPLASH_URL='http://splash:8050"

anyone can run this docker container success?

release中的版本安装项目依赖requirement.txt会报错，能看一下吗

错误代码：

PS D:\code\haipproxy-0.1\haipproxy-0.1> pip install -r requirements.txt
Collecting Scrapy==1.5.0 (from -r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/db/9c/cb15b2dc6003a805afd21b9b396e0e965800765b51da72fe17cf340b9be2/Scrapy-1.5.0-py2.py3-none-any.whl
Collecting requests==2.18.4 (from -r requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl
Collecting redis==2.10.6 (from -r requirements.txt (line 3))
Using cached https://files.pythonhosted.org/packages/3b/f6/7a76333cf0b9251ecf49efff635015171843d9b977e4ffcf59f9c4428052/redis-2.10.6-py2.py3-none-any.whl
Collecting schedule==0.5.0 (from -r requirements.txt (line 4))
Using cached https://files.pythonhosted.org/packages/df/2c/3a94d846682a4fb94966e65bca19a1acb6f7dd85977f4e4cece6e677b757/schedule-0.5.0-py2.py3-none-any.whl
Requirement already satisfied: scrapy-splash==0.7.2 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from -r requirements.txt (line 5)) (0.7.2)
Collecting click==6.7 (from -r requirements.txt (line 6))
Using cached https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl
Requirement already satisfied: service-identity==17.0.0 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from -r requirements.txt (line 7)) (17.0.0)
Requirement already satisfied: parsel>=1.1 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Scrapy==1.5.0->-r requirements.txt (line 1)) (1.5.0)
Requirement already satisfied: six>=1.5.2 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Scrapy==1.5.0->-r requirements.txt (line 1)) (1.11.0)
Requirement already satisfied: w3lib>=1.17.0 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Scrapy==1.5.0->-r requirements.txt (line 1)) (1.19.0)
Requirement already satisfied: lxml in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Scrapy==1.5.0->-r requirements.txt (line 1)) (4.2.4)
Requirement already satisfied: pyOpenSSL in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Scrapy==1.5.0->-r requirements.txt (line 1)) (18.0.0)
Collecting Twisted>=13.1.0 (from Scrapy==1.5.0->-r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2
Requirement already satisfied: cssselect>=0.9 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Scrapy==1.5.0->-r requirements.txt (line 1)) (1.0.3)
Requirement already satisfied: queuelib in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Scrapy==1.5.0->-r requirements.txt (line 1)) (1.5.0)
Requirement already satisfied: PyDispatcher>=2.0.5 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Scrapy==1.5.0->-r requirements.txt (line 1)) (2.0.5)
Collecting chardet<3.1.0,>=3.0.2 (from requests==2.18.4->-r requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests==2.18.4->-r requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/df/f7/04fee6ac349e915b82171f8e23cee63644d83663b34c539f7a09aed18f9e/certifi-2018.8.24-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests==2.18.4->-r requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl
Requirement already satisfied: idna<2.7,>=2.5 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from requests==2.18.4->-r requirements.txt (line 2)) (2.6)
Requirement already satisfied: pyasn1-modules in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from service-identity==17.0.0->-r requirements.txt (line 7)) (0.2.2)
Requirement already satisfied: pyasn1 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from service-identity==17.0.0->-r requirements.txt (line 7)) (0.4.4)
Requirement already satisfied: attrs in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from service-identity==17.0.0->-r requirements.txt (line 7)) (18.2.0)
Requirement already satisfied: cryptography>=2.2.1 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from pyOpenSSL->Scrapy==1.5.0->-r requirements.txt (line 1)) (2.3.1)
Requirement already satisfied: zope.interface>=4.4.2 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.5.0->-r requirements.txt (line 1)) (4.5.0)
Requirement already satisfied: constantly>=15.1 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.5.0->-r requirements.txt (line 1)) (15.1.0)
Requirement already satisfied: incremental>=16.10.1 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.5.0->-r requirements.txt (line 1)) (17.5.0)
Requirement already satisfied: Automat>=0.3.0 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.5.0->-r requirements.txt (line 1)) (0.7.0)
Requirement already satisfied: hyperlink>=17.1.1 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.5.0->-r requirements.txt (line 1)) (18.0.0)
Requirement already satisfied: PyHamcrest>=1.9.0 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from Twisted>=13.1.0->Scrapy==1.5.0->-r requirements.txt (line 1)) (1.9.0)
Requirement already satisfied: cffi!=1.11.3,>=1.7 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from cryptography>=2.2.1->pyOpenSSL->Scrapy==1.5.0->-r requirements.txt (line 1)) (1.11.5)
Requirement already satisfied: asn1crypto>=0.21.0 in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from cryptography>=2.2.1->pyOpenSSL->Scrapy==1.5.0->-r requirements.txt (line 1)) (0.24.0)
Requirement already satisfied: setuptools in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from zope.interface>=4.4.2->Twisted>=13.1.0->Scrapy==1.5.0->-r requirements.txt (line 1)) (39.0.1)
Requirement already satisfied: pycparser in c:\users\wujia\appdata\local\programs\python\python37\lib\site-packages (from cffi!=1.11.3,>=1.7->cryptography>=2.2.1->pyOpenSSL->Scrapy==1.5.0->-r requirements.txt (line 1)) (2.18)
Installing collected packages: Twisted, Scrapy, chardet, certifi, urllib3, requests, redis, schedule, click
Running setup.py install for Twisted ... error
Complete output from command c:\users\wujia\appdata\local\programs\python\python37\python.exe -u -c "import setuptools, tokenize;file='C:\Users\wujia\AppData\Local\Temp\pip-install-ot59h2zr\Twisted\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\wujia\AppData\Local\Temp\pip-record-9qtzrywr\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.7
creating build\lib.win-amd64-3.7\twisted
copying src\twisted\copyright.py -> build\lib.win-amd64-3.7\twisted
copying src\twisted\plugin.py -> build\lib.win-amd64-3.7\twisted
copying src\twisted_version.py -> build\lib.win-amd64-3.7\twisted
copying src\twisted_init_.py -> build\lib.win-amd64-3.7\twisted
copying src\twisted_main_.py -> build\lib.win-amd64-3.7\twisted
creating build\lib.win-amd64-3.7\twisted\application
copying src\twisted\application\app.py -> build\lib.win-amd64-3.7\twisted\application
copying src\twisted\application\internet.py -> build\lib.win-amd64-3.7\twisted\application
copying src\twisted\application\reactors.py -> build\lib.win-amd64-3.7\twisted\application
copying src\twisted\application\service.py -> build\lib.win-amd64-3.7\twisted\application
copying src\twisted\application\strports.py -> build\lib.win-amd64-3.7\twisted\application
copying src\twisted\application_init_.py -> build\lib.win-amd64-3.7\twisted\application
creating build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\avatar.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\checkers.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\endpoints.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\error.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\interfaces.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\ls.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\manhole.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\manhole_ssh.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\manhole_tap.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\mixin.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\recvline.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\stdio.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\tap.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\telnet.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\ttymodes.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch\unix.py -> build\lib.win-amd64-3.7\twisted\conch
copying src\twisted\conch_init_.py -> build\lib.win-amd64-3.7\twisted\conch
creating build\lib.win-amd64-3.7\twisted\cred
copying src\twisted\cred\checkers.py -> build\lib.win-amd64-3.7\twisted\cred
copying src\twisted\cred\credentials.py -> build\lib.win-amd64-3.7\twisted\cred
copying src\twisted\cred\error.py -> build\lib.win-amd64-3.7\twisted\cred
copying src\twisted\cred\portal.py -> build\lib.win-amd64-3.7\twisted\cred
copying src\twisted\cred\strcred.py -> build\lib.win-amd64-3.7\twisted\cred
copying src\twisted\cred_digest.py -> build\lib.win-amd64-3.7\twisted\cred
copying src\twisted\cred_init_.py -> build\lib.win-amd64-3.7\twisted\cred
creating build\lib.win-amd64-3.7\twisted\enterprise
copying src\twisted\enterprise\adbapi.py -> build\lib.win-amd64-3.7\twisted\enterprise
copying src\twisted\enterprise_init_.py -> build\lib.win-amd64-3.7\twisted\enterprise
creating build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\abstract.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\address.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\asyncioreactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\base.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\cfreactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\default.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\defer.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\endpoints.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\epollreactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\error.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\fdesc.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\gireactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\glib2reactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\gtk2reactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\gtk3reactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\inotify.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\interfaces.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\kqreactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\main.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\pollreactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\posixbase.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\process.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\protocol.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\pyuisupport.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\reactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\selectreactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\serialport.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\ssl.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\stdio.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\task.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\tcp.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\threads.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\tksupport.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\udp.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\unix.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\utils.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\win32eventreactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\wxreactor.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet\wxsupport.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_baseprocess.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_dumbwin32proc.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_glibbase.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_idna.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_newtls.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_pollingfile.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_posixserialport.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_posixstdio.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_producer_helpers.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_resolver.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_signals.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_sslverify.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_threadedselect.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_win32serialport.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_win32stdio.py -> build\lib.win-amd64-3.7\twisted\internet
copying src\twisted\internet_init_.py -> build\lib.win-amd64-3.7\twisted\internet
creating build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_buffer.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_file.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_filter.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_flatten.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_format.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_global.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_io.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_json.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_legacy.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_levels.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_logger.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_observer.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_stdlib.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_util.py -> build\lib.win-amd64-3.7\twisted\logger
copying src\twisted\logger_init_.py -> build\lib.win-amd64-3.7\twisted\logger
creating build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail\imap4.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail\interfaces.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail\pop3.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail\pop3client.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail\protocols.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail\relay.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail\smtp.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail_cred.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail_except.py -> build\lib.win-amd64-3.7\twisted\mail
copying src\twisted\mail_init_.py -> build\lib.win-amd64-3.7\twisted\mail
creating build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\authority.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\cache.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\client.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\common.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\dns.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\error.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\hosts.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\resolve.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\root.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\secondary.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\server.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\srvconnect.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names\tap.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names_rfc1982.py -> build\lib.win-amd64-3.7\twisted\names
copying src\twisted\names_init_.py -> build\lib.win-amd64-3.7\twisted\names
creating build\lib.win-amd64-3.7\twisted\pair
copying src\twisted\pair\ethernet.py -> build\lib.win-amd64-3.7\twisted\pair
copying src\twisted\pair\ip.py -> build\lib.win-amd64-3.7\twisted\pair
copying src\twisted\pair\raw.py -> build\lib.win-amd64-3.7\twisted\pair
copying src\twisted\pair\rawudp.py -> build\lib.win-amd64-3.7\twisted\pair
copying src\twisted\pair\testing.py -> build\lib.win-amd64-3.7\twisted\pair
copying src\twisted\pair\tuntap.py -> build\lib.win-amd64-3.7\twisted\pair
copying src\twisted\pair_init_.py -> build\lib.win-amd64-3.7\twisted\pair
creating build\lib.win-amd64-3.7\twisted\persisted
copying src\twisted\persisted\aot.py -> build\lib.win-amd64-3.7\twisted\persisted
copying src\twisted\persisted\crefutil.py -> build\lib.win-amd64-3.7\twisted\persisted
copying src\twisted\persisted\dirdbm.py -> build\lib.win-amd64-3.7\twisted\persisted
copying src\twisted\persisted\sob.py -> build\lib.win-amd64-3.7\twisted\persisted
copying src\twisted\persisted\styles.py -> build\lib.win-amd64-3.7\twisted\persisted
copying src\twisted\persisted_init_.py -> build\lib.win-amd64-3.7\twisted\persisted
creating build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\cred_anonymous.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\cred_file.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\cred_memory.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\cred_sshkeys.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\cred_unix.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_conch.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_core.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_ftp.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_inet.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_names.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_portforward.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_reactors.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_runner.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_socks.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_trial.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_web.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins\twisted_words.py -> build\lib.win-amd64-3.7\twisted\plugins
copying src\twisted\plugins_init_.py -> build\lib.win-amd64-3.7\twisted\plugins
creating build\lib.win-amd64-3.7\twisted\positioning
copying src\twisted\positioning\base.py -> build\lib.win-amd64-3.7\twisted\positioning
copying src\twisted\positioning\ipositioning.py -> build\lib.win-amd64-3.7\twisted\positioning
copying src\twisted\positioning\nmea.py -> build\lib.win-amd64-3.7\twisted\positioning
copying src\twisted\positioning_sentence.py -> build\lib.win-amd64-3.7\twisted\positioning
copying src\twisted\positioning_init_.py -> build\lib.win-amd64-3.7\twisted\positioning
creating build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\amp.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\basic.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\dict.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\finger.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\ftp.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\htb.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\ident.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\loopback.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\memcache.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\pcp.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\policies.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\portforward.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\postfix.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\sip.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\socks.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\stateful.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\tls.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols\wire.py -> build\lib.win-amd64-3.7\twisted\protocols
copying src\twisted\protocols_init_.py -> build\lib.win-amd64-3.7\twisted\protocols
creating build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\compat.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\components.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\constants.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\context.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\deprecate.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\failure.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\fakepwd.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\filepath.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\formmethod.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\htmlizer.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\lockfile.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\log.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\logfile.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\modules.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\monkey.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\procutils.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\randbytes.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\rebuild.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\reflect.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\release.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\roots.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\runtime.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\sendmsg.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\shortcut.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\syslog.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\systemd.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\text.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\threadable.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\threadpool.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\url.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\urlpath.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\usage.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\util.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\versions.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\win32.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\zippath.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python\zipstream.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_appdirs.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_inotify.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_oldstyle.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_release.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_setup.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_shellcomp.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_textattributes.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_tzhelper.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_url.py -> build\lib.win-amd64-3.7\twisted\python
copying src\twisted\python_init_.py -> build\lib.win-amd64-3.7\twisted\python
creating build\lib.win-amd64-3.7\twisted\runner
copying src\twisted\runner\inetd.py -> build\lib.win-amd64-3.7\twisted\runner
copying src\twisted\runner\inetdconf.py -> build\lib.win-amd64-3.7\twisted\runner
copying src\twisted\runner\inetdtap.py -> build\lib.win-amd64-3.7\twisted\runner
copying src\twisted\runner\procmon.py -> build\lib.win-amd64-3.7\twisted\runner
copying src\twisted\runner\procmontap.py -> build\lib.win-amd64-3.7\twisted\runner
copying src\twisted\runner_init_.py -> build\lib.win-amd64-3.7\twisted\runner
creating build\lib.win-amd64-3.7\twisted\scripts
copying src\twisted\scripts\htmlizer.py -> build\lib.win-amd64-3.7\twisted\scripts
copying src\twisted\scripts\trial.py -> build\lib.win-amd64-3.7\twisted\scripts
copying src\twisted\scripts\twistd.py -> build\lib.win-amd64-3.7\twisted\scripts
copying src\twisted\scripts_twistd_unix.py -> build\lib.win-amd64-3.7\twisted\scripts
copying src\twisted\scripts_twistw.py -> build\lib.win-amd64-3.7\twisted\scripts
copying src\twisted\scripts_init_.py -> build\lib.win-amd64-3.7\twisted\scripts
creating build\lib.win-amd64-3.7\twisted\spread
copying src\twisted\spread\banana.py -> build\lib.win-amd64-3.7\twisted\spread
copying src\twisted\spread\flavors.py -> build\lib.win-amd64-3.7\twisted\spread
copying src\twisted\spread\interfaces.py -> build\lib.win-amd64-3.7\twisted\spread
copying src\twisted\spread\jelly.py -> build\lib.win-amd64-3.7\twisted\spread
copying src\twisted\spread\pb.py -> build\lib.win-amd64-3.7\twisted\spread
copying src\twisted\spread\publish.py -> build\lib.win-amd64-3.7\twisted\spread
copying src\twisted\spread\util.py -> build\lib.win-amd64-3.7\twisted\spread
copying src\twisted\spread_init_.py -> build\lib.win-amd64-3.7\twisted\spread
creating build\lib.win-amd64-3.7\twisted\tap
copying src\twisted\tap\ftp.py -> build\lib.win-amd64-3.7\twisted\tap
copying src\twisted\tap\portforward.py -> build\lib.win-amd64-3.7\twisted\tap
copying src\twisted\tap\socks.py -> build\lib.win-amd64-3.7\twisted\tap
copying src\twisted\tap_init_.py -> build\lib.win-amd64-3.7\twisted\tap
creating build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\crash_test_dummy.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\iosim.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\mock_win32process.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\myrebuilder1.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\myrebuilder2.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\plugin_basic.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\plugin_extra1.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\plugin_extra2.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_cmdline.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_echoer.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_fds.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_getargv.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_getenv.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_linger.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_reader.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_signal.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_stdinreader.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_tester.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_tty.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\process_twisted.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\proto_helpers.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\reflect_helper_IE.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\reflect_helper_VE.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\reflect_helper_ZDE.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\ssl_helpers.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\stdio_test_consumer.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\stdio_test_halfclose.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\stdio_test_hostpeer.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\stdio_test_lastwrite.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\stdio_test_loseconn.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\stdio_test_producer.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\stdio_test_write.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\stdio_test_writeseq.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\testutils.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_abstract.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_adbapi.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_amp.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_application.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_compat.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_context.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_cooperator.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_defer.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_defgen.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_dict.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_dirdbm.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_error.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_factories.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_failure.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_fdesc.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_finger.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_formmethod.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_ftp.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_ftp_options.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_htb.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_ident.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_internet.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_iosim.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_iutils.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_lockfile.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_log.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_logfile.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_loopback.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_main.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_memcache.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_modules.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_monkey.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_nooldstyle.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_paths.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_pcp.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_persisted.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_plugin.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_policies.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_postfix.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_process.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_protocols.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_randbytes.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_rebuild.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_reflect.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_roots.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_shortcut.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_sip.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_sob.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_socks.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_ssl.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_sslverify.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_stateful.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_stdio.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_strerror.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_stringtransport.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_strports.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_task.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_tcp.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_tcp_internals.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_text.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_threadable.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_threadpool.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_threads.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_tpfile.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_twistd.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_twisted.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_udp.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_unix.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\test_usage.py -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test_init_.py -> build\lib.win-amd64-3.7\twisted\test
creating build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial\itrial.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial\reporter.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial\runner.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial\unittest.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial\util.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial_asyncrunner.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial_asynctest.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial_synctest.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial_init_.py -> build\lib.win-amd64-3.7\twisted\trial
copying src\twisted\trial_main_.py -> build\lib.win-amd64-3.7\twisted\trial
creating build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\client.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\demo.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\distrib.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\domhelpers.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\error.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\guard.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\html.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\http.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\http_headers.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\iweb.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\microdom.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\proxy.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\resource.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\rewrite.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\script.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\server.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\static.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\sux.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\tap.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\template.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\twcgi.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\util.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\vhost.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\wsgi.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web\xmlrpc.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web_element.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web_flatten.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web_http2.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web_newclient.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web_responses.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web_stan.py -> build\lib.win-amd64-3.7\twisted\web
copying src\twisted\web_init_.py -> build\lib.win-amd64-3.7\twisted\web
creating build\lib.win-amd64-3.7\twisted\words
copying src\twisted\words\ewords.py -> build\lib.win-amd64-3.7\twisted\words
copying src\twisted\words\iwords.py -> build\lib.win-amd64-3.7\twisted\words
copying src\twisted\words\service.py -> build\lib.win-amd64-3.7\twisted\words
copying src\twisted\words\tap.py -> build\lib.win-amd64-3.7\twisted\words
copying src\twisted\words\xmpproutertap.py -> build\lib.win-amd64-3.7\twisted\words
copying src\twisted\words_init_.py -> build\lib.win-amd64-3.7\twisted\words
creating build\lib.win-amd64-3.7\twisted_threads
copying src\twisted_threads_convenience.py -> build\lib.win-amd64-3.7\twisted_threads
copying src\twisted_threads_ithreads.py -> build\lib.win-amd64-3.7\twisted_threads
copying src\twisted_threads_memory.py -> build\lib.win-amd64-3.7\twisted_threads
copying src\twisted_threads_pool.py -> build\lib.win-amd64-3.7\twisted_threads
copying src\twisted_threads_team.py -> build\lib.win-amd64-3.7\twisted_threads
copying src\twisted_threads_threadworker.py -> build\lib.win-amd64-3.7\twisted_threads
copying src\twisted_threads_init_.py -> build\lib.win-amd64-3.7\twisted_threads
creating build\lib.win-amd64-3.7\twisted\application\runner
copying src\twisted\application\runner_exit.py -> build\lib.win-amd64-3.7\twisted\application\runner
copying src\twisted\application\runner_pidfile.py -> build\lib.win-amd64-3.7\twisted\application\runner
copying src\twisted\application\runner_runner.py -> build\lib.win-amd64-3.7\twisted\application\runner
copying src\twisted\application\runner_init_.py -> build\lib.win-amd64-3.7\twisted\application\runner
creating build\lib.win-amd64-3.7\twisted\application\test
copying src\twisted\application\test\test_internet.py -> build\lib.win-amd64-3.7\twisted\application\test
copying src\twisted\application\test\test_service.py -> build\lib.win-amd64-3.7\twisted\application\test
copying src\twisted\application\test_init_.py -> build\lib.win-amd64-3.7\twisted\application\test
creating build\lib.win-amd64-3.7\twisted\application\twist
copying src\twisted\application\twist_options.py -> build\lib.win-amd64-3.7\twisted\application\twist
copying src\twisted\application\twist_twist.py -> build\lib.win-amd64-3.7\twisted\application\twist
copying src\twisted\application\twist_init_.py -> build\lib.win-amd64-3.7\twisted\application\twist
creating build\lib.win-amd64-3.7\twisted\application\runner\test
copying src\twisted\application\runner\test\test_exit.py -> build\lib.win-amd64-3.7\twisted\application\runner\test
copying src\twisted\application\runner\test\test_pidfile.py -> build\lib.win-amd64-3.7\twisted\application\runner\test
copying src\twisted\application\runner\test\test_runner.py -> build\lib.win-amd64-3.7\twisted\application\runner\test
copying src\twisted\application\runner\test_init_.py -> build\lib.win-amd64-3.7\twisted\application\runner\test
creating build\lib.win-amd64-3.7\twisted\application\twist\test
copying src\twisted\application\twist\test\test_options.py -> build\lib.win-amd64-3.7\twisted\application\twist\test
copying src\twisted\application\twist\test\test_twist.py -> build\lib.win-amd64-3.7\twisted\application\twist\test
copying src\twisted\application\twist\test_init_.py -> build\lib.win-amd64-3.7\twisted\application\twist\test
creating build\lib.win-amd64-3.7\twisted\conch\client
copying src\twisted\conch\client\agent.py -> build\lib.win-amd64-3.7\twisted\conch\client
copying src\twisted\conch\client\connect.py -> build\lib.win-amd64-3.7\twisted\conch\client
copying src\twisted\conch\client\default.py -> build\lib.win-amd64-3.7\twisted\conch\client
copying src\twisted\conch\client\direct.py -> build\lib.win-amd64-3.7\twisted\conch\client
copying src\twisted\conch\client\knownhosts.py -> build\lib.win-amd64-3.7\twisted\conch\client
copying src\twisted\conch\client\options.py -> build\lib.win-amd64-3.7\twisted\conch\client
copying src\twisted\conch\client_init_.py -> build\lib.win-amd64-3.7\twisted\conch\client
creating build\lib.win-amd64-3.7\twisted\conch\insults
copying src\twisted\conch\insults\client.py -> build\lib.win-amd64-3.7\twisted\conch\insults
copying src\twisted\conch\insults\colors.py -> build\lib.win-amd64-3.7\twisted\conch\insults
copying src\twisted\conch\insults\helper.py -> build\lib.win-amd64-3.7\twisted\conch\insults
copying src\twisted\conch\insults\insults.py -> build\lib.win-amd64-3.7\twisted\conch\insults
copying src\twisted\conch\insults\text.py -> build\lib.win-amd64-3.7\twisted\conch\insults
copying src\twisted\conch\insults\window.py -> build\lib.win-amd64-3.7\twisted\conch\insults
copying src\twisted\conch\insults_init_.py -> build\lib.win-amd64-3.7\twisted\conch\insults
creating build\lib.win-amd64-3.7\twisted\conch\openssh_compat
copying src\twisted\conch\openssh_compat\factory.py -> build\lib.win-amd64-3.7\twisted\conch\openssh_compat
copying src\twisted\conch\openssh_compat\primes.py -> build\lib.win-amd64-3.7\twisted\conch\openssh_compat
copying src\twisted\conch\openssh_compat_init_.py -> build\lib.win-amd64-3.7\twisted\conch\openssh_compat
creating build\lib.win-amd64-3.7\twisted\conch\scripts
copying src\twisted\conch\scripts\cftp.py -> build\lib.win-amd64-3.7\twisted\conch\scripts
copying src\twisted\conch\scripts\ckeygen.py -> build\lib.win-amd64-3.7\twisted\conch\scripts
copying src\twisted\conch\scripts\conch.py -> build\lib.win-amd64-3.7\twisted\conch\scripts
copying src\twisted\conch\scripts\tkconch.py -> build\lib.win-amd64-3.7\twisted\conch\scripts
copying src\twisted\conch\scripts_init_.py -> build\lib.win-amd64-3.7\twisted\conch\scripts
creating build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\address.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\agent.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\channel.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\common.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\connection.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\factory.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\filetransfer.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\forwarding.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\keys.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\service.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\session.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\sexpy.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\transport.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh\userauth.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh_kex.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
copying src\twisted\conch\ssh_init_.py -> build\lib.win-amd64-3.7\twisted\conch\ssh
creating build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\keydata.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\loopback.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_address.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_agent.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_cftp.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_channel.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_checkers.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_ckeygen.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_conch.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_connection.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_default.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_endpoints.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_filetransfer.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_forwarding.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_helper.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_insults.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_keys.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_knownhosts.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_manhole.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_manhole_tap.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_mixin.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_openssh_compat.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_recvline.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_scripts.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_session.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_ssh.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_tap.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_telnet.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_text.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_transport.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_unix.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_userauth.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test\test_window.py -> build\lib.win-amd64-3.7\twisted\conch\test
copying src\twisted\conch\test_init_.py -> build\lib.win-amd64-3.7\twisted\conch\test
creating build\lib.win-amd64-3.7\twisted\conch\ui
copying src\twisted\conch\ui\ansi.py -> build\lib.win-amd64-3.7\twisted\conch\ui
copying src\twisted\conch\ui\tkvt100.py -> build\lib.win-amd64-3.7\twisted\conch\ui
copying src\twisted\conch\ui_init_.py -> build\lib.win-amd64-3.7\twisted\conch\ui
creating build\lib.win-amd64-3.7\twisted\cred\test
copying src\twisted\cred\test\test_cramauth.py -> build\lib.win-amd64-3.7\twisted\cred\test
copying src\twisted\cred\test\test_cred.py -> build\lib.win-amd64-3.7\twisted\cred\test
copying src\twisted\cred\test\test_digestauth.py -> build\lib.win-amd64-3.7\twisted\cred\test
copying src\twisted\cred\test\test_simpleauth.py -> build\lib.win-amd64-3.7\twisted\cred\test
copying src\twisted\cred\test\test_strcred.py -> build\lib.win-amd64-3.7\twisted\cred\test
copying src\twisted\cred\test_init_.py -> build\lib.win-amd64-3.7\twisted\cred\test
creating build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\iocpreactor\abstract.py -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\iocpreactor\const.py -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\iocpreactor\interfaces.py -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\iocpreactor\reactor.py -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\iocpreactor\setup.py -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\iocpreactor\tcp.py -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\iocpreactor\udp.py -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\iocpreactor_init_.py -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
creating build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\connectionmixins.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\fakeendpoint.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\modulehelpers.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\process_cli.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\process_connectionlost.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\process_gireactornocompat.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\process_helper.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\reactormixins.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_abstract.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_address.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_asyncioreactor.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_base.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_baseprocess.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_core.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_coroutines.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_default.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_endpoints.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_epollreactor.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_fdset.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_filedescriptor.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_gireactor.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_glibbase.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_inlinecb.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_inotify.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_iocp.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_kqueuereactor.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_main.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_newtls.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_pollingfile.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_posixbase.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_posixprocess.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_process.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_protocol.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_resolver.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_serialport.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_sigchld.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_socket.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_stdio.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_tcp.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_threads.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_time.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_tls.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_udp.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_udp_internals.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_unix.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_win32events.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test\test_win32serialport.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test_posixifaces.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test_win32ifaces.py -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test_init_.py -> build\lib.win-amd64-3.7\twisted\internet\test
creating build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_buffer.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_file.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_filter.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_flatten.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_format.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_global.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_io.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_json.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_legacy.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_levels.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_logger.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_observer.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_stdlib.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test\test_util.py -> build\lib.win-amd64-3.7\twisted\logger\test
copying src\twisted\logger\test_init_.py -> build\lib.win-amd64-3.7\twisted\logger\test
creating build\lib.win-amd64-3.7\twisted\mail\scripts
copying src\twisted\mail\scripts\mailmail.py -> build\lib.win-amd64-3.7\twisted\mail\scripts
creating build\lib.win-amd64-3.7\twisted\mail\test
copying src\twisted\mail\test\pop3testserver.py -> build\lib.win-amd64-3.7\twisted\mail\test
copying src\twisted\mail\test\test_imap.py -> build\lib.win-amd64-3.7\twisted\mail\test
copying src\twisted\mail\test\test_mailmail.py -> build\lib.win-amd64-3.7\twisted\mail\test
copying src\twisted\mail\test\test_pop3.py -> build\lib.win-amd64-3.7\twisted\mail\test
copying src\twisted\mail\test\test_pop3client.py -> build\lib.win-amd64-3.7\twisted\mail\test
copying src\twisted\mail\test\test_smtp.py -> build\lib.win-amd64-3.7\twisted\mail\test
copying src\twisted\mail\test_init_.py -> build\lib.win-amd64-3.7\twisted\mail\test
creating build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_cache.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_client.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_common.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_dns.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_examples.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_hosts.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_names.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_resolve.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_rfc1982.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_rootresolve.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_server.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_srvconnect.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_tap.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test\test_util.py -> build\lib.win-amd64-3.7\twisted\names\test
copying src\twisted\names\test_init_.py -> build\lib.win-amd64-3.7\twisted\names\test
creating build\lib.win-amd64-3.7\twisted\pair\test
copying src\twisted\pair\test\test_ethernet.py -> build\lib.win-amd64-3.7\twisted\pair\test
copying src\twisted\pair\test\test_ip.py -> build\lib.win-amd64-3.7\twisted\pair\test
copying src\twisted\pair\test\test_rawudp.py -> build\lib.win-amd64-3.7\twisted\pair\test
copying src\twisted\pair\test\test_tuntap.py -> build\lib.win-amd64-3.7\twisted\pair\test
copying src\twisted\pair\test_init_.py -> build\lib.win-amd64-3.7\twisted\pair\test
creating build\lib.win-amd64-3.7\twisted\persisted\test
copying src\twisted\persisted\test\test_styles.py -> build\lib.win-amd64-3.7\twisted\persisted\test
copying src\twisted\persisted\test_init_.py -> build\lib.win-amd64-3.7\twisted\persisted\test
creating build\lib.win-amd64-3.7\twisted\positioning\test
copying src\twisted\positioning\test\receiver.py -> build\lib.win-amd64-3.7\twisted\positioning\test
copying src\twisted\positioning\test\test_base.py -> build\lib.win-amd64-3.7\twisted\positioning\test
copying src\twisted\positioning\test\test_nmea.py -> build\lib.win-amd64-3.7\twisted\positioning\test
copying src\twisted\positioning\test\test_sentence.py -> build\lib.win-amd64-3.7\twisted\positioning\test
copying src\twisted\positioning\test_init_.py -> build\lib.win-amd64-3.7\twisted\positioning\test
creating build\lib.win-amd64-3.7\twisted\protocols\haproxy
copying src\twisted\protocols\haproxy_exceptions.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy
copying src\twisted\protocols\haproxy_info.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy
copying src\twisted\protocols\haproxy_interfaces.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy
copying src\twisted\protocols\haproxy_parser.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy
copying src\twisted\protocols\haproxy_v1parser.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy
copying src\twisted\protocols\haproxy_v2parser.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy
copying src\twisted\protocols\haproxy_wrapper.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy
copying src\twisted\protocols\haproxy_init_.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy
creating build\lib.win-amd64-3.7\twisted\protocols\test
copying src\twisted\protocols\test\test_basic.py -> build\lib.win-amd64-3.7\twisted\protocols\test
copying src\twisted\protocols\test\test_tls.py -> build\lib.win-amd64-3.7\twisted\protocols\test
copying src\twisted\protocols\test_init_.py -> build\lib.win-amd64-3.7\twisted\protocols\test
creating build\lib.win-amd64-3.7\twisted\protocols\haproxy\test
copying src\twisted\protocols\haproxy\test\test_parser.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy\test
copying src\twisted\protocols\haproxy\test\test_v1parser.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy\test
copying src\twisted\protocols\haproxy\test\test_v2parser.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy\test
copying src\twisted\protocols\haproxy\test\test_wrapper.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy\test
copying src\twisted\protocols\haproxy\test_init_.py -> build\lib.win-amd64-3.7\twisted\protocols\haproxy\test
creating build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\deprecatedattributes.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\modules_helpers.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\pullpipe.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_appdirs.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_components.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_constants.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_deprecate.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_dist3.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_fakepwd.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_htmlizer.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_inotify.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_release.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_runtime.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_sendmsg.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_setup.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_shellcomp.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_syslog.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_systemd.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_textattributes.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_tzhelper.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_url.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_urlpath.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_util.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_versions.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_zippath.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test\test_zipstream.py -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\python\test_init_.py -> build\lib.win-amd64-3.7\twisted\python\test
creating build\lib.win-amd64-3.7\twisted\runner\test
copying src\twisted\runner\test\test_inetdconf.py -> build\lib.win-amd64-3.7\twisted\runner\test
copying src\twisted\runner\test\test_procmon.py -> build\lib.win-amd64-3.7\twisted\runner\test
copying src\twisted\runner\test\test_procmontap.py -> build\lib.win-amd64-3.7\twisted\runner\test
copying src\twisted\runner\test_init_.py -> build\lib.win-amd64-3.7\twisted\runner\test
creating build\lib.win-amd64-3.7\twisted\scripts\test
copying src\twisted\scripts\test\test_scripts.py -> build\lib.win-amd64-3.7\twisted\scripts\test
copying src\twisted\scripts\test_init_.py -> build\lib.win-amd64-3.7\twisted\scripts\test
creating build\lib.win-amd64-3.7\twisted\spread\test
copying src\twisted\spread\test\test_banana.py -> build\lib.win-amd64-3.7\twisted\spread\test
copying src\twisted\spread\test\test_jelly.py -> build\lib.win-amd64-3.7\twisted\spread\test
copying src\twisted\spread\test\test_pb.py -> build\lib.win-amd64-3.7\twisted\spread\test
copying src\twisted\spread\test\test_pbfailure.py -> build\lib.win-amd64-3.7\twisted\spread\test
copying src\twisted\spread\test_init_.py -> build\lib.win-amd64-3.7\twisted\spread\test
creating build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\detests.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\erroneous.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\mockcustomsuite.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\mockcustomsuite2.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\mockcustomsuite3.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\mockdoctest.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\moduleself.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\moduletest.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\novars.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\ordertests.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\packages.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\sample.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\scripttest.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\skipping.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\suppression.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_assertions.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_asyncassertions.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_deferred.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_doctest.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_keyboard.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_loader.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_log.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_output.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_plugins.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_pyunitcompat.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_reporter.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_runner.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_script.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_suppression.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_testcase.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_tests.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_util.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\test_warning.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test\weird.py -> build\lib.win-amd64-3.7\twisted\trial\test
copying src\twisted\trial\test_init_.py -> build\lib.win-amd64-3.7\twisted\trial\test
creating build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist\distreporter.py -> build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist\disttrial.py -> build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist\managercommands.py -> build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist\options.py -> build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist\worker.py -> build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist\workercommands.py -> build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist\workerreporter.py -> build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist\workertrial.py -> build\lib.win-amd64-3.7\twisted\trial_dist
copying src\twisted\trial_dist_init_.py -> build\lib.win-amd64-3.7\twisted\trial_dist
creating build\lib.win-amd64-3.7\twisted\trial_dist\test
copying src\twisted\trial_dist\test\test_distreporter.py -> build\lib.win-amd64-3.7\twisted\trial_dist\test
copying src\twisted\trial_dist\test\test_disttrial.py -> build\lib.win-amd64-3.7\twisted\trial_dist\test
copying src\twisted\trial_dist\test\test_options.py -> build\lib.win-amd64-3.7\twisted\trial_dist\test
copying src\twisted\trial_dist\test\test_worker.py -> build\lib.win-amd64-3.7\twisted\trial_dist\test
copying src\twisted\trial_dist\test\test_workerreporter.py -> build\lib.win-amd64-3.7\twisted\trial_dist\test
copying src\twisted\trial_dist\test\test_workertrial.py -> build\lib.win-amd64-3.7\twisted\trial_dist\test
copying src\twisted\trial_dist\test_init_.py -> build\lib.win-amd64-3.7\twisted\trial_dist\test
creating build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\requesthelper.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_agent.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_cgi.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_client.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_distrib.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_domhelpers.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_error.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_flatten.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_html.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_http.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_http2.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_httpauth.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_http_headers.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_newclient.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_proxy.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_resource.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_script.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_stan.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_static.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_tap.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_template.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_util.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_vhost.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_web.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_webclient.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_web__responses.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_wsgi.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_xml.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test\test_xmlrpc.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test_util.py -> build\lib.win-amd64-3.7\twisted\web\test
copying src\twisted\web\test_init_.py -> build\lib.win-amd64-3.7\twisted\web\test
creating build\lib.win-amd64-3.7\twisted\web_auth
copying src\twisted\web_auth\basic.py -> build\lib.win-amd64-3.7\twisted\web_auth
copying src\twisted\web_auth\digest.py -> build\lib.win-amd64-3.7\twisted\web_auth
copying src\twisted\web_auth\wrapper.py -> build\lib.win-amd64-3.7\twisted\web_auth
copying src\twisted\web_auth_init_.py -> build\lib.win-amd64-3.7\twisted\web_auth
creating build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\im\baseaccount.py -> build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\im\basechat.py -> build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\im\basesupport.py -> build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\im\interfaces.py -> build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\im\ircsupport.py -> build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\im\locals.py -> build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\im\pbsupport.py -> build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\im_init_.py -> build\lib.win-amd64-3.7\twisted\words\im
creating build\lib.win-amd64-3.7\twisted\words\protocols
copying src\twisted\words\protocols\irc.py -> build\lib.win-amd64-3.7\twisted\words\protocols
copying src\twisted\words\protocols_init_.py -> build\lib.win-amd64-3.7\twisted\words\protocols
creating build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_basechat.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_basesupport.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_domish.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_irc.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_ircsupport.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_irc_service.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabberclient.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabbercomponent.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabbererror.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabberjid.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabberjstrports.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabbersasl.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabbersaslmechanisms.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabberxmlstream.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_jabberxmppstringprep.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_service.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_tap.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_xishutil.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_xmlstream.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_xmpproutertap.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test\test_xpath.py -> build\lib.win-amd64-3.7\twisted\words\test
copying src\twisted\words\test_init_.py -> build\lib.win-amd64-3.7\twisted\words\test
creating build\lib.win-amd64-3.7\twisted\words\xish
copying src\twisted\words\xish\domish.py -> build\lib.win-amd64-3.7\twisted\words\xish
copying src\twisted\words\xish\utility.py -> build\lib.win-amd64-3.7\twisted\words\xish
copying src\twisted\words\xish\xmlstream.py -> build\lib.win-amd64-3.7\twisted\words\xish
copying src\twisted\words\xish\xpath.py -> build\lib.win-amd64-3.7\twisted\words\xish
copying src\twisted\words\xish\xpathparser.py -> build\lib.win-amd64-3.7\twisted\words\xish
copying src\twisted\words\xish_init_.py -> build\lib.win-amd64-3.7\twisted\words\xish
creating build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\client.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\component.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\error.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\ijabber.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\jid.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\jstrports.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\sasl.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\sasl_mechanisms.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\xmlstream.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber\xmpp_stringprep.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
copying src\twisted\words\protocols\jabber_init_.py -> build\lib.win-amd64-3.7\twisted\words\protocols\jabber
creating build\lib.win-amd64-3.7\twisted_threads\test
copying src\twisted_threads\test\test_convenience.py -> build\lib.win-amd64-3.7\twisted_threads\test
copying src\twisted_threads\test\test_memory.py -> build\lib.win-amd64-3.7\twisted_threads\test
copying src\twisted_threads\test\test_team.py -> build\lib.win-amd64-3.7\twisted_threads\test
copying src\twisted_threads\test\test_threadworker.py -> build\lib.win-amd64-3.7\twisted_threads\test
copying src\twisted_threads\test_init_.py -> build\lib.win-amd64-3.7\twisted_threads\test
running egg_info
writing src\Twisted.egg-info\PKG-INFO
writing dependency_links to src\Twisted.egg-info\dependency_links.txt
writing entry points to src\Twisted.egg-info\entry_points.txt
writing requirements to src\Twisted.egg-info\requires.txt
writing top-level names to src\Twisted.egg-info\top_level.txt
reading manifest file 'src\Twisted.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '.misc' found under directory 'src\twisted'
warning: no previously-included files matching '.bugfix' found under directory 'src\twisted'
warning: no previously-included files matching '.doc' found under directory 'src\twisted'
warning: no previously-included files matching '.feature' found under directory 'src\twisted'
warning: no previously-included files matching '.removal' found under directory 'src\twisted'
warning: no previously-included files matching 'NEWS' found under directory 'src\twisted'
warning: no previously-included files matching 'README' found under directory 'src\twisted'
warning: no previously-included files matching 'newsfragments' found under directory 'src\twisted'
warning: no previously-included files found matching 'src\twisted\topfiles\CREDITS'
warning: no previously-included files found matching 'src\twisted\topfiles\ChangeLog.Old'
warning: no previously-included files found matching 'pyproject.toml'
warning: no previously-included files found matching 'codecov.yml'
warning: no previously-included files found matching 'appveyor.yml'
warning: no previously-included files found matching '.circleci'
warning: no previously-included files matching '' found under directory '.circleci'
no previously-included directories found matching 'bin'
no previously-included directories found matching 'admin'
no previously-included directories found matching '.travis'
no previously-included directories found matching '.github'
warning: no previously-included files found matching 'docs\historic\2003'
warning: no previously-included files matching '*' found under directory 'docs\historic\2003'
writing manifest file 'src\Twisted.egg-info\SOURCES.txt'
copying src\twisted\python\twisted-completion.zsh -> build\lib.win-amd64-3.7\twisted\python
creating build\lib.win-amd64-3.7\twisted\python_pydoctortemplates
copying src\twisted\python_pydoctortemplates\common.html -> build\lib.win-amd64-3.7\twisted\python_pydoctortemplates
copying src\twisted\python_pydoctortemplates\index.html -> build\lib.win-amd64-3.7\twisted\python_pydoctortemplates
copying src\twisted\python_pydoctortemplates\summary.html -> build\lib.win-amd64-3.7\twisted\python_pydoctortemplates
copying src\twisted\test\cert.pem.no_trailing_newline -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\key.pem.no_trailing_newline -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\test\server.pem -> build\lib.win-amd64-3.7\twisted\test
copying src\twisted\internet\iocpreactor\notes.txt -> build\lib.win-amd64-3.7\twisted\internet\iocpreactor
copying src\twisted\internet\test_awaittests.py.3only -> build\lib.win-amd64-3.7\twisted\internet\test
copying src\twisted\internet\test_yieldfromtests.py.3only -> build\lib.win-amd64-3.7\twisted\internet\test
creating build\lib.win-amd64-3.7\twisted\internet\test\fake_CAs
copying src\twisted\internet\test\fake_CAs\chain.pem -> build\lib.win-amd64-3.7\twisted\internet\test\fake_CAs
copying src\twisted\internet\test\fake_CAs\not-a-certificate -> build\lib.win-amd64-3.7\twisted\internet\test\fake_CAs
copying src\twisted\internet\test\fake_CAs\thing1.pem -> build\lib.win-amd64-3.7\twisted\internet\test\fake_CAs
copying src\twisted\internet\test\fake_CAs\thing2-duplicate.pem -> build\lib.win-amd64-3.7\twisted\internet\test\fake_CAs
copying src\twisted\internet\test\fake_CAs\thing2.pem -> build\lib.win-amd64-3.7\twisted\internet\test\fake_CAs
copying src\twisted\mail\test\rfc822.message -> build\lib.win-amd64-3.7\twisted\mail\test
copying src\twisted\python\test_deprecatetests.py.3only -> build\lib.win-amd64-3.7\twisted\python\test
copying src\twisted\words\im\instancemessenger.glade -> build\lib.win-amd64-3.7\twisted\words\im
copying src\twisted\words\xish\xpathparser.g -> build\lib.win-amd64-3.7\twisted\words\xish
running build_ext
building 'twisted.test.raiser' extension
creating build\temp.win-amd64-3.7
creating build\temp.win-amd64-3.7\Release
creating build\temp.win-amd64-3.7\Release\src
creating build\temp.win-amd64-3.7\Release\src\twisted
creating build\temp.win-amd64-3.7\Release\src\twisted\test
D:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DWIN32=1 -Ic:\users\wujia\appdata\local\programs\python\python37\include -Ic:\users\wujia\appdata\local\programs\python\python37\include "-ID:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\ATLMFC\include" "-ID:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-ID:\Windows Kits\10\include\10.0.17134.0\ucrt" "-ID:\Windows Kits\10\include\10.0.17134.0\shared" "-ID:\Windows Kits\10\include\10.0.17134.0\um" "-ID:\Windows Kits\10\include\10.0.17134.0\winrt" "-ID:\Windows Kits\10\include\10.0.17134.0\cppwinrt" /Tcsrc/twisted/test/raiser.c /Fobuild\temp.win-amd64-3.7\Release\src/twisted/test/raiser.obj
raiser.c
D:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:c:\users\wujia\appdata\local\programs\python\python37\libs /LIBPATH:c:\users\wujia\appdata\local\programs\python\python37\PCbuild\amd64 "/LIBPATH:D:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\ATLMFC\lib\x64" "/LIBPATH:D:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:D:\Windows Kits\10\lib\10.0.17134.0\ucrt\x64" "/LIBPATH:D:\Windows Kits\10\lib\10.0.17134.0\um\x64" /EXPORT:PyInit_raiser build\temp.win-amd64-3.7\Release\src/twisted/test/raiser.obj /OUT:build\lib.win-amd64-3.7\twisted\test\raiser.cp37-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.7\Release\src/twisted/test\raiser.cp37-win_amd64.lib
Creating library build\temp.win-amd64-3.7\Release\src/twisted/test\raiser.cp37-win_amd64.lib and object build\temp.win-amd64-3.7\Release\src/twisted/test\raiser.cp37-win_amd64.exp
Generating code
Finished generating code
building 'twisted.internet.iocpreactor.iocpsupport' extension
creating build\temp.win-amd64-3.7\Release\src\twisted\internet
creating build\temp.win-amd64-3.7\Release\src\twisted\internet\iocpreactor
creating build\temp.win-amd64-3.7\Release\src\twisted\internet\iocpreactor\iocpsupport
D:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DWIN32=1 -Ic:\users\wujia\appdata\local\programs\python\python37\include -Ic:\users\wujia\appdata\local\programs\python\python37\include "-ID:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\ATLMFC\include" "-ID:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-ID:\Windows Kits\10\include\10.0.17134.0\ucrt" "-ID:\Windows Kits\10\include\10.0.17134.0\shared" "-ID:\Windows Kits\10\include\10.0.17134.0\um" "-ID:\Windows Kits\10\include\10.0.17134.0\winrt" "-ID:\Windows Kits\10\include\10.0.17134.0\cppwinrt" /Tcsrc/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c /Fobuild\temp.win-amd64-3.7\Release\src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.obj
iocpsupport.c
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(1933): warning C4047: '=': '__pyx_t_11iocpsupport_HANDLE' differs in levels of indirection from 'HANDLE'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(2077): warning C4022: 'CreateIoCompletionPort': pointer mismatch for actual parameter 1
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(2077): warning C4022: 'CreateIoCompletionPort': pointer mismatch for actual parameter 2
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(2077): warning C4047: '=': '__pyx_t_11iocpsupport_HANDLE' differs in levels of indirection from 'HANDLE'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(2192): warning C4022: 'GetQueuedCompletionStatus': pointer mismatch for actual parameter 1
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(2537): warning C4022: 'PostQueuedCompletionStatus': pointer mismatch for actual parameter 1
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(2648): warning C4022: 'CloseHandle': pointer mismatch for actual parameter 1
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7194): warning C4020: 'function through pointer': too many actual parameters
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7637): error C2039: 'exc_type': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7638): error C2039: 'exc_value': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7639): error C2039: 'exc_traceback': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7640): error C2039: 'exc_type': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7641): error C2039: 'exc_value': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7642): error C2039: 'exc_traceback': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7664): error C2039: 'exc_type': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7665): error C2039: 'exc_value': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7666): error C2039: 'exc_traceback': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7667): error C2039: 'exc_type': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7668): error C2039: 'exc_value': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7669): error C2039: 'exc_traceback': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7688): error C2039: 'exc_type': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7689): error C2039: 'exc_value': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7690): error C2039: 'exc_traceback': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7697): error C2039: 'exc_type': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7698): error C2039: 'exc_value': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7699): error C2039: 'exc_traceback': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7700): error C2039: 'exc_type': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7701): error C2039: 'exc_value': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
src/twisted/internet/iocpreactor/iocpsupport/iocpsupport.c(7702): error C2039: 'exc_traceback': is not a member of '_ts'
c:\users\wujia\appdata\local\programs\python\python37\include\pystate.h(209): note: see declaration of '_ts'
error: command 'D:\Visual Studio Professional\VisualStudio\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\cl.exe' failed with exit status 2

----------------------------------------

Command "c:\users\wujia\appdata\local\programs\python\python37\python.exe -u -c "import setuptools, tokenize;file='C:\Users\wujia\AppData\Local\Temp\pip-install-ot59h2zr\Twisted\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\wujia\AppData\Local\Temp\pip-record-9qtzrywr\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\wujia\AppData\Local\Temp\pip-install-ot59h2zr\Twisted
PS D:\code\haipproxy-0.1\haipproxy-0.1>

卡在一个地方，获取不到代理ip了

没看到什么明显的错误

redis里面只有这两个队列

微博代理问题

用代理尝试请求微博，有一些代理可以用，有些不行，不是特别稳定。

关于 GFW_PROXY 的困惑

找遍了文档和 Issues 还是不太理解设置里面 GFW_PROXY 这一项，当前是将 haipproxy 运行在境外的服务器上面。

看 rules.py 默认是启用所有的规则，请问那如果不去设置 GFW_PROXY 还能正常爬取 GFW 相关的 rules 吗？因为服务器本身在墙外，也没必要去配置 GFW_PROXY 了。

谢谢！

Docker 部署问题

docker-compose up 的时候 到了14/15 报错 尝试了各种解决方案 各种操作系统 都不行 就这一个问题折腾好几个小时了
以下是报错

Collecting cffi>=1.7 (from cryptography>=2.1.4->pyOpenSSL->Scrapy==1.5.0->-r requirements.txt (line 1))
Downloading https://pypi.doubanio.com/packages/59/cc/0e1635b4951021ef35f5c92b32c865ae605fac2a19d724fb6ff99d745c81/cffi-1.11.5-cp35-cp35m-manylinux1_x86_64.whl (420kB)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python3/dist-packages (from zope.interface>=4.0.2->Twisted>=13.1.0->Scrapy==1.5.0->-r requirements.txt (line 1))
Collecting pycparser (from cffi>=1.7->cryptography>=2.1.4->pyOpenSSL->Scrapy==1.5.0->-r requirements.txt (line 1))
Downloading https://pypi.doubanio.com/packages/8c/2d/aad7f16146f4197a11f8e91fb81df177adcc2073d36a17b1491fd09df6ed/pycparser-2.18.tar.gz (245kB)
Building wheels for collected packages: PyDispatcher, Twisted, cryptography, pycparser
Running setup.py bdist_wheel for PyDispatcher: started
Running setup.py bdist_wheel for PyDispatcher: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/a4/bb/5b/62151ae4ace1811e779c41f59a3a7b1a2243fa9a5611be4a7d
Running setup.py bdist_wheel for Twisted: started
Running setup.py bdist_wheel for Twisted: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/de/88/b1/0662e9a08f542b8943f0e13306a6aaf096643b8741c943edd0
Running setup.py bdist_wheel for cryptography: started
Running setup.py bdist_wheel for cryptography: finished with status 'error'
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-2k1hxp0x/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" bdist_wheel -d /tmp/tmp7cz9qgczpip-wheel- --python-tag cp35:
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
compilation terminated.
Traceback (most recent call last):
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 161, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 147, in call_command
self.run_command(cmdname)
File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 23, in run
self.build()
File "/usr/lib/python3.5/distutils/command/install_lib.py", line 109, in build
self.run_command('build_ext')
File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 49, in run
_build_ext.run(self)
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 338, in run
self.build_extensions()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 447, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 472, in _build_extensions_serial
self.build_extension(ext)
File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 174, in build_extension
_build_ext.build_extension(self, ext)
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 532, in build_extension
depends=ext.depends)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 243, in run_setup
DirectorySandbox(setup_dir).run(runner)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 273, in run
return func()
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 242, in runner
_execfile(setup_script, ns)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-rxlnrgt4/cffi-1.11.5/setup.py", line 240, in
def run_tests(self):
File "/usr/lib/python3.5/distutils/core.py", line 163, in setup
raise SystemExit("error: " + str(msg))
SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1087, in run_setup
run_setup(setup_script, args)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 246, in run_setup
raise
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/lib/python3/dist-packages/pkg_resources/_vendor/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 243, in run_setup
DirectorySandbox(setup_dir).run(runner)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 273, in run
return func()
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 242, in runner
_execfile(setup_script, ns)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-rxlnrgt4/cffi-1.11.5/setup.py", line 240, in
def run_tests(self):
File "/usr/lib/python3.5/distutils/core.py", line 163, in setup
raise SystemExit("error: " + str(msg))
SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-2k1hxp0x/cryptography/setup.py", line 319, in
**keywords_with_side_effects(sys.argv)
File "/usr/lib/python3.5/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 269, in init
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 313, in fetch_build_eggs
replace_conflicting=True,
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 826, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 1092, in best_match
return self.obtain(req, installer)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 1104, in obtain
return installer(requirement)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 380, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 663, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 693, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 873, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1101, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1089, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

---

Failed building wheel for cryptography
Running setup.py clean for cryptography
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-2k1hxp0x/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" clean --all:
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
compilation terminated.
Traceback (most recent call last):
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 161, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 147, in call_command
self.run_command(cmdname)
File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 23, in run
self.build()
File "/usr/lib/python3.5/distutils/command/install_lib.py", line 109, in build
self.run_command('build_ext')
File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 49, in run
_build_ext.run(self)
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 338, in run
self.build_extensions()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 447, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 472, in _build_extensions_serial
self.build_extension(ext)
File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 174, in build_extension
_build_ext.build_extension(self, ext)
File "/usr/lib/python3.5/distutils/command/build_ext.py", line 532, in build_extension
depends=ext.depends)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 243, in run_setup
DirectorySandbox(setup_dir).run(runner)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 273, in run
return func()
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 242, in runner
_execfile(setup_script, ns)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-f142_vde/cffi-1.11.5/setup.py", line 240, in
def run_tests(self):
File "/usr/lib/python3.5/distutils/core.py", line 163, in setup
raise SystemExit("error: " + str(msg))
SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1087, in run_setup
run_setup(setup_script, args)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 246, in run_setup
raise
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/lib/python3/dist-packages/pkg_resources/_vendor/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 243, in run_setup
DirectorySandbox(setup_dir).run(runner)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 273, in run
return func()
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 242, in runner
_execfile(setup_script, ns)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-f142_vde/cffi-1.11.5/setup.py", line 240, in
def run_tests(self):
File "/usr/lib/python3.5/distutils/core.py", line 163, in setup
raise SystemExit("error: " + str(msg))
SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-2k1hxp0x/cryptography/setup.py", line 319, in
**keywords_with_side_effects(sys.argv)
File "/usr/lib/python3.5/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 269, in init
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 313, in fetch_build_eggs
replace_conflicting=True,
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 826, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 1092, in best_match
return self.obtain(req, installer)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 1104, in obtain
return installer(requirement)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 380, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 663, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 693, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 873, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1101, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1089, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

---

Failed cleaning build dir for cryptography
Running setup.py bdist_wheel for pycparser: started
Running setup.py bdist_wheel for pycparser: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/37/33/b4/350db3d57a6f530e7b9315c3e0fcedd631d06e4f47d269ec26
Successfully built PyDispatcher Twisted pycparser
Failed to build cryptography
Installing collected packages: six, PyDispatcher, idna, asn1crypto, pycparser, cffi, cryptography, pyOpenSSL, w3lib, zope.interface, constantly, incremental, attrs, Automat, hyperlink, Twisted, cssselect, lxml, parsel, pyasn1, pyasn1-modules, service-identity, queuelib, Scrapy, urllib3, certifi, chardet, requests, redis, schedule, scrapy-splash, click
Running setup.py install for cryptography: started
Running setup.py install for cryptography: finished with status 'error'
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-2k1hxp0x/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-9a9fk4sl-record/install-record.txt --single-version-externally-managed --compile:
/usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.5
creating build/lib.linux-x86_64-3.5/cryptography
copying src/cryptography/init.py -> build/lib.linux-x86_64-3.5/cryptography
copying src/cryptography/utils.py -> build/lib.linux-x86_64-3.5/cryptography
copying src/cryptography/about.py -> build/lib.linux-x86_64-3.5/cryptography
copying src/cryptography/fernet.py -> build/lib.linux-x86_64-3.5/cryptography
copying src/cryptography/exceptions.py -> build/lib.linux-x86_64-3.5/cryptography
creating build/lib.linux-x86_64-3.5/cryptography/hazmat
copying src/cryptography/hazmat/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat
creating build/lib.linux-x86_64-3.5/cryptography/x509
copying src/cryptography/x509/extensions.py -> build/lib.linux-x86_64-3.5/cryptography/x509
copying src/cryptography/x509/init.py -> build/lib.linux-x86_64-3.5/cryptography/x509
copying src/cryptography/x509/name.py -> build/lib.linux-x86_64-3.5/cryptography/x509
copying src/cryptography/x509/oid.py -> build/lib.linux-x86_64-3.5/cryptography/x509
copying src/cryptography/x509/base.py -> build/lib.linux-x86_64-3.5/cryptography/x509
copying src/cryptography/x509/certificate_transparency.py -> build/lib.linux-x86_64-3.5/cryptography/x509
copying src/cryptography/x509/general_name.py -> build/lib.linux-x86_64-3.5/cryptography/x509
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/bindings
copying src/cryptography/hazmat/bindings/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/bindings
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/constant_time.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/mac.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/serialization.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/hmac.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/cmac.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/padding.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/keywrap.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
copying src/cryptography/hazmat/primitives/hashes.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/backends
copying src/cryptography/hazmat/backends/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends
copying src/cryptography/hazmat/backends/interfaces.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/bindings/openssl
copying src/cryptography/hazmat/bindings/openssl/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/bindings/openssl
copying src/cryptography/hazmat/bindings/openssl/_conditional.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/bindings/openssl
copying src/cryptography/hazmat/bindings/openssl/binding.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/bindings/openssl
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/twofactor
copying src/cryptography/hazmat/primitives/twofactor/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/twofactor
copying src/cryptography/hazmat/primitives/twofactor/utils.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/twofactor
copying src/cryptography/hazmat/primitives/twofactor/totp.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/twofactor
copying src/cryptography/hazmat/primitives/twofactor/hotp.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/twofactor
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
copying src/cryptography/hazmat/primitives/asymmetric/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
copying src/cryptography/hazmat/primitives/asymmetric/x25519.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
copying src/cryptography/hazmat/primitives/asymmetric/dsa.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
copying src/cryptography/hazmat/primitives/asymmetric/utils.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
copying src/cryptography/hazmat/primitives/asymmetric/dh.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
copying src/cryptography/hazmat/primitives/asymmetric/rsa.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
copying src/cryptography/hazmat/primitives/asymmetric/padding.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
copying src/cryptography/hazmat/primitives/asymmetric/ec.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/asymmetric
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/kdf
copying src/cryptography/hazmat/primitives/kdf/hkdf.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/kdf
copying src/cryptography/hazmat/primitives/kdf/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/kdf
copying src/cryptography/hazmat/primitives/kdf/x963kdf.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/kdf
copying src/cryptography/hazmat/primitives/kdf/scrypt.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/kdf
copying src/cryptography/hazmat/primitives/kdf/pbkdf2.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/kdf
copying src/cryptography/hazmat/primitives/kdf/concatkdf.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/kdf
copying src/cryptography/hazmat/primitives/kdf/kbkdf.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/kdf
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/ciphers
copying src/cryptography/hazmat/primitives/ciphers/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/ciphers
copying src/cryptography/hazmat/primitives/ciphers/modes.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/ciphers
copying src/cryptography/hazmat/primitives/ciphers/aead.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/ciphers
copying src/cryptography/hazmat/primitives/ciphers/algorithms.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/ciphers
copying src/cryptography/hazmat/primitives/ciphers/base.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/primitives/ciphers
creating build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/decode_asn1.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/init.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/x25519.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/dsa.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/aead.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/utils.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/x509.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/ciphers.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/hmac.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/dh.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/rsa.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/cmac.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/ec.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/encode_asn1.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/backend.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
copying src/cryptography/hazmat/backends/openssl/hashes.py -> build/lib.linux-x86_64-3.5/cryptography/hazmat/backends/openssl
running egg_info
writing src/cryptography.egg-info/PKG-INFO
writing top-level names to src/cryptography.egg-info/top_level.txt
writing dependency_links to src/cryptography.egg-info/dependency_links.txt
writing requirements to src/cryptography.egg-info/requires.txt
warning: manifest_maker: standard file '-c' not found

reading manifest file 'src/cryptography.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'docs/_build'
warning: no previously-included files matching '*' found under directory 'vectors'
writing manifest file 'src/cryptography.egg-info/SOURCES.txt'
running build_ext
generating cffi module 'build/temp.linux-x86_64-3.5/_padding.c'
creating build/temp.linux-x86_64-3.5
generating cffi module 'build/temp.linux-x86_64-3.5/_constant_time.c'
generating cffi module 'build/temp.linux-x86_64-3.5/_openssl.c'
building '_openssl' extension
creating build/temp.linux-x86_64-3.5/build
creating build/temp.linux-x86_64-3.5/build/temp.linux-x86_64-3.5
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.5m -c build/temp.linux-x86_64-3.5/_openssl.c -o build/temp.linux-x86_64-3.5/build/temp.linux-x86_64-3.5/_openssl.o -Wconversion -Wno-error=sign-conversion
build/temp.linux-x86_64-3.5/_openssl.c:493:30: fatal error: openssl/opensslv.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-2k1hxp0x/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-9a9fk4sl-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-2k1hxp0x/cryptography/
You are using pip version 8.1.1, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
ERROR: Service 'haipproxy' failed to build: The command '/bin/sh -c pip install -i https://pypi.douban.com/simple/ -r requirements.txt' returned a non-zero code: 1

别的程序里面怎么获取IP

我想把这个程序当做一个提供代理的服务端，然后别的程序里面调用某个链接来获取IP。
这个怎么实现？

ARM support?

在树莓派3b+上尝试部署docker:

pi@raspberrypi:~/haipproxy $ sudo docker-compose up
Starting haipproxy_splash_1 ... done
Starting haipproxy_redis_1  ... done
Starting haipproxy_haipproxy_1 ... done
Attaching to haipproxy_redis_1, haipproxy_splash_1, haipproxy_haipproxy_1
redis_1      | 1:C 08 Apr 05:27:23.229 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1      | 1:C 08 Apr 05:27:23.229 # Redis version=4.0.9, bits=32, commit=00000000, modified=0, pid=1, just started
redis_1      | 1:C 08 Apr 05:27:23.229 # Configuration loaded
redis_1      | 1:M 08 Apr 05:27:23.237 # Warning: 32 bit instance detected but no memory limit set. Setting 3 GB maxmemory limit with 'noeviction' policy now.
redis_1      | 1:M 08 Apr 05:27:23.238 * Running mode=standalone, port=6379.
redis_1      | 1:M 08 Apr 05:27:23.239 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1      | 1:M 08 Apr 05:27:23.239 # Server initialized
redis_1      | 1:M 08 Apr 05:27:23.239 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1      | 1:M 08 Apr 05:27:23.239 * Ready to accept connections
splash_1     | standard_init_linux.go:190: exec user process caused "exec format error"
haipproxy_1  | standard_init_linux.go:190: exec user process caused "exec format error"
haipproxy_splash_1 exited with code 1
haipproxy_haipproxy_1 exited with code 1

猜测是架构的问题？

api 接口 bug

1. api 获取代理全部为 https 的代理

haipproxy/haipproxy/api/core.py

Line 22 in 9e11cde

usage_registry = {task['name']: ProxyFetcher('task') for task in VALIDATOR_TASKS}

因为 ProxyFetcher('task') 传入为字符串'task' 。

haipproxy/haipproxy/client/py_cli.py

Line 112 in 9e11cde

if usage not in score_map.keys():

在判断不在 sorce_map 中会赋值为 https

haipproxy/haipproxy/client/py_cli.py

Line 114 in 9e11cde

usage = 'https'

所以这里逻辑有点问题。正确获取对应 usage 的 proxy 应该是

- usage_registry = {task['name']: ProxyFetcher('task') for task in VALIDATOR_TASKS}
+ usage_registry = {task['name']: ProxyFetcher(task['name']) for task in VALIDATOR_TASKS}

2. delete 接口参数设置错误

haipproxy/haipproxy/api/core.py

Line 58 in 9e11cde

@app.route("/proxy/delete/<usage>/<proxy>")

使用这个接口的方式应该是

http://127.0.0.1:5000/proxy/delete/zhihu/http://1.63.153.153:80

但是很明显这个 URL 是有问题的 对于 proxy 参数，如果要正常传入应该在 request body 中发送。

建议优化接口参数接收方式和数据接收方式。

使用docker部署时遇见一个错误

使用docker部署时遇见两个错误：

#1 c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory

#2 build/temp.linux-x86_64-3.5/_openssl.c:493:30: fatal error: openssl/opensslv.h: No such file or directory

环境：CentOS 7.4，
但是我已经安装了相关联的devel

crawler start and finish in 1 second

hey, first thank you of your great work,
i have some problem about this repo,

now my machine on aliyun, and status is:
splash: install by docker, run at localhost:8058
squid: install by yum, run at *:3128
and i use pip to install your all requirements

my python version is 3.6.4

python crawler_booter.py --usage crawler

and i see log in logs directory like this:

2018-05-22 16:24:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-05-22 16:24:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-05-22 16:24:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-05-22 16:24:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-05-22 16:24:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 267658),
'log_count/INFO': 4,
'memusage/max': 46108672,
'memusage/startup': 46108672,
'start_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 242833)}
2018-05-22 16:24:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 269282),
'log_count/INFO': 4,
'memusage/max': 46108672,
'memusage/startup': 46108672,
'start_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 250614)}
2018-05-22 16:24:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 270711),
'log_count/INFO': 4,
'memusage/max': 46309376,
'memusage/startup': 46309376,
'start_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 258190)}
2018-05-22 16:24:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 272169),
'log_count/INFO': 4,
'memusage/max': 46559232,
'memusage/startup': 46559232,
'start_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 265645)}

can you tell me how can i run this? thanks

运行crawler_booter.py --usage crawler 提示 no module scheduler

作者源码包导入是有问题吗 运行提示包找不到

0 proxies have been fetched

为什么我现在运行完一个代理都没有了呢？

代理ip不更新

您好，我在Ubuntu16.04系统下docker部署，运行正常。可是过了两天后，代理ip不更新一直稳定在400个。这是什么情况

haipproxy:ttl:http,haipproxy:ttl:https的score值一直不会更新

使用的haipproxy master分支
redis中
haipproxy:ttl:
下的元素score值一直不会更新。
代码里边更新haipproxy:ttl中元素的逻辑在哪里？

haipproxy_1 | squid: ERROR: No running copy

When I run docker-compose up, I got this error and can not use proxy with ip:3128:

haipproxy_1  | 2018-03-08 09:19:18 [client] INFO: no proxies got at this turn
haipproxy_1  | squid: ERROR: No running copy
haipproxy_1  | 2018-03-08 09:19:18 [client] INFO: update squid conf successfully
haipproxy_1  | 2018-03-08 09:22:18 [schedule] INFO: Running job Every 3 minutes do update_conf() (last run: 2018-03-08 09:19:18, next run: 2018-03-08 09:22:18)
haipproxy_1  | squid: ERROR: No running copy
haipproxy_1  | 2018-03-08 09:22:18 [client] INFO: update squid conf successfully
haipproxy_1  | 2018-03-08 09:25:18 [schedule] INFO: Running job Every 3 minutes do update_conf() (last run: 2018-03-08 09:22:18, next run: 2018-03-08 09:25:18)
haipproxy_1  | squid: ERROR: No running copy
haipproxy_1  | 2018-03-08 09:25:19 [client] INFO: update squid conf successfully
haipproxy_1  | 2018-03-08 09:28:19 [schedule] INFO: Running job Every 3 minutes do update_conf() (last run: 2018-03-08 09:25:19, next run: 2018-03-08 09:28:19)
haipproxy_1  | squid: ERROR: No running copy
haipproxy_1  | 2018-03-08 09:28:19 [client] INFO: update squid conf successfully

请问怎么获取gfw的ip，需要在机器上配置什么gfw工具呢

你好，我看项目中的gfw_spider是通过127.0.0.1:8123这个端口来提供fanqiang的服务，请问在服务端需要配置什么工具呢，，ss可以吗 有具体的配置方法吗，，我尝试了很多种方法都失败了

docker-compose up 启动异常

在执行 docker-compose up 时，启动出现错误：

这里是执行的 log：squid.log

因为使用的是云服务器，配置比较低，如下：

我查看过 log 的错误，貌似是无法连接到 Redis 导致的，但是报错信息是无法获取内存，不知道有没有可能是机子内存过小导致的。麻烦查看下是什么问题。

关于IP归属地的支持.....

建议作者加入每次ip加入的时候判断归属地的功能,这样我们可以根据IP归属地来输出可用的ip.
推荐一个好的IP归属地查询工具:https://gitee.com/lionsoul/ip2region (码云的,请放心食用)

Unhandled error in Deferred:

版本配置：centos7，python3.5.2，Twisted-18.4.0
启动scrapy worker，包括代理IP采集器和校验器时，报Unhandled error in Deferred:，请问是什么问题？谢谢

报个 Docker 部署异常

• platform: Windows 10
• source: master branch at latest commit 7423e1d

$ docker-compose.exe up
Removing haipproxy_haipproxy_1
haipproxy_splash_1 is up-to-date
haipproxy_redis_1 is up-to-date
Recreating b29055e49998_haipproxy_haipproxy_1 ... error

ERROR: for b29055e49998_haipproxy_haipproxy_1  Cannot start service haipproxy: b"error while creating mount source path '/host_mnt/c/Users/<omitted>/workspace/haipproxy': mkdir /host_mnt/c/Users/<omitted>/workspace: file exists"

ERROR: for haipproxy  Cannot start service haipproxy: b"error while creating mount source path '/host_mnt/c/Users/<omitted>/workspace/haipproxy': mkdir /host_mnt/c/Users/<omitted>/workspace: file exists"
ERROR: Encountered errors while bringing up the project.

修改 docker-compose.yml 注释掉如下2行后可用

version: "2.0"
services:
  redis:
    image: redis
    ports:
      - "6379:6379"
    command: >
      --requirepass 123456
  splash:
    image: scrapinghub/splash
    ports:
      - "8050:8050"
  haipproxy:
    image: registry.cn-hangzhou.aliyuncs.com/weakiwi/haipproxy-alpine-image:stable
    # volumes:            <----------
    #   - .:/haipproxy    <----------
    command: sh run.sh
    ports:
      - "3128:3128"
    links:
      - redis
      - splash

启动crawler_booter.py时候出现Unhandled error in Deferred:

是哪里的配置出现问题了吗？

GFW_PROXY 需要单独设置吗？

1.GFW_PROXY = 'http://127.0.0.1:8123'
这里的意思是要单独设置一个可以翻墙的代理吗？难道不能用内置的ip代理池自动抓取吗？

2.scrapy-splash需要单独启动吗？
能不能用一个命令启动服务，方便直接单机部署，想集群部署才用多个启动命令。

3.client.py_cli可以移动位置吗？
使用其他爬虫框架，如pyspider。想要用内置的py_cli模式，是要把client目录copy到第三方框架路径下吗？或者如何跨目录调用？

目前问题如上，请作者不吝赐教

一段奇怪的报错

haipproxy_1 | 2018/03/10 09:53:43| WARNING: HTTP: Invalid Response: Bad header encountered from http://userapi.plu.cn/user/collect?roomId=2115304 AKA userapi.plu.cn/user/collect?roomId=2115304

上下文:
haipproxy_1 | 2018/03/10 09:53:18| TCP connection to 160.16.223.156/8080 failed
haipproxy_1 | 2018/03/10 09:53:20| TCP connection to 160.16.223.156/8080 failed
haipproxy_1 | 2018/03/10 09:53:20| Detected DEAD Parent: proxy-34
haipproxy_1 | 2018/03/10 09:53:20| Detected REVIVED Parent: proxy-34
haipproxy_1 | 2018/03/10 09:53:24| WARNING: HTTP: Invalid Response: Bad header encountered from http://userapi.longzhu.com/user/collect?roomId=2220760 AKA userapi.longzhu.com/user/collect?roomId=2220760
haipproxy_1 | 2018/03/10 09:53:28| local=172.18.0.4:3128 remote=120.76.222.200:63864 FD 223 flags=1: read/write failure: (32) Broken pipe
haipproxy_1 | 2018/03/10 09:53:30| TCP connection to 160.16.214.186/8080 failed
haipproxy_1 | 2018/03/10 09:53:35| TCP connection to 160.16.223.146/8080 failed
haipproxy_1 | 2018/03/10 09:53:35| TCP connection to 160.16.214.186/8080 failed
haipproxy_1 | 2018/03/10 09:53:35| TCP connection to 95.215.25.233/8080 failed
haipproxy_1 | 2018/03/10 09:53:35| Detected DEAD Parent: proxy-31
haipproxy_1 | 2018/03/10 09:53:35| Detected REVIVED Parent: proxy-31
haipproxy_1 | 2018/03/10 09:53:42| TCP connection to 95.215.25.233/8080 failed
haipproxy_1 | 2018/03/10 09:53:42| TCP connection to 160.16.213.241/8080 failed
haipproxy_1 | 2018/03/10 09:53:43| WARNING: HTTP: Invalid Response: Bad header encountered from http://userapi.plu.cn/user/collect?roomId=2115304 AKA userapi.plu.cn/user/collect?roomId=2115304

请问我没做龙珠的爬虫，怎么就出现龙珠直播的API了，难道服务器被黑了

AttributeError: 'NoneType' object has no attribute 'decode'

validator scheduler is starting...
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/haipproxy/haipproxy/scheduler/scheduler.py", line 161, in schedule_task_with_lock
release_lock(conn, task_name, lock_indentifier)
File "/haipproxy/haipproxy/utils/redis_util.py", line 41, in release_lock
identifier_origin = pipe.get(lock_name).decode()
AttributeError: 'NoneType' object has no attribute 'decode'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "scheduler_booter.py", line 22, in
scheduler_start()
File "/usr/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/haipproxy/haipproxy/scheduler/scheduler.py", line 197, in scheduler_start
scheduler.schedule_all_right_now()
File "/haipproxy/haipproxy/scheduler/scheduler.py", line 73, in schedule_all_right_now
pool.map(self.schedule_task_with_lock, self.tasks)
File "/usr/lib/python3.6/multiprocessing/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
AttributeError: 'NoneType' object has no attribute 'decode'

Https校验器问题

1.请问在校验器里能够拿到当前校验的代理IP么
2.有的代理虽然能够访问https://httpbin.org/ip，但它未必是HTTPS协议的，所以这个校验器是不是有点问题
class HttpsValidator(BaseValidator, ValidatorRedisSpider):
"""This validator check the liveness of https proxy resources"""
name = 'https'
urls = [
'https://httpbin.org/ip',
]
task_queue = TEMP_HTTPS_QUEUE
score_queue = VALIDATED_HTTPS_QUEUE
ttl_queue = TTL_HTTPS_QUEUE
speed_queue = SPEED_HTTPS_QUEUE

structure of setting.py in docker installation

What are the values of SPLASH_URL and REDIS_HOST when we install docker?

I used sudo docker-compose up with default settings and I get errors

haipproxy_1  | 2018-03-08 12:40:38 [client] INFO: the updating task is starting...
haipproxy_1  | Traceback (most recent call last):
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 484, in connect
haipproxy_1  |     sock = self._connect()
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 541, in _connect
haipproxy_1  |     raise err
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 529, in _connect
haipproxy_1  |     sock.connect(socket_address)
haipproxy_1  | ConnectionRefusedError: [Errno 111] Connection refused
haipproxy_1  | 
haipproxy_1  | During handling of the above exception, another exception occurred:
haipproxy_1  | 
haipproxy_1  | Traceback (most recent call last):
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/client.py", line 2879, in execute
haipproxy_1  |     return execute(conn, stack, raise_on_error)
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/client.py", line 2810, in _execute_pipeline
haipproxy_1  |     connection.send_packed_command(all_cmds)
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 585, in send_packed_command
haipproxy_1  |     self.connect()
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 489, in connect
haipproxy_1  |     raise ConnectionError(self._error_message(e))
haipproxy_1  | redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:6379. Connection refused.
haipproxy_1  | 
haipproxy_1  | During handling of the above exception, another exception occurred:
haipproxy_1  | 
haipproxy_1  | Traceback (most recent call last):
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 484, in connect
haipproxy_1  |     sock = self._connect()
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 541, in _connect
haipproxy_1  |     raise err
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 529, in _connect
haipproxy_1  |     sock.connect(socket_address)
haipproxy_1  | ConnectionRefusedError: [Errno 111] Connection refused
haipproxy_1  | 
haipproxy_1  | During handling of the above exception, another exception occurred:
haipproxy_1  | 
haipproxy_1  | Traceback (most recent call last):
haipproxy_1  |   File "squid_update.py", line 19, in <module>
haipproxy_1  |     squid_conf_update()
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 722, in __call__
haipproxy_1  |     return self.main(*args, **kwargs)
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 697, in main
haipproxy_1  |     rv = self.invoke(ctx)
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 895, in invoke
haipproxy_1  |     return ctx.invoke(self.callback, **ctx.params)
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 535, in invoke
haipproxy_1  |     return callback(*args, **kwargs)
haipproxy_1  |   File "/haipproxy/scheduler/scheduler.py", line 234, in squid_conf_update
haipproxy_1  |     client.update_conf()
haipproxy_1  |   File "/haipproxy/client/squid.py", line 52, in update_conf
haipproxy_1  |     scored_proxies, ttl_proxies, speed_proxies = pipe.execute()
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/client.py", line 2894, in execute
haipproxy_1  |     return execute(conn, stack, raise_on_error)
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/client.py", line 2810, in _execute_pipeline
haipproxy_1  |     connection.send_packed_command(all_cmds)
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 585, in send_packed_command
haipproxy_1  |     self.connect()
haipproxy_1  |   File "/usr/local/lib/python3.5/dist-packages/redis/connection.py", line 489, in connect
haipproxy_1  |     raise ConnectionError(self._error_message(e))
haipproxy_1  | redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:6379. Connection refused.
haipproxy_haipproxy_1 exited with code 1

squid模式下，如何区分http和https？

比如用代理访问一个https的网页，会不会轮询到一个http的代理（不支持https）。如果是，有没有解决的办法？

windows 7 docker compose up报错

windows平台docker compose up报错，错误代码如下：
pywintypes.error: (2, 'WaitNamedPipe', '\xcf\xb5\xcd\xb3\xd5\xd2\xb2\xbb\xb5\xbd\xd6\xb8\xb6\xa8\xb5\xc4\xce\xc4\xbc\xfe\xa1\xa3')
Failed to execute script docker-compose
请问有解决方法么？

如何清空squid的缓存？

因为跑了一些可用性低的ip我想清空squid的缓存，重新跑一遍。
我尝试删除了squid.conf文件，restart squid服务，也把服务端和调度器重启了
再次跑的时候发现旧数据几百条ip又存入到了squid.conf里了...

可以加一个代理存活时间的属性么

项目需要一些长期有效的代理

来自知乎的issue。代理IP池很多报timeout以及connection refused

在项目中使用了你的代理IP池，并按照文档编写了针对自己所爬网站的校验器，使用的是py_cli客户端，且在代理成功及失败都会有反馈。
具体下载中间件代码如下所示：

`class JobboleProxyMiddleware(object):

def __init__(self):
    args = dict(host='127.0.0.1', port=6379, password='123456', db=0)
    self.scheme = 'wcgroup'
    self.fetcher = ProxyFetcher(self.scheme, strategy='greedy', redis_args=args)
    self.proxy = self.get_next_proxy()
    self.start = time.time() * 1000

def process_request(self, request, spider):
    self.start = time.time() * 1000
    logger.info("this is request ip:" + self.proxy)
    request.meta['proxy'] = self.proxy

    return None

def process_response(self, request, response, spider):
    end = time.time() * 1000
    # 如果返回的response状态不是200，重新生成当前request对象
    if response.status != 200:
        self.fetcher.proxy_feedback('failure', self.proxy)
        logger.info('Current ip is blocked! The proxy is {}'.format(self.proxy))
        # 对当前request换下一个代理
        self.proxy = self.get_next_proxy()
        request.meta['proxy'] = self.proxy
        return request
    else:
        spider.logger.info('Request succeeded! The proxy is {}'.format(self.proxy))
        # if you use greedy strategy, you must feedback
        duration=int(end - self.start)
        self.fetcher.proxy_feedback('success', self.proxy, duration)
        return response

def process_exception(self, request, exception, spider):
    logger.error('Request failed!The proxy is {}, {}. Exception:{}'.format(request.meta['proxy'], self.proxy, exception))
    # it's important to feedback, otherwise you may use the bad proxy next time
    self.fetcher.proxy_feedback('failure', self.proxy)
    # 对当前request换下一个代理
    self.proxy = self.get_next_proxy()
    request.meta['proxy'] = self.proxy
    return request

def get_next_proxy(self):
    # 获取一个可用代理
    return self.fetcher.get_proxy()

def spider_opened(self, spider):
    logger.info('Spider opened: %s' % spider.name)

`

报错信息如下：

2018-05-30 21:28:20 [root] ERROR: Request failed!The proxy is http://210.26.122.78:8080. Exception:User timeout caused connection failure: Getting https://www.weixinqun.com/openid?p=127 took longer than 30.0 seconds.. 2018-05-30 21:28:20 [root] INFO: monitor process_request:https://www.weixinqun.com/openid?p=146 2018-05-30 21:28:20 [root] INFO: this is request ip:http://89.249.242.123:53281 2018-05-30 21:28:20 [root] INFO: RotateUserAgentMiddleware: new ua:Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.36 Safari/536.5 2018-05-30 21:28:21 [root] INFO: respd process_exception:https://www.weixinqun.com/openid?p=128 2018-05-30 21:28:21 [root] ERROR: Request failed!The proxy is http://89.249.242.123:53281. Exception:User timeout caused connection failure: Getting https://www.weixinqun.com/openid?p=128 took longer than 30.0 seconds.. 2018-05-30 21:28:21 [root] INFO: monitor process_request:https://www.weixinqun.com/openid?p=147 2018-05-30 21:28:21 [root] INFO: this is request ip:http://202.29.237.250:80 2018-05-30 21:28:21 [root] INFO: RotateUserAgentMiddleware: new ua:Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.36 Safari/536.5 2018-05-30 21:28:22 [root] INFO: respd process_exception:https://www.weixinqun.com/openid?p=130 2018-05-30 21:28:22 [root] ERROR: Request failed!The proxy is http://202.29.237.250:80. Exception:User timeout caused connection failure: Getting https://www.weixinqun.com/openid?p=130 took longer than 30.0 seconds.. 2018-05-30 21:28:22 [root] INFO: monitor process_request:https://www.weixinqun.com/openid?p=148 2018-05-30 21:28:22 [root] INFO: this is request ip:http://218.75.144.25:9000 2018-05-30 21:28:22 [root] INFO: RotateUserAgentMiddleware: new ua:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3 2018-05-30 21:28:23 [root] INFO: respd process_exception:https://www.weixinqun.com/openid?p=131 2018-05-30 21:28:23 [root] ERROR: Request failed!The proxy is http://218.75.144.25:9000. Exception:User timeout caused connection failure: Getting https://www.weixinqun.com/openid?p=131 took longer than 30.0 seconds.. 2018-05-30 21:28:23 [root] INFO: monitor process_request:https://www.weixinqun.com/openid?p=149 2018-05-30 21:28:23 [root] INFO: this is request ip:http://139.59.2.223:8888 2018-05-30 21:28:23 [root] INFO: RotateUserAgentMiddleware: new ua:Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/19.77.34.5 Safari/537.1 2018-05-30 21:28:25 [root] INFO: respd process_exception:https://www.weixinqun.com/openid?p=144 2018-05-30 21:28:25 [root] ERROR: Request failed!The proxy is http://139.59.2.223:8888. Exception:Connection was refused by other side: 111: Connection refused. 2018-05-30 21:28:25 [root] INFO: monitor process_request:https://www.weixinqun.com/openid?p=150 2018-05-30 21:28:25 [root] INFO: this is request ip:http://39.104.58.6:8080 2018-05-30 21:28:25 [root] INFO: RotateUserAgentMiddleware: new ua:Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.0 Safari/536.3 2018-05-30 21:28:26 [root] INFO: respd process_exception:https://www.weixinqun.com/openid?p=133 2018-05-30 21:28:26 [root] ERROR: Request failed!The proxy is http://39.104.58.6:8080. Exception:User timeout caused connection failure: Getting https://www.weixinqun.com/openid?p=133 took longer than 30.0 seconds.. 2018-05-30 21:28:26 [root] INFO: monitor process_request:https://www.weixinqun.com/openid?p=151 2018-05-30 21:28:26 [root] INFO: this is request ip:http://49.51.68.122:1080 2018-05-30 21:28:26 [root] INFO: RotateUserAgentMiddleware: new ua:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3 2018-05-30 21:28:29 [root] INFO: respd process_exception:https://www.weixinqun.com/openid?p=134 2018-05-30 21:28:29 [root] ERROR: Request failed!The proxy is http://49.51.68.122:1080. Exception:User timeout caused connection failure: Getting https://www.weixinqun.com/openid?p=134 took longer than 30.0 seconds.. 2018-05-30 21:28:29 [root] INFO: monitor process_request:https://www.weixinqun.com/openid?p=152 2018-05-30 21:28:29 [root] INFO: this is request ip:http://95.180.242.245:53281

Auto try anothor ip proxy when the current ip is disabled.

我的设置: {'https': '127.0.0.1:3128', 'http': '127.0.0.1:3128'}
每次访问外网ip都会自动切换，但是有一个问题，如果当前代理ip已经失效或者请求错误，并没有自动尝试切换为可用ip的能力．

我想要的效果是：当我使用127.0.0.1:3128作为代理时，如果后端代理ip失效，自动切换其它代理ip,保证我的应用能收到正确的返回．

执行 python crawler_booter.py --usage crawler 出现语法错误

Hello，阿里云ECS上部署haipproxy，服务器端部署过程中，执行 “python crawler_booter.py --usage crawler” 还有其它三个命令出现以下错误：

Traceback (most recent call last):
File "crawler_booter.py", line 19, in
from scheduler import crawler_start
File "/var/www/myproxy/haipproxy-0.1/scheduler/init.py", line 5, in
from .scheduler import (
File "/var/www/myproxy/haipproxy-0.1/scheduler/scheduler.py", line 21, in
from crawler.spiders import all_spiders
File "/var/www/myproxy/haipproxy-0.1/crawler/spiders/init.py", line 45, in
from .common_spider import CommonSpider
File "/var/www/myproxy/haipproxy-0.1/crawler/spiders/common_spider.py", line 6, in
from ..redis_spiders import RedisSpider
File "/var/www/myproxy/haipproxy-0.1/crawler/redis_spiders.py", line 112
yield from self.next_requests_process(self.task_queue)
^
SyntaxError: invalid syntax

谢谢！

修改一下 requirements.txt 中版本信息

虚拟环境安装的时候会提示版本问题

docker 2.7.0 has requirement requests!=2.18.0,>=2.14.2, but you'll have requests 2.13.0 which is incompatible.

请教下docker方式部署

用 docker 方式部署在 VPS 上，顺便修改了 setting 里的密码。本地连接的时候发现密码修改没有生效，用默认的 123456 可以连接上。这个是怎么回事呢？
另外，获取可用代理列表一直是空的，除了按照 WIKI 里的 docker-compose up 之外，还需要执行其他操作么？

Scrapoxy Integration

Hello,

Did you try to integrate Scrapoxy.io into haipproxy ?

Best regards,
Fabien

添加新的代理IP供应网站

首先感谢大神们的开源:)
如果想添加新的代理IP供应网站，应该修改哪些配置或者代码呢？
是否考虑添加这个feature呢？毕竟项目提供的这些代理IP供应网站可能难以满足有些特定的需求，比如对某些国外网站的爬取:)

运行时候报错：Unhandled error in Deferred:

环境搭建好了，按照文档执行python crawler_booter.py --usage crawler 出现以下错误：

root@~#python crawler_booter.py --usage crawler
Unhandled error in Deferred:

Unhandled error in Deferred:

Unhandled error in Deferred:

Unhandled error in Deferred:

Unhandled error in Deferred:

---

root@~#scrapy version -v

Scrapy : 1.5.0
lxml : 4.2.4.0
libxml2 : 2.9.8
cssselect : 1.0.3
parsel : 1.5.0
w3lib : 1.19.0
Twisted : 17.1.0
Python : 3.5.6 (default, Aug 8 2018, 18:36:31) - [GCC 4.4.7 20120313 (Red Hat 4.4.7-18)]
pyOpenSSL : 18.0.0 (OpenSSL 1.1.0h 27 Mar 2018)
cryptography : 2.3
Platform : Linux-2.6.32-504.el6.x86_64-x86_64-with-centos-6.6-Final

---

root@~#docker run -p 8050:8050 scrapinghub/splash

2018-08-09 03:18:26+0000 [-] Log opened.
2018-08-09 03:18:26.263532 [-] Splash version: 3.2
2018-08-09 03:18:26.264100 [-] Qt 5.9.1, PyQt 5.9, WebKit 602.1, sip 4.19.3, Twisted 16.1.1, Lua 5.2
2018-08-09 03:18:26.264174 [-] Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609]
2018-08-09 03:18:26.264231 [-] Open files limit: 10240
2018-08-09 03:18:26.264281 [-] Can't bump open files limit
2018-08-09 03:18:26.367108 [-] Xvfb is started: ['Xvfb', ':2053627847', '-screen', '0', '1024x768x24', '-nolisten', 'tcp']
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
2018-08-09 03:18:26.442532 [-] proxy profiles support is enabled, proxy profiles path: /etc/splash/proxy-profiles
2018-08-09 03:18:26.521230 [-] verbosity=1
2018-08-09 03:18:26.521334 [-] slots=50
2018-08-09 03:18:26.521397 [-] argument_cache_max_entries=500
2018-08-09 03:18:26.521640 [-] Web UI: enabled, Lua: enabled (sandbox: enabled)
2018-08-09 03:18:26.521711 [-] Server listening on 0.0.0.0:8050
2018-08-09 03:18:26.522198 [-] Site starting on 8050
2018-08-09 03:18:26.522284 [-] Starting factory <twisted.web.server.Site object at 0x7fb37add67b8>

---

Google了很多方案都解决不了，有知道怎么解决这个问题的吗？感谢~

调度启动失败

2018-04-13 11:20:22 [validator] INFO: crawler scheduler is starting...
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/custom-disk1/haipproxy-0.1/scheduler/scheduler.py", line 112, in schedule_task_with_lock
    if not r or (now - int(r.decode('utf-8'))) >= internal * 60:
TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "scheduler_booter.py", line 22, in <module>
    scheduler_start()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/custom-disk1/haipproxy-0.1/scheduler/scheduler.py", line 186, in scheduler_start
    scheduler.schedule_all_right_now()
  File "/custom-disk1/haipproxy-0.1/scheduler/scheduler.py", line 73, in schedule_all_right_now
    pool.map(self.schedule_task_with_lock, self.tasks)
  File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'

Docker. Squid is already running

Hi.
I am trying to use haipproxy with docker:

~/$ docker version
Client:
 Version:	18.02.0-ce
 API version:	1.36
 Go version:	go1.9.3
 Git commit:	fc4de44
 Built:	Wed Feb  7 21:16:33 2018
 OS/Arch:	linux/amd64
 Experimental:	false
 Orchestrator:	swarm

Server:
 Engine:
  Version:	18.02.0-ce
  API version:	1.36 (minimum version 1.12)
  Go version:	go1.9.3
  Git commit:	fc4de44
  Built:	Wed Feb  7 21:15:05 2018
  OS/Arch:	linux/amd64
  Experimental:	false

on Ubuntu 16.04.3 LTS

And I am constantly getting error:

haipproxy_1 | 2018/03/12 14:46:39| Squid is already running! Process ID 12

I added to docker-compose:

    tty: true
    restart: always

but without too much success:

haipproxy_1 | 2018/03/12 14:53:01| Set Current Directory to /var/spool/squid
haipproxy_1 | 2018/03/12 14:53:01| Starting Squid Cache version 3.5.12 for x86_64-pc-linux-gnu...
haipproxy_1 | 2018/03/12 14:53:01| Service Name: squid
haipproxy_1 | 2018/03/12 14:53:01| Process ID 12
haipproxy_1 | 2018/03/12 14:53:01| Process Roles: master worker
haipproxy_1 | 2018/03/12 14:53:01| With 1048576 file descriptors available
haipproxy_1 | 2018/03/12 14:53:01| Initializing IP Cache...
haipproxy_1 | 2018/03/12 14:53:01| DNS Socket created at [::], FD 8
haipproxy_1 | 2018/03/12 14:53:01| DNS Socket created at 0.0.0.0, FD 9
haipproxy_1 | 2018/03/12 14:53:01| Adding nameserver 127.0.0.11 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:53:01| Adding ndots 1 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:53:01| Logfile: opening log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:01| Logfile Daemon: opening log /var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:01| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
haipproxy_1 | 2018/03/12 14:53:01| Store logging disabled
haipproxy_1 | 2018/03/12 14:53:01| Swap maxSize 0 + 262144 KB, estimated 20164 objects
haipproxy_1 | 2018/03/12 14:53:01| Target number of buckets: 1008
haipproxy_1 | 2018/03/12 14:53:01| Using 8192 Store buckets
haipproxy_1 | 2018/03/12 14:53:01| Max Mem size: 262144 KB
haipproxy_1 | 2018/03/12 14:53:01| Max Swap size: 0 KB
haipproxy_1 | 2018/03/12 14:53:01| Using Least Load store dir selection
haipproxy_1 | 2018/03/12 14:53:01| Set Current Directory to /var/spool/squid
haipproxy_1 | 2018/03/12 14:53:01| Finished loading MIME types and icons.
haipproxy_1 | 2018/03/12 14:53:01| HTCP Disabled.
haipproxy_1 | 2018/03/12 14:53:01| commBind: Cannot bind socket FD 13 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:53:01| commBind: Cannot bind socket FD 14 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:53:01| ERROR: Failed to create helper child read FD: UDP[::1]
haipproxy_1 | 2018/03/12 14:53:01| Squid plugin modules loaded: 0
haipproxy_1 | 2018/03/12 14:53:01| Adaptation support is off.
haipproxy_1 | 2018/03/12 14:53:01| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 12 flags=9
haipproxy_1 | 2018/03/12 14:53:02| storeLateRelease: released 0 objects
haipproxy_1 | 2018/03/12 14:53:04| Reconfiguring Squid Cache (version 3.5.12)...
haipproxy_1 | 2018/03/12 14:53:04| Closing HTTP port [::]:3128
haipproxy_1 | 2018/03/12 14:53:04| Logfile: closing log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:04| Logfile Daemon: closing log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initializing Authentication Schemes ...
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication Scheme 'basic'
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication Scheme 'digest'
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication Scheme 'negotiate'
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication Scheme 'ntlm'
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication.
haipproxy_1 | 2018/03/12 14:53:04| Processing Configuration File: /etc/squid/squid.conf (depth 0)
haipproxy_1 | 2018/03/12 14:53:04| Logfile: opening log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:04| Logfile Daemon: opening log /var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:04| Squid plugin modules loaded: 0
haipproxy_1 | 2018/03/12 14:53:04| Adaptation support is off.
haipproxy_1 | 2018/03/12 14:53:04| Store logging disabled
haipproxy_1 | 2018/03/12 14:53:04| DNS Socket created at [::], FD 10
haipproxy_1 | 2018/03/12 14:53:04| DNS Socket created at 0.0.0.0, FD 11
haipproxy_1 | 2018/03/12 14:53:04| Adding nameserver 127.0.0.11 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:53:04| Adding ndots 1 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:53:04| HTCP Disabled.
haipproxy_1 | 2018/03/12 14:53:04| commBind: Cannot bind socket FD 13 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:53:04| commBind: Cannot bind socket FD 14 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:53:04| ERROR: Failed to create helper child read FD: UDP[::1]
haipproxy_1 | 2018/03/12 14:53:04| Finished loading MIME types and icons.
haipproxy_1 | 2018/03/12 14:53:04| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 12 flags=9
haipproxy_1 | 2018/03/12 14:56:04| Reconfiguring Squid Cache (version 3.5.12)...
haipproxy_1 | 2018/03/12 14:56:04| Closing HTTP port [::]:3128
haipproxy_1 | 2018/03/12 14:56:04| Logfile: closing log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:56:04| Logfile Daemon: closing log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initializing Authentication Schemes ...
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication Scheme 'basic'
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication Scheme 'digest'
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication Scheme 'negotiate'
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication Scheme 'ntlm'
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication.
haipproxy_1 | 2018/03/12 14:56:04| Processing Configuration File: /etc/squid/squid.conf (depth 0)
haipproxy_1 | 2018/03/12 14:56:05| Logfile: opening log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:56:05| Logfile Daemon: opening log /var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:56:05| Squid plugin modules loaded: 0
haipproxy_1 | 2018/03/12 14:56:05| Adaptation support is off.
haipproxy_1 | 2018/03/12 14:56:05| Store logging disabled
haipproxy_1 | 2018/03/12 14:56:05| DNS Socket created at [::], FD 10
haipproxy_1 | 2018/03/12 14:56:05| DNS Socket created at 0.0.0.0, FD 11
haipproxy_1 | 2018/03/12 14:56:05| Adding nameserver 127.0.0.11 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:56:05| Adding ndots 1 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:56:05| HTCP Disabled.
haipproxy_1 | 2018/03/12 14:56:05| commBind: Cannot bind socket FD 13 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:56:05| commBind: Cannot bind socket FD 14 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:56:05| ERROR: Failed to create helper child read FD: UDP[::1]
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 89.236.17.108/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 61.220.26.97/80/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 159.89.195.153/8118/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 35.196.26.166/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 34.231.147.235/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 216.177.233.181/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 103.74.246.124/65205/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 153.126.201.100/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 121.129.127.209/80/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 109.105.40.39/53281/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 179.106.37.39/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 144.76.176.72/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 181.49.24.126/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 191.210.166.52/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 137.74.254.242/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 192.116.142.153/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 137.74.168.174/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 117.6.161.118/53281/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 128.199.138.67/8118/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 186.227.8.21/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 82.196.12.94/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Finished loading MIME types and icons.
haipproxy_1 | 2018/03/12 14:56:05| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 12 flags=9
haipproxy_1 | 2018/03/12 14:56:31| netdbExchangeHandleReply: corrupt data, aborting
haipproxy_1 | 2018/03/12 14:56:39| netdbExchangeHandleReply: corrupt data, aborting
haipproxy_1 | 2018/03/12 15:02:03| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:04| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:05| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:06| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:07| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:09| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:13| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:20| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:33| Squid is already running! Process ID 12
haipproxy_haipproxy_1 exited with code 0

Is it normal? Or maybe I should change some other settings?

other server can't use squid make https request

i am sorry i come here again,,

Now i deploy haipproxy on server A. I have other server B, i need B request server A, and get a agent ip.

after my test, i get this below:

run this same code:

import requests
proxies = {'https': 'http://myip:3128'}
resp = requests.get('https://httpbin.org/ip', proxies=proxies)
print(resp.text)

on server A, i can get a agent ip ( not same with my actually ip )
on server B, i can't get a agent ip and it raise exception like this:

requests.exceptions.ProxyError: HTTPSConnectionPool(host='httpbin.org', port=443): Max retries exceeded with url: /ip (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x041155F0>: Failed to establish a new connection: [WinError 10060]

and when i change the code
proxies = {'https': 'http://myip:3128'}
to
proxies = {'http': 'http://myip:3128'}
it return a ip on server B, but it is my real ip, not a agent ip, can you help me fix this ?

my squid config like this:

acl localnet src servier_a_ip/8
acl localnet src server_b_ip/8
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager

http_access allow localnet
http_access allow localhost

http_access deny all

http_port 3128

i also try to make a CA certificate and use https_port config, but it useless, everything not change, server b can't use https proxy.

thanks