germey / cookiespool

CookiesPool Based on Redis

CookiesPool / Cookies池

可扩展的Cookies池，目前对接了新浪微博，可自行扩展其他站点

优化版(推荐)：https://github.com/Python3WebSpider/CookiesPool

安装

pip3 install -r requirements.txt

基础配置

修改cookiespool/config.py

数据库配置

account:weibo:账号

cookies:weibo:账号

Value分别为密码和Cookies

账号自行某宝购买

Redis连接信息到cookiespool/config文件修改

云打码平台配置

到yundama.com注册开发者和普通用户。

开发者申请应用ID和KEY，普通用户用于充值登录。

配置信息到cookiespool/config文件修改

进程开关

配置信息到cookiespool/config文件修改

运行

python3 run.py

批量导入

python3 importer.py

请输入账号密码组, 输入exit退出读入
18459748505----astvar3647
14760253606----gmidy8470
14760253607----uoyuic8427
18459749258----rktfye8937
账号 18459748505 密码 astvar3647
录入成功
账号 14760253606 密码 gmidy8470
录入成功
账号 14760253607 密码 uoyuic8427
录入成功
账号 18459749258 密码 rktfye8937
录入成功
exit

运行效果

开启Generator、API，关闭Tester为例：

Generating Cookies
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Getting 526 accounts from Redis
Getting Cookies of  weibo [email protected] 1nr0nkwes7
Generating Cookies of [email protected]
出现验证码，开始识别验证码
Retrying:  1400009488 Count:  1
{'cid': 1400009488, 'method': 'result'}
{'cid': 1400009488, 'ret': -3002, 'text': ''}
云打码验证码还在识别
Retrying:  1400009488 Count:  2
{'cid': 1400009488, 'method': 'result'}
{'cid': 1400009488, 'ret': -3002, 'text': ''}
云打码验证码还在识别
Retrying:  1400009488 Count:  3
{'cid': 1400009488, 'method': 'result'}
{'cid': 1400009488, 'ret': -3002, 'text': ''}
云打码验证码还在识别
Retrying:  1400009488 Count:  4
{'cid': 1400009488, 'method': 'result'}
{'cid': 1400009488, 'ret': -3002, 'text': ''}
云打码验证码还在识别
Retrying:  1400009488 Count:  5
{'cid': 1400009488, 'method': 'result'}
{'cid': 1400009488, 'ret': 0, 'text': '4uh4h'}
登录成功
[{'expiry': 1525092087.375918, 'secure': False, 'value': '0cS3YKcgWrDRsV', 'httpOnly': False, 'name': 'SUHB', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1525092087.374765, 'secure': False, 'value': '0033WrSXqPxfM725Ws9jqgMF55529P9D9W5H8du8O9J-.X.2Zd-6LbEK5JpX5o2p5NHD95QfS0ecSKe7e0n4Ws4Dqcj.i--fiK.7iKn4i--ci-z7i-zRi--fi-2fiK.ci--4i-2pi-i2', 'httpOnly': False, 'name': 'SUBP', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1525092087.373734, 'secure': False, 'value': '_2A250AasnDeThGeNJ6FQU8y7PwzWIHXVXDTVvrDV6PUJbktBeLRfukW0wFLR2922NnApkDOZb_eNrRfNBZQ..', 'httpOnly': True, 'name': 'SUB', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1496148087.677672, 'secure': False, 'value': 'b2d2d84c5aafcc02c0dac2faa4acc43f', 'httpOnly': True, 'name': '_T_WM', 'path': '/', 'domain': '.weibo.cn'}, {'secure': False, 'value': '1493556087', 'httpOnly': False, 'name': 'SSOLoginState', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1808916087.372439, 'secure': False, 'value': 'AnPRL-vKWx_-0LWAO4Mk79GSyD6FxNqsFzhSNFOZI1MxKni8Ok7C3thzDOuIdEdh-0SvYb25zbF-znxYY_Z5hK4.', 'httpOnly': True, 'name': 'SCF', 'path': '/', 'domain': '.weibo.cn'}, {'expiry': 1496148083.263736, 'secure': False, 'value': '1496148083', 'httpOnly': False, 'name': 'ALF', 'path': '/', 'domain': '.weibo.cn'}]
{'SUHB': '0cS3YKcgWrDRsV', 'SUB': '_2A250AasnDeThGeNJ6FQU8y7PwzWIHXVXDTVvrDV6PUJbktBeLRfukW0wFLR2922NnApkDOZb_eNrRfNBZQ..', 'SUBP': '0033WrSXqPxfM725Ws9jqgMF55529P9D9W5H8du8O9J-.X.2Zd-6LbEK5JpX5o2p5NHD95QfS0ecSKe7e0n4Ws4Dqcj.i--fiK.7iKn4i--ci-z7i-zRi--fi-2fiK.ci--4i-2pi-i2', 'SCF': 'AnPRL-vKWx_-0LWAO4Mk79GSyD6FxNqsFzhSNFOZI1MxKni8Ok7C3thzDOuIdEdh-0SvYb25zbF-znxYY_Z5hK4.', 'SSOLoginState': '1493556087', '_T_WM': 'b2d2d84c5aafcc02c0dac2faa4acc43f', 'ALF': '1496148083'}
成功获取到Cookies
Saving Cookies to Redis [email protected] {"SUHB": "0cS3YKcgWrDRsV", "SUB": "_2A250AasnDeThGeNJ6FQU8y7PwzWIHXVXDTVvrDV6PUJbktBeLRfukW0wFLR2922NnApkDOZb_eNrRfNBZQ..", "SUBP": "0033WrSXqPxfM725Ws9jqgMF55529P9D9W5H8du8O9J-.X.2Zd-6LbEK5JpX5o2p5NHD95QfS0ecSKe7e0n4Ws4Dqcj.i--fiK.7iKn4i--ci-z7i-zRi--fi-2fiK.ci--4i-2pi-i2", "SCF": "AnPRL-vKWx_-0LWAO4Mk79GSyD6FxNqsFzhSNFOZI1MxKni8Ok7C3thzDOuIdEdh-0SvYb25zbF-znxYY_Z5hK4.", "SSOLoginState": "1493556087", "_T_WM": "b2d2d84c5aafcc02c0dac2faa4acc43f", "ALF": "1496148083"}

cookiespool's Issues

求助

你好，我看到您的cookies池的教程
但是运行run.py时却报错了
错误是：('Error 10061 connecting to :6379. 由于目标计算机积极拒绝，无法连接。.',)
()
希望您能告诉我是哪里出了问题
再次感谢您的python教程

运行run.py 报错Error - Unable to load Atom 'find_element' from file ':/ghostdriver/./third_party/webdriver-atoms/find_element.js

Error - Unable to load Atom 'find_element' from file ':/ghostdriver/./third_party/webdriver-atoms/find_element.js

这个错误是什么原因？

db里面get方法用json.loads更好么？

db里有个get方法取出cookies。可以对照着set方法使用json.loads。结果和decode方法有时候略有不同。
def get(self, key): try: return json.loads(self._db.get(self._key(key))) except: return None

关于Internal Server Error

博主您好，我看你的教程使用该cookiesPool代理池时浏览器的http://127.0.0.1:5000/weibo/random

无法获取random cookies 网页显示的错误是
Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

在api.py中的if name =='main'中按照stackoverflow
https://stackoverflow.com/questions/10219486/flask-post-request-is-causing-server-to-crash

加入app.debug=True 也无法调式获得错误 ，更换不同浏览器也出现同样的错误

运行python3 run.py报错

clone到本地然后运行python3 run.py报错如下，不知道要怎么解决？

E:\python\Pycharm\CookiesPool>python3 run.py
Traceback (most recent call last):
  File "run.py", line 8, in <module>
    main()
  File "run.py", line 5, in main
    s.run()
  File "E:\python\Pycharm\CookiesPool\cookiespool\scheduler.py", line 46, in run
    generate_process.start()
  File "C:\Python34\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Python34\lib\multiprocessing\context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Python34\lib\multiprocessing\context.py", line 313, in _Popen
    return Popen(process_obj)
  File "C:\Python34\lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Python34\lib\multiprocessing\reduction.py", line 59, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function Scheduler.generate_cookie at 0x02A03DF8>: attribute lookup generate_co
okie on cookiespool.scheduler failed

E:\python\Pycharm\CookiesPool>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python34\lib\multiprocessing\spawn.py", line 106, in spawn_main
    exitcode = _main(fd)
  File "C:\Python34\lib\multiprocessing\spawn.py", line 116, in _main
    self = pickle.load(from_parent)
EOFError: Ran out of input

求助

小弟在微博搜索验证码自动识别的一个小例子里借用了大哥的代码。可是response = requests.get(url, cookies=cookies_dict)获取验证码这里，是附带cookie获取的，云打码也识别了。点击提交的时候总是提示验证码不对。为什么获取的验证码和当前用户关联不上呢。希望大哥给点儿思路

求助

我是新手，请问在importer中输入的账号和密码组，如何获得