GithubHelp home page GithubHelp logo

graysilver / wencai Goto Github PK

View Code? Open in Web Editor NEW
273.0 17.0 119.0 181 KB

This is a wencai crawler.(i问财的策略回测接口的Pythonic工具包)

License: MIT License

Python 38.97% JavaScript 61.03%
crawler wencai finance pandas quant quantitative-finance tushare

wencai's People

Contributors

graysilver avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wencai's Issues

返回的数据和直接搜索的数据不一致。

strategy = wc.get_strategy("上升通道")
print(strategy)

返回

 日期    股票代码  股票简称   涨跌幅  当日收盘价(元)  dde大单净量(%)  股本规模   换手率
0  2018-11-06  000002   万科A -2.36     24.04       -0.10  超大盘股  0.60
1  2018-11-06  000008  神州高铁  3.43      4.52       -0.04   中盘股  2.03

和问财主页上返回的结果不一致

滑动验证处理

请问各位大哥目前的验证码机制可以如何破解呢,是否有相关的项目可以参考呢,谢谢

hexin.js找不到

FileNotFoundError: [Errno 2] No such file or directory: 'AppData\Local\Programs\Python\Python37-32\lib\site-packages\wencai/js/hexin.js'

wc.search(query='当前热股') 出错

result = wc.search(query='当前热股')

File "D:\wencai\core\event.py", line 48, in search
return Wencai(proxies=globals()['global_proxies'], verify=globals()['verify']).search(query_string=query)
File "D:\wencai\core\crawler.py", line 102, in search
result = r.json()['data']['answer'][0]['txt'][0]['content']['components'][0]['data']['datas']
TypeError: list indices must be integers or slices, not str

# wc.get_strategy(策略回测) 运行结果不显示 #历史明细

print(r.profit_data) # 累计收益数据
print(r.backtest_data) # 报告评级
print(r.condition_data) # 准确回测语句
print(r.history_detail(period='2')) # 历史明细查询
print(r.history_pick(trade_date='2020-11-19', hold_num=1)) # 策略选股

history_detail 这一句输出的dataframe为空是怎么回事?

Empty DataFrame
Columns: []
Index: []

ws.search不能用了吗? requests.exceptions.JSONDecodeError

Traceback (most recent call last):
File "D:\develope\IdeaProjects\PythonHelloword.idea\src\WenCaiAPI.py", line 192, in
str =wc.search('今日热股')
File "D:\develope\IdeaProjects\python3.9_space\lib\site-packages\wencai\core\event.py", line 48, in search
return Wencai(proxies=globals()['global_proxies'], verify=globals()['verify']).search(query_string=query)
File "D:\develope\IdeaProjects\python3.9_space\lib\site-packages\wencai\core\crawler.py", line 102, in search
result = r.json()['data']['answer'][0]['txt'][0]['content']['components'][0]['data']['datas']
File "D:\develope\IdeaProjects\python3.9_space\lib\site-packages\requests\models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

这个是被反爬限制了吗?

问财搜索条件是:当前热股 的时候啥都没有

www.iwencai.com
搜索条件: 当前热股

然后啥都没有返回啊 谢谢大牛牛指教

r = wc.get_strategy(query='当前热股',
start_date='2018-10-09',
end_date='2019-07-16',
period='1',
fall_income=1,
lower_income=5,
upper_income=9,
day_buy_stock_num=1,
stock_hold=2)
print(r.profit_data) # 累计收益数据
print(r.backtest_data) # 报告评级
print(r.condition_data) # 准确回测语句
print(r.history_detail(period='1')) # 历史明细查询
print(r.history_pick(trade_date='2019-07-16', hold_num=1)) # 策略选股

问财有验证码机制了

爬一段时间后,会需要你再填一次验证码才能继续爬,我登陆网页输入验证码后,接着用本地的cookies可以接着爬取,这个作者能维护一下吗?

search功能能否完善

目前如使用wc.search(“找出今天的涨停板”) 只会返回50条数据,能否完善返回全部的数据呢。谢谢!

search报缺少必要参数

{'status_code': -302, 'status_msg': '缺少必要参数', 'data': [], 'logid': '7795b8d93a43ba01234686dc6edbf071'}

connection or something of lack

D:.virtualenvs\study\lib\site-packages\urllib3\connectionpool.py:981: InsecureRequestWarning: Unverified HTTPS request is being made to host 'search.10jqka.com.cn'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
warnings.warn(
Traceback (most recent call last):
File "d:\sourcecodes\projects\seekentries.py", line 3, in
result = wc.search(query="当前热股")
File "D:.virtualenvs\study\lib\site-packages\wencai\core\event.py", line 48, in search
return Wencai(proxies=globals()['global_proxies'], verify=globals()['verify']).search(query_string=query)
File "D:.virtualenvs\study\lib\site-packages\wencai\core\crawler.py", line 102, in search
result = r.json()['data']['answer'][0]['txt'][0]['content']['components'][0]['data']['datas']
TypeError: list indices must be integers or slices, not str

wc.seach接口报错

wc.search接口报错,好像用不了了:
`Traceback (most recent call last):
File "C:\Users\Administrator\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 700, in urlopen
self._prepare_proxy(conn)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 994, in _prepare_proxy
conn.connect()
File "C:\Users\Administrator\Anaconda3\lib\site-packages\urllib3\connection.py", line 364, in connect
self.sock = conn = self._connect_tls_proxy(hostname, conn)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\urllib3\connection.py", line 505, in connect_tls_proxy
ssl_context=ssl_context,
File "C:\Users\Administrator\Anaconda3\lib\site-packages\urllib3\util\ssl
.py", line 453, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl(sock, context, tls_in_tls)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\urllib3\util\ssl
.py", line 495, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock)
File "C:\Users\Administrator\Anaconda3\lib\ssl.py", line 412, in wrap_socket
session=session
File "C:\Users\Administrator\Anaconda3\lib\ssl.py", line 850, in _create
self.do_handshake()
File "C:\Users\Administrator\Anaconda3\lib\ssl.py", line 1108, in do_handshake
self._sslobj.do_handshake()
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\Administrator\Anaconda3\lib\site-packages\requests\adapters.py", line 499, in send
timeout=timeout,
File "C:\Users\Administrator\Anaconda3\lib\site-packages\urllib3\connectionpool.py", line 786, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "C:\Users\Administrator\Anaconda3\lib\site-packages\urllib3\util\retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='search.10jqka.com.cn', port=443): Max retries exceeded with url: /unifiedwap/unified-wap/v2/result/get-robot-data (Caused by ProxyError('Cannot connect to proxy.', FileNotFoundError(2, 'No such file or directory')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Administrator\Anaconda3\lib\site-packages\wencai\core\event.py", line 48, in search
return Wencai(proxies=globals()['global_proxies'], verify=globals()['verify']).search(query_string=query)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\wencai\core\crawler.py", line 101, in search
data=payload, force_cookies=True)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\wencai\core\session.py", line 51, in post_result
return super(Session, self).post(url=url, data=data, json=json, **kwargs)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\requests\sessions.py", line 635, in post
return self.request("POST", url, data=data, json=json, **kwargs)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\requests\sessions.py", line 723, in send
history = [resp for resp in gen]
File "C:\Users\Administrator\Anaconda3\lib\site-packages\requests\sessions.py", line 723, in
history = [resp for resp in gen]
File "C:\Users\Administrator\Anaconda3\lib\site-packages\requests\sessions.py", line 274, in resolve_redirects
**adapter_kwargs,
File "C:\Users\Administrator\Anaconda3\lib\site-packages\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Administrator\Anaconda3\lib\site-packages\requests\adapters.py", line 559, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='search.10jqka.com.cn', port=443): Max retries exceeded with url: /unifiedwap/unified-wap/v2/result/get-robot-data (Caused by ProxyError('Cannot connect to proxy.', FileNotFoundError(2, 'No such file or directory')))`

recommend: 希望有一个单一功能的查询接口

r = wc.get_strategy() 会返回一个策略选股表,但其中无法查询所属行业、概念、首次涨停时间、最终涨停时间等信息。
建议有如下查询功能:

wc.query(w= r"上一交易日没有涨停;市值大于100亿;今日涨停时间", 
fields=["涨停时间",“总市值”])-> pd.DataFrame,   

search功能运行json报错

多次循环运行search功能出现json报错:
JSONDecodeError: Extra data: line 1 column 1 (char 0)

望大佬解决

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.