网际风数据源 about datafeed

[yinhm]

出现此错误的原因是您还没有任何数据。您可以 put_report 提交数据之后再行测试。

用例方面可以参考 tests，testcase还是很全的。

网际风的话，直接用 providers 中的 tongshi.py 就可以，需要一个 config.py 指定服务器地址。

数据存盘是自动的，网际风会在一定时间自动收盘，网际风收盘不够迅速，所以生产环境我的方法是退出 tongshi.py ，做一个计划任务，在收盘后一定时间内再次启动 tongshi.py ，如此强制网际风收盘。

[dindom999]

哗，没想到回复这么神速！
你说的Config是指这个吗：

“# Server address
”# Used for WJF client.
SERVER_ADDR = '10.0.0.2'

“# Set password to enable authtication
”# Default to None, no auth needed.
“# AUTH_PASSWORD = 'YourPassword'
AUTH_PASSWORD = None

用户名不需要吗？

其实我主要是没搞明白它的结构，或者是角色
如你的例子：
import config
from datafeed.providers.tongshi import run_tongshi_win
if name=='main':
run_tongshi_win(config.SERVER_ADDR, config.AUTH_PASSWORD)
这样运行之后，结果是等于启动了网际风的接口吗？等于它已经在接收推送的数据吗？
那怎么取这些数据呢？

[yinhm]

datafeed server 是单用户系统，不存在用户名。

您这样启动后，会启动网际风 client， 接收网际风数据，并 push 到 server_addr 10.0.0.2，也就是说 datafeed server的 IP 地址为 10.0.0.2。

网际风需要跑在 widnows上，而 datafeed 通常是部署在 linux 机器的。

[dindom999]

我现在有点头绪了，Datafeed Server是网际风的下游，对吧？
那么这个Config.py的账号是属于Datafeed的，而不是网际风的，对吗？那网际风的账号设置在哪里呢？

[yinhm]

不是这样的。

首先 datafeed server是独立服务器，网际风这些不过是数据源，跟服务器不相干的，你完全可以换成 Wind, iFind而不影响 datafeed server。

其实 config文件服务器用了 auth pass 而 wjf 会用 server_addr 以及 auth pass。

网际风账号在网际风自己界面设定。

[dindom999]

嗯，我说的下游的意思，就是数据源这个意思，这个结构我现在明白了，我不明白的是，网际风物理上指的不就是Stock.dll吗？
还有一个问题，既然是两层，那么数据存放是不是也各自保存，datafeed server保存一份，网际网保存一份？

[yinhm]

网际风自己还有一个 Main.exe 控制登陆，接受何种消息等。

网际风本身不存在数据，网际风客户端是从服务器接受数据后自行处理，本例中保存到我们自建服务器 datafeed server。

[dindom999]

好的，太感谢你了！！！我再看看代码整理一下思路

[dindom999]

数据终于对接上了，不过通过网际风补历史5分钟线的时候，Server.py 和wjf.py都出现了大量错误，不知道是什么原因造成的。
server.py的错误摘录：
ds[:] = sliced_qs
File "h5py_objects.pyx", line 54, in h5py.objects.with_phil.wrapper (D:
Build\h5py\h5py-2.5.x\h5py_objects.c:2413)
File "h5py_objects.pyx", line 55, in h5py.objects.with_phil.wrapper (D:
Build\h5py\h5py-2.5.x\h5py_objects.c:2370)
File "e:\Anaconda\lib\site-packages\h5py_hl\dataset.py", line 585, in __
etitem
for fspace in selection.broadcast(mshape):
File "e:\Anaconda\lib\site-packages\h5py_hl\selections.py", line 304, in
broadcast
raise TypeError("Can't broadcast %s -> %s" % (target_shape, count))
TypeError: Can't broadcast (19,) -> (48,)
[E 150526 19:17:00 iostream:540] Uncaught exception, closing connection.
Traceback (most recent call last):
File "e:\Anaconda\lib\site-packages\tornado\iostream.py", line 537, in wr
pper
return callback(_args)
File "e:\Anaconda\lib\site-packages\tornado\stack_context.py", line 275,
n null_wrapper
return fn(_args, *_kwargs)
File "D:\datafeed\datafeed\server.py", line 245, in _on_argument_data
self.request_callback(self._request)
File "D:\datafeed\datafeed\server.py", line 306, in call
handler._execute()
File "D:\datafeed\datafeed\server.py", line 607, in _execute
getattr(self, self.request.method)(_arguments)
File "D:\datafeed\datafeed\server.py", line 556, in put_5minute
self.dbm.fiveminstore.update(symbol, np.load(StringIO(data)))
File "D:\datafeed\datafeed\datastore.py", line 433, in update
self._update(symbol, quotes)
File "D:\datafeed\datafeed\datastore.py", line 465, in _update
ds[:] = sliced_qs
File "h5py_objects.pyx", line 54, in h5py.objects.with_phil.wrapper (D:
Build\h5py\h5py-2.5.x\h5py_objects.c:2413)
File "h5py_objects.pyx", line 55, in h5py.objects.with_phil.wrapper (D:
Build\h5py\h5py-2.5.x\h5py_objects.c:2370)
File "e:\Anaconda\lib\site-packages\h5py_hl\dataset.py", line 585, in __
etitem
for fspace in selection.broadcast(mshape):
File "e:\Anaconda\lib\site-packages\h5py_hl\selections.py", line 304, in
broadcast
raise TypeError("Can't broadcast %s -> %s" % (target_shape, count))
TypeError: Can't broadcast (8,) -> (48,)
[E 150526 19:17:00 ioloop:588] Exception in callback <functools.partial object
t 0x04D68C00>
Traceback (most recent call last):
File "e:\Anaconda\lib\site-packages\tornado\ioloop.py", line 568, in _run
callback
ret = callback()
File "e:\Anaconda\lib\site-packages\tornado\stack_context.py", line 275,
n null_wrapper
return fn(_args, *_kwargs)
File "e:\Anaconda\lib\site-packages\tornado\iostream.py", line 537, in wr
pper
return callback(_args)
File "e:\Anaconda\lib\site-packages\tornado\stack_context.py", line 275,
n null_wrapper
return fn(_args, **kwargs)
File "D:\datafeed\datafeed\server.py", line 245, in _on_argument_data
self.request_callback(self._request)
File "D:\datafeed\datafeed\server.py", line 306, in call
handler._execute()
File "D:\datafeed\datafeed\server.py", line 607, in _execute
getattr(self, self.request.method)(*arguments)
File "D:\datafeed\datafeed\server.py", line 556, in put_5minute
self.dbm.fiveminstore.update(symbol, np.load(StringIO(data)))
File "D:\datafeed\datafeed\datastore.py", line 433, in update
self._update(symbol, quotes)
File "D:\datafeed\datafeed\datastore.py", line 465, in _update
ds[:] = sliced_qs
File "h5py_objects.pyx", line 54, in h5py.objects.with_phil.wrapper (D:
Build\h5py\h5py-2.5.x\h5py_objects.c:2413)
File "h5py_objects.pyx", line 55, in h5py.objects.with_phil.wrapper (D:
Build\h5py\h5py-2.5.x\h5py_objects.c:2370)
File "e:\Anaconda\lib\site-packages\h5py_hl\dataset.py", line 585, in __
etitem
for fspace in selection.broadcast(mshape):
File "e:\Anaconda\lib\site-packages\h5py_hl\selections.py", line 304, in
broadcast
raise TypeError("Can't broadcast %s -> %s" % (target_shape, count))
TypeError: Can't broadcast (8,) -> (48,)

[dindom999]

Wjf.py的错误摘录：其中Unknown file data.也出现在部分的日线数据上
File "D:\datafeed\datafeed\client.py", line 118, in execute_command
return self._execute_command(args[0], args[-1], self._build_data(_args))
File "D:\datafeed\datafeed\client.py", line 126, in _execute_command
return self._parse_response(command, format)
File "D:\datafeed\datafeed\client.py", line 143, in _parse_response
raise StandardError("Socket closed on remote end")
StandardError: Socket closed on remote end
Python WNDPROC handler failed
Traceback (most recent call last):
File "D:\datafeed\datafeed\providers\tongshi.py", line 418, in _on_data_receiv
e
self.client.put_5minute(key, rec)
File "D:\datafeed\datafeed\client.py", line 252, in put_5minute
return self.execute_command('PUT_5MINUTE', symbol, memfile.getvalue(), 'npy'
)
File "D:\datafeed\datafeed\client.py", line 118, in execute_command
return self._execute_command(args[0], args[-1], self._build_data(_args))
File "D:\datafeed\datafeed\client.py", line 126, in _execute_command
return self._parse_response(command, format)
File "D:\datafeed\datafeed\client.py", line 143, in _parse_response
raise StandardError("Socket closed on remote end")
StandardError: Socket closed on remote end
Python WNDPROC handler failed
Traceback (most recent call last):
File "D:\datafeed\datafeed\providers\tongshi.py", line 418, in _on_data_receiv
e
self.client.put_5minute(key, rec)
File "D:\datafeed\datafeed\client.py", line 252, in put_5minute
return self.execute_command('PUT_5MINUTE', symbol, memfile.getvalue(), 'npy'
)
File "D:\datafeed\datafeed\client.py", line 118, in execute_command
return self._execute_command(args[0], args[-1], self._build_data(_args))
File "D:\datafeed\datafeed\client.py", line 126, in _execute_command
return self._parse_response(command, format)
File "D:\datafeed\datafeed\client.py", line 143, in _parse_response
raise StandardError("Socket closed on remote end")
StandardError: Socket closed on remote end
Unknown file data.
Unknown file data.
Python WNDPROC handler failed
Traceback (most recent call last):
File "D:\datafeed\datafeed\providers\tongshi.py", line 418, in _on_data_receiv
e
self.client.put_5minute(key, rec)
File "D:\datafeed\datafeed\client.py", line 252, in put_5minute
return self.execute_command('PUT_5MINUTE', symbol, memfile.getvalue(), 'npy'
)
File "D:\datafeed\datafeed\client.py", line 118, in execute_command
return self._execute_command(args[0], args[-1], self._build_data(_args))
File "D:\datafeed\datafeed\client.py", line 126, in _execute_command
return self._parse_response(command, format)
File "D:\datafeed\datafeed\client.py", line 143, in _parse_response
raise StandardError("Socket closed on remote end")
StandardError: Socket closed on remote end
Python WNDPROC handler failed
Traceback (most recent call last):
File "D:\datafeed\datafeed\providers\tongshi.py", line 418, in _on_data_receiv
e
self.client.put_5minute(key, rec)
File "D:\datafeed\datafeed\client.py", line 252, in put_5minute
return self.execute_command('PUT_5MINUTE', symbol, memfile.getvalue(), 'npy'
)
File "D:\datafeed\datafeed\client.py", line 118, in execute_command
return self._execute_command(args[0], args[-1], self._build_data(*args))
File "D:\datafeed\datafeed\client.py", line 126, in _execute_command
return self._parse_response(command, format)
File "D:\datafeed\datafeed\client.py", line 143, in _parse_response
raise StandardError("Socket closed on remote end")
StandardError: Socket closed on remote end
Unknown file data.

[dindom999]

分析了一下源码，问题似乎出在保存数据那里
File "D:\datafeed\datafeed\datastore.py", line 465, in _update
ds[:] = sliced_qs

但代码应该没问题，是不是因为数据量太大，电脑处理不过来呢，我用的是笔记本CPU T7200、2G内存

[yinhm]

不是数据量的问题，datafeed处理这点数据绰绰有余。

这里的问题是存储时 broadcast 出错，意思是创建的 datasets shape 和将要存储的数据 shape 不同。出现这个问题可能是网际风数据错误，也可能是你本地的数据文件有问题。

[dindom999]

果然是数据源的问题，换了个银江的接口就正常了，终于搞定了！！！周末再弄个画K线的模块，就能将原来股软里的公式移植过来做成自动化交易系统里，再次谢谢你的帮助和贡献！

另外，有好用的K线模块推荐吗？

[yinhm]

帮人帮到底，送佛送到西。K线施主可以用 ChartDirector。

贴一个我以前写的 demo，你需要修改才能用。具体不详述。

#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Copyright 2011 yinhm
import time
import platform
import sys


if platform.system() == 'Darwin':
    sys.path[0:0] = ['mac']
elif platform.system() == 'linux':
    sys.path[0:0] = ['linux']

from datafeed.client import Client
from datafeed.dividend import adjust
from FinanceChartZh import *

from pychartdir import *


symbol = 'SH000001'
client = Client()


y = client.get_day(symbol, 150)
try:
    dividends = client.get_dividend(symbol)
except:
    dividends = []

y = adjust(y, dividends)
#if len(y) > 250:
#    y = y.asfreq('W@FRI', method='pad')

index_ts = [chartTime(dt.year, dt.month, dt.day) for dt in y.index]

# Create a FinanceChart object of width 640 pixels
ylen = 960
c = FinanceChart(ylen)
c.addTitle("图例", "msyhbd.ttf", 12)
extraDays = 30

# Set the data into the finance chart object
c.setData(
    index_ts,
    y['high'].values,
    y['low'].values,
    y['open'].values,
    y['close'].values,
    y['volume'].values,
    extraDays)

# Add the main chart with 240 pixels in height
main_chart = c.addMainChart(ylen * 0.5)

# Add a 5 period simple moving average to the main chart, using brown color
# c.addSimpleMovingAvg(5, 0x663300)

# Add a 20 period simple moving average to the main chart, using purple color
c.addSimpleMovingAvg(20, 0x9900ff)

# Add HLOC symbols to the main chart, using green/red for up/down days
# c.addHLOC('0x008000', '0xcc0000')
c.addCandleStick(0xff0000, 0x0033ff33)

# Add 20 days bollinger band to the main chart, using light blue (9999ff) as the
# border and semi-transparent blue (c06666ff) as the fill color
# c.addBollingerBand(20, 2, 0x9999ff, 0xc06666ff)

# Add a 75 pixels volume bars sub-chart to the bottom of the main chart, using
# green/red/grey for up/down/flat days
c.addVolIndicator(75, 0x99ff99, 0xff9999, 0x808080)

# Append a 14-days RSI indicator chart (75 pixels high) after the main chart. The
# main RSI line is purple (800080). Set threshold region to +/- 20 (that is, RSI = 50
# +/- 25). The upper/lower threshold regions will be filled with red (ff0000)/blue
# (0000ff).
# c.addRSI(75, 14, 0x800080, 20, 0xff0000, 0x0000ff)

# Append a 12-days momentum indicator chart (75 pixels high) using blue (0000ff)
# color.
# c.addMomentum(75, 12, 0x0000ff)

#your scatter layer data, initialized to NoValue
signal_data = [NoValue] * len(index_ts)
#put a symbol at the closing price position of the 21st session
signal_data[-10] = y['low'].values[-10]
#add the symbol to the main price chart
main_chart.addScatterLayer(None, signal_data, "买点", TriangleSymbol, 13, 0x9900ff)


filename = '/tmp/chart-%s.png' % (symbol,)
# Output the chart
c.makeChart(filename)

[dindom999]

wowowo!有机会去广州请你吃饭，哈哈

[yinhm]

@dindom999

换了个银江的接口就正常了

可否说一下银江的数据接口从哪里买的，回头我整理一下文档添加进去，也让后来人方便一些。

[dindom999]

淘宝买的，他们也有官网http://www.000562.com/，速度不快，但看上去比较稳定。