lifego / abquant-data Goto Github PK

# -*- mode:org; org-confirm-babel-evaluate: nil -*-
#+TITLE: abquant
#+AUTHOR: Jimmy M. Gong
#+EMAIL: [email protected]
#+LANGUAGE: zh-Hans
#+OPTIONS: H:3 num:nil toc:nil \n:t ::t |:t ^:nil -:nil f:t *:t <:t html-postamble:nil html-preamble:t tex:t
#+URI: /blog/%y/%m/%d/
#+DATE: 2020-01-01
#+LAYOUT: post
#+TAGS: OFFICE(o) COMPUTER(c) HOME(h) PROJECT(p) READING(r)
#+CATEGORIES:
#+DESCRIPTON: A&B professional platform for quantitative finance. God is asking & bidding Me. ABQ. GABQ. GABM.
#+KEYWORDS: quant c++
#+STARTUP: overview
#+STARTUP: logdone

* Description
A&B professional platform for quantitative finance. God is asking & bidding Me.
ABQ.

abquant 分为三大模块 a) 数据获取 b) 策略回测 c) 模拟/实盘交易
- abquant-data ::
  开源项目, 快速对 *通达信* 数据的获取，除权除息，各种指数的计算
- abquant-strategy ::
  闭源项目，基于 zeromg/nanomsg
- abquant-trade ::
  [[https://github.com/yssource/rqalpha-mod-xtrade][xtrade]]
  闭源项目
* Note
  [WIP]
  开发中 ...

  ABQuant is an early developer preview, and is not suitable for general usage yet. Features and implementation are subject to change.
* 构建
  作者本人是在 x86_64 Linux 4.19.105-1-MANJARO 上构建本项目，暂时不支持
  Windows. \\
  你需要安装好 clang++/llvm ccache ninja python pip poetry conan cmake/qmake 等基础开发工具
  - 核心模块 cpp
  - 中间层 binding ([[https://github.com/pybind/pybind11.git][pybind11]] [[https://doc.qt.io/qtforpython/shiboken2/][ +shiboken2+ ]])
  - 用户接口层 python
** cpp 依赖
   - ORM 层 [[https://github.com/yssource/treefrog-framework.git][treefrog-framework]] ::
#+name: treefrog-framework
#+begin_src shell :exports code
  git clone -b feature/withConan https://github.com/yssource/treefrog-framework.git
  cd treefrog-framework && mkdir -v qbuild && cd qbuild
  conan install .. --build missing --profile ../tf.profile
  conan build ..
  sudo make install
  sudo ldconfig
#+end_src
   - abquant 核心应用层 ::
#+name: abquant-building
#+begin_src shell :exports code
  git clone https://github.com/yssource/abquant-data.git
  cd abquant-data && mkdir -v qbuild && cd qbuild
  conan install .. --build missing --profile ../abquant.profile
  conan build ..
  sudo make install
  sudo ldconfig

  # binding
  cd abquant-data && mkdir -v cbuild && cd cbuild
  conan install .. --build missing --profile ../abquant.profile
  conan build ..

  # todo:: qbuild, cbuild 将来会直接放在 setup.py 中作为 ext_modules 完成。
  # 用户只需 poetry install
#+end_src
** python 依赖安装
   #+begin_src python :exports code
     poetry install
     poetry update
   #+end_src
** 环境变量
   请参照 .envrc 文件, 推荐使用 direnv 管理环境变量
   #+begin_src shell :exports code
     export ABQ_HOME=${PWD}
     export ABQ_CXX_HOME="${ABQ_HOME}/cxx"
     export LD_LIBRARY_PATH="${ABQ_HOME}/lib:${LD_LIBRARY_PATH}"
     export PYTHONPATH="${ABQ_HOME}/lib:${PYTHONPATH}"
   #+end_src
* Usage
** abquant config
     #+name: config
     #+begin_src shell :exports code
       # 当前项目根目录下，拷贝 config 文件夹
       mkdir -pv ~/.config/abquant && cp -rfv $PWD/cxx/config ~/.config/abquant/
     #+end_src
** abquant 数据获取
   - python 层获取数据源，支持 mongodb
     #+name: data-saving
     #+begin_src shell :exports code
       # 保存所有数据
       # 但是不包含除权数据，除权数据不是经常更新，请手动 "abquant stock xdxr"
       abquant save base

       # 股票和指数列表获取
       abquant stock list

       # 保存所有股票日线，日线指数
       abquant stock day
       abquant stock day --add '["000001", "300001", "600001"]'

       # 保存所有股票分钟线，分钟指数
       # Default 保存所有频率 (1min, 5min, 15min, 30min, 60min)
       abquant stock min
       abquant stock min --add '["000001", "300001", "600001"]'

       # 保存股票分钟线，分钟指数不同频率数据 (1min, 5min, 15min, 30min, 60min)
       # 1min 一分钟
       abquant stock min "1min"
       abquant stock min -f '["1min"]' --add '["000001", "300001", "600001"]'
       # 5min 五分钟
       abquant stock min -f '["5min"]'
       abquant stock min -f '["5min"]' --add '["000001", "300001", "600001"]'
       # ...
       # 1min 一分钟和 5min 五分钟
       abquant stock min -f '["1min", "5min"]'
       abquant stock min -f '["1min", "5min"]' --add '["000001", "300001", "600001"]'

       # 股票除权除息
       abquant stock xdxr
       abquant stock xdxr --add '["000001", "300001", "600001"]'

       # 股票数据板块
       abquant stock block

       # 股票数据明细信息
       abquant stock info
       abquant stock info --add '["000001", "300001", "600001"]'

       # 股票财务数据
       abquant stock financial

       # ETF 列表获取
       abquant etf list

       # 保存所有 ETF 日线，日线指数
       abquant etf day
       abquant etf day --add '["159001", "510010", "510300"]'

       # 保存所有 etf 分钟线，分钟指数
       # Default 保存所有频率 (1min, 5min, 15min, 30min, 60min)
       abquant etf min
       abquant etf min --add '["159001", "510010", "510300"]'

       # 保存 etf 分钟线，分钟指数不同频率数据 (1min, 5min, 15min, 30min, 60min)
       # 1min 一分钟
       abquant etf min "1min"
       abquant etf min -f '["1min"]' --add '["159001", "510010", "510300"]'
       # 5min 五分钟
       abquant etf min -f '["5min"]'
       abquant etf min -f '["5min"]' --add '["159001", "510010", "510300"]'
       # ...
       # 1min 一分钟和 5min 五分钟
       abquant etf min -f '["1min", "5min"]'
       abquant etf min -f '["1min", "5min"]' --add '["159001", "510010", "510300"]'

       # ...
     #+end_src

     #+name: error-data-saving
     #+begin_src shell :results output :exports both
       # 查看保存数据后的错误信息
       bat ~/.config/abquant/log/error_codes.json
     #+end_src

     #+RESULTS: error-data-saving
     : {"Stock_index_day": ["395005"], "Stock_index_min": ["395041", "395005"], "Stock_stock_min": ["300822", "603949", "603353", "300819", "300821", "002976", "688051"], "Stock_xdxr": ["002976", "603949", "300822", "688051", "603353"]}

     #+name: error-data-saving
     #+begin_src shell :results output :exports both
       # 如果在保存数据后，发现数据错误，可以使用
       abquant stock day --add '["395005"]'
       abquant stock min --add '["395041", "395005", "300822", "603949", "603353", "300819", "300821", "002976", "688051"]'
       abquant stock xdxr --add '["002976", "603949", "300822", "688051", "603353"]'

       # 注意：这些数据错误可能是退市，已经不存在的数据。也有可能是网络原因，数据不完整，你需要具体分析。
       # 在下一次保存数据时，最好先
       echo "" 1> ~/.config/abquant/log/error_codes.json
     #+end_src
** abquant 数据使用
   #+name: cpp-data-using
   #+begin_src C++ :exports code
     #include <chrono>

     #include "abquant/actions/abquant.hpp"
     #include "abquant/actions/stockmin.hpp"
     #include "abquant/actions/utils.hpp"

     using namespace abq;

     int main(int argc, char* argv[])
     {
         // abq entry_point
         Abquant::start();

         // QStringList codes = {"000001", "000002", "000003"};
         QStringList codes = {"000001"};
         const char* start = "2017-01-01";
         const char* end   = "2019-12-01";

         MIN_FREQ freq = MIN_FREQ::FIVE;
         StockMinAction sma(codes, start, end, freq);
         int N      = 10;
         auto begin = std::chrono::high_resolution_clock::now();
         for (int i = 0; i < N; ++i) {
             // 分钟前复权
             sma.to_fq_type(FQ_TYPE::PRE);
             // 分钟后复权
             // sma.to_fq_type(FQ_TYPE::POST);
         }
         auto finish_                          = std::chrono::high_resolution_clock::now();
         std::chrono::duration<double> elapsed = finish_ - begin;
         qDebug() << "Elapsed time: " << elapsed.count() << " s\n";

         // abq exit_point
         Abquant::finish();
         return 0;
     }
   #+end_src
** binding 层
   #+name: binding-plot
   #+begin_src shell :exports both
     python ./bind11/test/test_plot.py
   #+end_src

   #+RESULTS: binding-plot
   #+begin_example
                         open  close   high    low        vol        amount    date_stamp
     code   date
     000001 2019-01-02   9.39   9.19   9.42   9.16   539386.0  4.986951e+08  1.546358e+09
            2019-01-03   9.18   9.28   9.33   9.15   415537.0  3.844577e+08  1.546445e+09
            2019-01-04   9.24   9.75   9.82   9.22  1481159.0  1.422150e+09  1.546531e+09
            2019-01-07   9.84   9.74   9.85   9.63   865687.0  8.411664e+08  1.546790e+09
            2019-01-08   9.73   9.66   9.74   9.62   402388.0  3.892478e+08  1.546877e+09
            2019-01-09   9.74   9.94  10.08   9.70  1233486.0  1.229465e+09  1.546963e+09
            2019-01-10   9.87  10.10  10.20   9.86  1071817.0  1.079711e+09  1.547050e+09
            2019-01-11  10.11  10.20  10.22  10.05   696364.0  7.080018e+08  1.547136e+09
            2019-01-14  10.22  10.11  10.25  10.07   500443.0  5.078629e+08  1.547395e+09
            2019-01-15  10.11  10.24  10.28  10.09   542160.0  5.530273e+08  1.547482e+09
   #+end_example
*** 注意 entry_point && exit_point
    #+name: note_c++
    #+begin_src C++ :exports code
      int main(int argc, char* argv[])
      {
          // abq entry_point
          Abquant::start();
          // Your C++ code here.
          // abq exit_point
          Abquant::finish();
          return 0;
      }
     #+end_src
     #+name: note_python
     #+begin_src python :exports code
       from pyabquant import PyAbquant

       # abq entry_point
       PyAbquant.start()
       # Your python code here.

       code = "000001.XSHE"
       start = "2020-01-01 00:00:00"
       end = "2020-01-01 23:55:00"
       fields = ["open", "close"]
       actual = get_price(code, start, end, frequency="5min", fields=fields, adjust_type="pre")
       ulog.debug(actual)
       assert isinstance(actual, (pd.DataFrame))
       assert actual.empty is True

       # abq exit_point
       PyAbquant.finish()
     #+end_src

*** plotting
   [[file:./screenshot/pyabqstockday.png]]
* DONE Development:
  CLOSED: [2021-01-01 Fri 00:00]
  :LOGBOOK:
  - State "DONE"       from "STARTED"    [2021-01-01 Fri 00:00]
  :END:
  - 兼容 rqalpha apis ::
  1. [X] get_price
  2. [X] get_fundamentals[fn:1]
  - 兼容 jointquant apis ::
  1. [X] get_all_securities
  2. [X] get_security_info
  - 东方财富 概念 apis ::
  1. [X] easymoney concept base get_blocks
  2. [X] easymoney concept history get_price
* WAITING Development:
  :LOGBOOK:
  - State "STARTED"    from "DONE"       [2021-02-13 Sat 00:00]
  :END:
  - 对 Docker, K8s 的支持 ::
* 开发初衷
  作者本人专业是理论物理，工作经验则是程序开发，很偶然在网上了解到了量化交易，觉得可以利用上以前的知识和经验，于是乎喜欢上了量化交易。

交易三部曲，a) 数据获取 b) 策略回测 c) 模拟/实盘交易 \\
市面上开源的 a,b,c 项目大部分都是 python 开发，但是在实践过程中性能都不理想，于是想
到站在巨人肩上利用 C++ 造轮子。核心模块 C++，通过中间层 binding 交给用户 python 层。
* 感谢
- [[https://github.com/QUANTAXIS/QUANTAXIS.git][QUANTAXIS]]
  abquant-data 兼容 quantaxis 数据库
* 打赏
  欢迎请作者喝杯咖啡 👍
  | 微信                               | 支付宝                             |
  |------------------------------------+------------------------------------|
  | [[file:./screenshot/jimmy.wechat.png]] | [[file:./screenshot/jimmy.alipay.png]] |
* LICENSE
Copyright (c) 2020-present Jimmy M. Gong \\
All rights reserved.

除非遵守当前许可，否则不得使用本软件。
** 非商业用途（非商业用途指个人出于非商业目的使用本软件，或者高校、研究所等非营利机构出于教育、科研等目的使用本软件）：
    遵守 [[https://www.tldrlegal.com/l/gpl-3.0][GPL3]] 和 [[https://www.tldrlegal.com/l/lgpl-3.0][LGPL3]]

    除非法律有要求或以书面形式达成协议，否则本软件分发时需保持当前许可“原样”不变，且不得附加任何条件。

** 商业用途（商业用途指个人出于任何商业目的使用本软件，或者法人或其他组织出于任何目的使用本软件）：
    未经原作者授权，任何个人不得出于任何商业目的使用本软件（包括但不限于向第三方
    提供、销售、出租、出借、转让本软件、本软件的衍生产品、引用或借鉴了本软件功能
    或源代码的产品或服务），任何法人或其他组织不得出于任何目的使用本软件，否则原
    作者有权追究相应的知识产权侵权责任。 \\
    在此前提下，对本软件的使用同样需要遵守 [[https://www.tldrlegal.com/l/gpl-3.0][GPL3]] 和 [[https://www.tldrlegal.com/l/lgpl-3.0][LGPL3]] 许可， [[https://www.tldrlegal.com/l/gpl-3.0][GPL3]] 和 [[https://www.tldrlegal.com/l/lgpl-3.0][LGPL3]] 许可与本
    许可冲突之处，以本许可为准。

    详细的授权流程，请联系 [email protected] 获取。

* Footnotes

[fn:1]闭源