trademaster-ntu / trademaster

TradeMaster is an open-source platform for quantitative trading empowered by reinforcement learning :fire: :zap: :rainbow:

TradeMaster: An RL Platform for Trading

 

---

TradeMaster is a first-of-its kind, best-in-class open-source platform for quantitative trading (QT) empowered by reinforcement learning (RL), which covers the full pipeline for the design, implementation, evaluation and deployment of RL-based algorithms.

⭐ What's NEW! ⏰

Update	Status
Update TradeMaster Website (Market Simualtor)	🔨 Updated on 21 Sep 2023
Update TradeMaster Website (Market Dynamics Modeling Tool)	🔧 Updated on 7 July 2023
Support automatic feature generation and selection	🔨 Update tutorial on 11 May 2023
Release TradeMaster Python package	🐳 Updated on 11 May 2023
Build TradeMaster website	🐳 Available at here on 23 April
Write TradeMaster software documentation	💬 Updated on 11 April 2023
Release Colab version	💬 Updated on 29 March 2023
Incldue HK Stock and Future datasets	🧭 Updated #131 #132 on 27 March 2023
Support Alpha158	🔨 Updated #123 #124 on 20 March 2023
Release TradeMaster 1.0.0	Released v1.0.0 on 5 March 2023

Outline

• TradeMaster: An RL Platform for Trading
  • Outline
  • Overview
  • Installation
  • Tutorial
  • Useful Script
  • Dataset
  • Model Zoo
  • Visualization Toolkit
  • File Structure
  • Publications
  • News
  • Team
  • Competition
  • Contact Us

Overview

TradeMaster is composed of 6 key modules: 1) multi-modality market data of different financial assets at multiple granularity; 2) whole data preprocessing pipeline; 3) a series of high-fidelity data-driven market simulators for mainstream QT tasks; 4) efficient implementations of over 13 novel RL-based trading algorithms; 5) systematic evaluation toolkits with 6 axes and 17 measures; 6) different interfaces for interdisciplinary users.

Installation

Here are the installation tutorials for different operating systems and docker:

• Installation on Linux/Windows/MacOS
• Installation with Docker

Tutorial

We provide tutorials covering core features of TradeMaster for users to get start with.

Algorithm	Dataset	Market	Task	Code Link
EIIE	DJ 30	US Stock	Portfolio Management	tutorial
DeepScalper	BTC	Crypto	Intraday Trading	tutorial
SARL	DJ 30	US Stock	Portfolio Management	tutorial
PPO	SSE 50	China Stock	Portfolio Management	tutorial
ETEO	Bitcoin	Crypto	Order Execution	tutorial
Double DQN	Bitcoin	Crypto	High Frequency Trading	tutorial

We also provide a colab version of these tutorials that can be run directly. (colab tutorial)

Useful Script

• Automatic hyperparameter tuning
• Automatic feature generation
• Financial data imputation with diffusion models
• Train RL agents with Alpha158 technical indicators

TradeMaster Sandbox

• White paper
• Market dynamics modeling
• Website API
  1. Agent training Demo
  2. Market dynamics modeling tool
  3. Market simulator web service

Dataset

Dataset	Data Source	Type	Range and Frequency	Raw Data	Datasheet
S&P500	Yahoo	US Stock	2000/01/01-2022/01/01, 1day	OHLCV	SP500
DJ30	Yahoo	US Stock	2012/01/01-2021/12/31, 1day	OHLCV	DJ30
BTC	Kaggle	Foreign Exchange	2000/01/01-2019/12/31, 1day	OHLCV	FX
Crypto	Kaggle	Crypto	2013/04/29-2021/07/06, 1day	OHLCV	Crypto
SSE50	Yahoo	China Stock	2009/01/02-2021/01/01, 1day	OHLCV	SSE50
Bitcoin	Binance	Crypto	2021/04/07-2021/04/19, 1min	LOB	Binance
Future	AKshare	Future	2023/03/07-2023/03/28, 5min	OHLCV	Future
HS30	AKShare	HK Stock	1988/12/30-2023/03/27, 1day	OHLCV	HS30

Dates are in YY/MM/DD format.

OHLCV: open, high, low, and close prices; volume: corresponding trading volume; LOB: Limit order book.

Users can download data of the above datasets from Google Drive or Baidu Cloud (extraction code:x24b)

Model Zoo

TradeMaster provides efficient implementations of the following algorithms:

DeepScalper based on Pytorch (Shuo Sun et al, CIKM 22)

OPD based on Pytorch (Fang et al, AAAI 21)

DeepTrader based on Pytorch (Wang et al, AAAI 21)

SARL based on Pytorch (Yunan Ye et al, AAAI 20)

ETEO based on Pytorch (Lin et al, 20)

Investor-Imitator based on Pytorch (Yi Ding et al, KDD 18)

EIIE based on Pytorch (Jiang et al, 17)

Classic RL based on Pytorch and Ray: PPO A2C Rainbow SAC DDPG DQN PG TD3

Visualization Toolkit

TradeMaster provides many visualization toolkits for a systematic evaluation of RL-based quantitative trading methods. Please check this paper and repository for details. Some examples are as follows:

PRIDE-Star is a star plot containing normalized score of 8 key financial measures such total return (TR) and Sharpe ratio (SR) to evaluate profitability,risk-control and diversity:

File Structure

| TradeMaster
| ├── configs
| ├── data
| │   ├── algorithmic_trading 
| │   ├── high_frequency_trading  
| │   ├── order_excution          
| │   └── porfolio_management
| ├── deploy
| │   ├── backend_client.py
| │   ├── backend_client_test.py 
| │   └── backend_service.py        
| │   ├── backend_service_test.py  
| ├── docs
| ├── figure
| ├── installation
| │   ├── docker.md
| │   ├── requirements.md
| ├── tools
| │   ├── algorithmic_trading          
| │   ├── data_preprocessor
| │   ├── high_frequency_trading
| │   ├── market_dynamics_labeling
| │   ├── missing_value_imputation  
| │   ├── order_excution  
| │   ├── porfolio_management  
| │   ├── __init__.py      
| ├── tradmaster       
| │   ├── agents   
| │   ├── datasets 
| │   ├── enviornments 
| │   ├── evaluation 
| │   ├── imputation 
| │   ├── losses
| │   ├── nets
| │   ├── preprocessor
| │   ├── optimizers
| │   ├── pretrained
| │   ├── trainers
| │   ├── transition
| │   ├── utils
| │   └── __init__.py     
| ├── unit_testing
| ├── Dockerfile
| ├── LICENSE
| ├── README.md
| ├── pyproject.toml
| └── requirements.txt

Publications

PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets (Transactions on Machine learning Research 2023)

Reinforcement Learning for Quantitative Trading (Survey) (ACM Transactions on Intelligent Systems and Technology 2023)

Deep Reinforcement Learning for Quantitative Trading: Challenges and Opportunities (IEEE Intelligent Systems 2022)

DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities (CIKM 2022)

Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management (AAAI 21)

News

• [AI科技评论]TradeMaster最新发布沙盒工具箱，探索量化交易的全新维度
• [AAAI Summer Syposium]Call for participation on RL4Fintech syposium in Singapore
• [知乎]用强化学习在金融市场上赚钱？南洋理工发布全新基于强化学习的量化交易平台TradeMaster
• [机器之心]南洋理工发布量化交易大师TradeMaster，涵盖15种强化学习算法
• [运筹OR帷幄]量化交易大师TradeMaster
• [Medium]Introduction on TradeMaster
• [AAAI-23 Bridge Program]Host a tutorial on reinforcement learning for quantitative trading with TradeMaster

Team

• This repository is developed and maintained by AMI group at Nanyang Technological University.
• We have positions for software engineer, research associate and postdoc. If you are interested in working at the intersection of RL and quantitative trading, feel free to send us an email with your CV.

Competition

TradeMaster Cup 2022

Contact Us

If you have any further questions of this project, please contact [email protected]

trademaster's Issues

max drawdown calculation

Is the max drawdown calculation correct in the evaluate function in TradeMaster\env\PM\portfolio_for_EIIE.py
It is calculated as
mdd = max((max(df["total assets"]) - df["total assets"]) /max(df["total assets"]))
A very big increase in total assets may also be considered as max drawdown, e.g. An asset list like [1,5,4,3,2].

Enquiry about the jupyter memory problem

HI Teams,

When I tried to run the "DeepScalper" tutorial, I met the below issue:

"your server is down"

I think it is because the mac can support the memory used by jupyter. Could you help find another solution.

Many thanks!

Can trademaster train using tick data?

Can trademaster train using tick data?

If not, how to modify the project?

difference between save_model and save_best_model

在 train_and_valid 中
1)每次valid 时 调用 save_model
2)当整个train_and_valid 结束时，会调用 save_best_model

save_best_model 保存的是 【训练结束时的 模型】，还是 epoch = np.argmax(valid_score_list) 时【 表现最佳的模型】？

从代码看，属于保存 训练结束时的 模型? 并不是保存的 【表现最佳的模型】？

Validation/Test Performance Gap in Tutorial Notebook

Hi, thanks for making your great work public.

I'm a bit confused about the huge test/validation performance gap in some of your tutorials. In particular, the tutorial for DeepScalper achieves a negative return, but a +200% return on the test set, which does not make sense to me. Similarly, in the notebook for double DQN, the validation return is also negative, while achieving a positive return on the test set.

Intuitively, it's hard to believe that we are able to achieve a positive return in the future if we cannot even achieve a positive return on the validation set.

Am I missing something here?

No such file or directory:adam_mse/checkpoints/best.pth

Hi, When I installed, run "python tools/algorithmic_trading/train.py" , then error msg is:

FileNotFoundError: [Errno 2] No such file or directory: '/root/TradeMaster/work_dir/algorithmic_trading_FX_deepscalper_deepscalper_adam_mse/checkpoints/best.pth'.

Would you please tell me how to solve this problem? thanks.

No Simulator

It seems like simulator based on Market-GAN is missing in this repository, which is included in 3) a series of high-fidelity data-driven market simulators for mainstream QT tasks of README.md.

UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 18715: character maps to <undefined>

Windows 10, Anaconda

(tmaster) D:>pip install git+https://github.com/TradeMaster-NTU/TradeMaster.git
Collecting git+https://github.com/TradeMaster-NTU/TradeMaster.git
Cloning https://github.com/TradeMaster-NTU/TradeMaster.git to c:\users\user\appdata\local\temp\pip-req-build-ehfyt5hh
Running command git clone --filter=blob:none --quiet https://github.com/TradeMaster-NTU/TradeMaster.git 'C:\Users\user\AppData\Local\Temp\pip-req-build-ehfyt5hh'
Resolved https://github.com/TradeMaster-NTU/TradeMaster.git to commit d973794
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "C:\Users\user\AppData\Local\Temp\pip-req-build-ehfyt5hh\setup.py", line 4, in
long_description = fh.read()
File "D:\Programs\Anaconda\envs\tmaster\lib\encodings\cp1251.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 18715: character maps to
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

no cuda issue while installation issue

Hi developers,

My server: region-42.seetacloud.com
SSH Port: 33754
root
5ikP1F1Tj6

When I try to run the command: "python tools/algorithmic_trading/train.py"

I met the issue:
ERROR: No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'

But I followed the instructions step by step to install it.

Would you please help see what the problem is? Thank you!
The server information is as above. Thanks.

DeepTrader

It seems the dir of './tools/portpolio_management/train_deeptrader' is not runing, coding is not finishing

How to train the LSTM in the SARL?

I noticed sarl_environment.py use pretrained parameters from LSTM.pth. Just wonder how to train the LSTM model and get the LSTM.pth by myself.

pip install ray[rllib]==1.13.0 error

Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [16 lines of output]
/root/micromamba/lib/python3.9/site-packages/setuptools/dist.py:745: SetuptoolsDeprecationWarning: Invalid dash-separated options
!!

          ********************************************************************************
          Usage of dash-separated 'index-url' will not be supported in future
          versions. Please use the underscore name 'index_url' instead.
  
          By 2023-Sep-26, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.
  
          See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
          ********************************************************************************
  
  !!
    opt = self.warn_dash_deprecation(opt, section)
  error in gym setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.

Should we upgrade dependencies?

Logging losses and reward with Tensorboard

I have an issue with critic losses (increasing) when training AlphaMix+, please help to take a look at this issue.

TradeMaster-NTU/PRUDEX-Compass#23

The same is applied for DeepTrader implementation in this repo (critic losses creasing). I wonder is there any issue with the code regarding critic losses, could you please have a check/comment on this.

Thank you.

Questions about the implementation of DeepScalper

Hi, first I'd like to thank you all for this brilliant repo!

I found this repo by searching the relevant information of DeepScalper. I see this repo is maintained by a team of NTU led by Prof Bo An, who is also one of the authors of DeepScalper, so it's reasonable to assume that what is implemented here should be the same as what is described in the paper. However, I have found some major discrepancies:

1. The paper says DeepScalper uses both macro-level(OHLCV data and technical indicators) and micro-level(LOB data) as features, but in this repo only macro-level data is used.
2. The paper uses per-minute OHLCV data, while this repo only uses daily OHLCV data.
3. The paper states that a two-dimensional action space of (price, volume) is used:

We use a more realistic two-dimensional
fine-grained action space for intraday trading, which represents
a limit order as a tuple (𝑝𝑡𝑎𝑟𝑔𝑒𝑡, ±𝑞𝑡𝑎𝑟𝑔𝑒𝑡). 𝑝𝑡𝑎𝑟𝑔𝑒𝑡 is the target
price, 𝑞𝑡𝑎𝑟𝑔𝑒𝑡 is the target quantity and ± is the trading direction
(long/short).

However, in this repo only the volume dimension is present, and the execution price is simply assumed to be the latest trade price(close price) of the last bar, which is not a realistic assumption in the context of high frequency trading, as it ignores bid-ask spread and slippage.

All the above makes the implementation looks like only a prototype instead of a real one. I wonder what is the reason behind it? Your reply would be appreciated, thanks!

what is the definition of "buys" and "sells" in the dataset for eteo environment?

whis is the definition of the dataset of eteo ？
1、midpoint
2、buys
3、sells
4、bids_distance_0
5、bids_notaional_0
6、bids_cancel_notional_0
7、bids_limit_notional_0
8、bids_market_notional_0

I can guess some,not very sure

Is the algorithmic_trading using DDQN or DQN?

Hi

Is the algorithmic_trading using DDQN or DQN? If not, is it possible to convert it to DDQN?

Which example is about Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management (AAAI 2021)

It is a great honor to read this paper published in AAAI in 2021,
Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management.

I would like to know which example I can refer to for the HRPM architecture of this paper.

Thanks.

Shape error of the actor forward function in DeepTrader task

import os
import sys
from pathlib import Path
import warnings
warnings.filterwarnings("ignore")
ROOT = os.path.dirname(os.path.abspath("."))
sys.path.append(ROOT)
import torch
import argparse
import os.path as osp
from mmcv import Config
from trademaster.utils import replace_cfg_vals
from trademaster.nets.builder import build_net
from trademaster.environments.builder import build_environment
from trademaster.datasets.builder import build_dataset
from trademaster.agents.builder import build_agent
from trademaster.optimizers.builder import build_optimizer
from trademaster.losses.builder import build_loss
from trademaster.trainers.builder import build_trainer
from trademaster.transition.builder import build_transition
from trademaster.utils import plot
from trademaster.utils import set_seed
set_seed(2023)

parser = argparse.ArgumentParser(description='Download dj30 Datasets')
parser.add_argument("--config", default=osp.join(ROOT, "configs", "portfolio_management", "portfolio_management_dj30_deeptrader_deeptrader_adam_mse.py"),
                    help="download datasets config file path")
parser.add_argument("--task_name", type=str, default="train")

args, _= parser.parse_known_args()
cfg = Config.fromfile(args.config)
task_name = args.task_name
cfg = replace_cfg_vals(cfg)

dataset = build_dataset(cfg)

train_environment = build_environment(cfg, default_args=dict(dataset=dataset, task="train"))
valid_environment = build_environment(cfg, default_args=dict(dataset=dataset, task="valid"))
test_environment = build_environment(cfg, default_args=dict(dataset=dataset, task="test"))


action_dim = train_environment.action_dim # 29
state_dim = train_environment.state_dim # 16
input_dim = len(train_environment.tech_indicator_list)

act = build_net(cfg.act_net)
cri = build_net(cfg.cri_net)
market = build_net(cfg.market_net)
act_optimizer = build_optimizer(cfg, default_args=dict(params=act.parameters()))
cri_optimizer = build_optimizer(cfg, default_args=dict(params=cri.parameters()))
market_optimizer = build_optimizer(cfg, default_args=dict(params=market.parameters()))
criterion = build_loss(cfg)

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
agent = build_agent(cfg, default_args=dict(action_dim=action_dim, 
                        state_dim=state_dim,act=act,cri=cri,
                        market=market,act_optimizer=act_optimizer, \
                        cri_optimizer = cri_optimizer, criterion = criterion,\
                        market_optimizer = market_optimizer,device = device))

trainer = build_trainer(cfg,
                        default_args=dict(train_environment=train_environment,
                                          valid_environment=valid_environment,
                                          test_environment=test_environment,
                                          agent=agent,device=device))
work_dir = os.path.join(ROOT, cfg.trainer.work_dir)

if not os.path.exists(work_dir):
    os.makedirs(work_dir)
cfg.dump(osp.join(work_dir, osp.basename(args.config)))

trainer.train_and_valid()

And I got the error :

Traceback (most recent call last):
  File "C:\Users\HuZetian\Desktop\TradeMaster\tutorial\Tutorial_DeepTrader.py", line 72, in <module>
    trainer.train_and_valid()
  File "C:\Users\HuZetian\Desktop\TradeMaster\trademaster\trainers\portfolio_management\deeptrader_trainer.py", line 173, in train_and_valid
    buffer_items = self.agent.explore_env(self.train_environment, self.horizon_len)
  File "C:\Users\HuZetian\Desktop\TradeMaster\trademaster\agents\portfolio_management\deeptrader.py", line 216, in explore_env    
    action = get_action(state,market_state,corr_matrix)
  File "C:\Users\HuZetian\Desktop\TradeMaster\trademaster\agents\portfolio_management\deeptrader.py", line 173, in get_action 
    asset_scores = self.act(state, corr_matrix)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl     
    return forward_call(*input, **kwargs)
  File "C:\Users\HuZetian\Desktop\TradeMaster\trademaster\nets\deeptrader.py", line 221, in forward
    H_L = self.TCN(x)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl     
    return forward_call(*input, **kwargs)
  File "C:\Users\HuZetian\Desktop\TradeMaster\trademaster\nets\deeptrader.py", line 135, in forward
    return self.network(x)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl     
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\container.py", line 139, in forward      
    input = module(input)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl     
    return forward_call(*input, **kwargs)
  File "C:\Users\HuZetian\Desktop\TradeMaster\trademaster\nets\deeptrader.py", line 89, in forward
    out = self.net(x)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl     
    return forward_call(*input, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\container.py", line 139, in forward      
    input = module(input)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\module.py", line 1148, in _call_impl     
    result = forward_call(*input, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\conv.py", line 307, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "C:\ProgramData\Anaconda3\envs\TradeMaster\lib\site-packages\torch\nn\modules\conv.py", line 303, in _conv_forward     
    return F.conv1d(input, weight, bias, self.stride,
RuntimeError: Expected 2D (unbatched) or 3D (batched) input to conv1d, but got input of size: [1, 29, 16, 10]

Can help me with it?

where was rllib used in trade_master?

ROOT path not working with pyenv and interactive shell

Hi. I tried setting up one of the tutorials but got an annoying error with the ROOT path because file returns a path relative to the virtualenv.

To make it work with all cases it we just have to change to:

import os
FILE_PATH = os.path.dirname(os.path.abspath("__file__"))
ROOT = str(Path(FILE_PATH).resolve().parents[3])

example for AlgorithmicTradingDataset

Is there any datasheet for the dataset in `high_frequency_trading`

I'm learning the LOB data but i can only find explanations on Kaggle which is very different from the actual data I saw in high_frequency_trading folder. For example, buy_volume_oe, sell_volume_oe. Is there a formal doc explaining all the columns in that sheet? Thanks

  | Unnamed: 0 | bid1_price | bid1_size | bid2_price | bid2_size | bid3_price | bid3_size | bid4_price | bid4_size | bid5_price | bid5_size | ask1_price | ask1_size | ask2_price | ask2_size | ask3_price | ask3_size | ask4_price | ask4_size | ask5_price | ask5_size | buy_volume_oe | sell_volume_oe | bid1_size_n | bid2_size_n | bid3_size_n | bid4_size_n | bid5_size_n | ask1_size_n | ask2_size_n | ask3_size_n | ask4_size_n | ask5_size_n | buy_spread_oe | sell_spread_oe | imblance_volume_oe | open | high | close | low | wap | trade_diff | trade_speard | kmid | klen | kmid2 | kup | kup2 | klow | klow2 | ksft | ksft2 | roc_10 | roc_30 | roc_60 | ma_10 | ma_30 | ma_60 | std_10 | std_30 | std_60 | beta_10 | beta_30 | beta_60 | max_10 | max_30 | max_60 | min_10 | min_30 | min_60 | qtlu_10 | qtlu_30 | qtlu_60 | qtld_10 | qtld_30 | qtld_60 | rsv_10 | rsv_30 | rsv_60 | imax_10 | imax_30 | imax_60 | imin_10 | imin_30 | imin_60 | imxd_10 | imxd_30 | imxd_60 | cntp_10 | cntp_30 | cntp_60 | cntn_10 | cntn_30 | cntn_60 | cntd_10 | cntd_30 | cntd_60
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --

Enquiry about the installation issue

Hi developers,

When I try to run the command: "pip install git+https://github.com/TradeMaster-NTU/TradeMaster.git"

I met the issue:
ERROR: git+https://github.com/TradeMaster-NTU/TradeMaster.git does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

Will the team releases more documents

The project is complex and hard to fully understand. It is appreciate to release more documents.

Feature Request

First of all - thank you so much for the project. Your code, organization and documentation are excellent. It's great to find a project that works and examples / notebooks are helpful and work well. Thank you again. I love reading your papers also, very informative. Thank you for sharing this information freely and openly.

A few feature requests that would be helpful: -

1. Can you provide details on how to implement a paper trading environment similar to FinRL and Alpaca? This would be very helpful, allowing the testing of model on live trading data. Similar to:

https://github.com/AI4Finance-Foundation/FinRL/blob/master/examples/FinRL_PaperTrading_Demo.ipynb

2. Tutorials for integrating PRUDEX-Compass into TradeMaster and more reporting and logging of the DRL agent's actions/steps. Many of the reports output the return/sharpe ratio, but not easy to find details, csv, report on the trades, portfolio allocations (buys and sells, etc). Again, this would be useful to understand deeper into the best model and it's behavior and further optimization in different market conditions and asset types.

Short/Long Margin

Hi

Is there any example of going long or short with margin?

How can I retrain the sarl encoder?

I want to add some features to the SARL agent. However, since it will go through a pre-trained encoder whose dimension is fixed, I need to retrain the SARL encoder for the new features. Can you describe the training pipeline, or better yet, upload it to the repository?

Does the net and environment of high_frequency_trading support adding a new input "datetime"？

The data of small_BTC lacks datetime.

AttributeError: 'EntryPoints' object has no attribute 'get'

Hi, I got into this problem when I tried to run train.py to test my installation.

Message='EntryPoints' object has no attribute 'get'
Source=C:\Users\Jeff\Desktop\TraderMaster\TradeMaster-1.0.0\TradeMaster-1.0.0\trademaster\environments\custom.py
StackTrace:
File "C:\Users\Jeff\Desktop\TraderMaster\TradeMaster-1.0.0\TradeMaster-1.0.0\trademaster\environments\custom.py", line 2, in
import gym
File "C:\Users\Jeff\Desktop\TraderMaster\TradeMaster-1.0.0\TradeMaster-1.0.0\trademaster\environments_init_.py", line 1, in
from .custom import Environments
File "C:\Users\Jeff\Desktop\TraderMaster\TradeMaster-1.0.0\TradeMaster-1.0.0\tools\algorithmic_trading\train.py", line 15, in
from trademaster.environments.builder import build_environment
AttributeError: 'EntryPoints' object has no attribute 'get'

I'm currently on Cuda 12.1 with pytorch2

What does it mean when hold_volume is negative?

Hi, i'm trying to figure out the code logic for AlgorithmicTradingEnvironment.

I don't quite understand the below line as when I'm running Tutorial2_DeepScalper.ipynb, i can see negative hold_volume. Could someone help explain what does it stands for?

Default branch change

When installing using the instructions for docker for arm, some issues are encountered. I believe its because of the default branch being 1.0.0 in the dockerfile. Thus, one can't run the experiment for checking installation. I changed it to specify the branch to main for myself but I presume others might encounter this issue as well

Small Error in Tutorial 9: openfe()

The following error could be specific to my environment but having issues with ofe = openfe()

Running the following code:

n_jobs=4
ofe = openfe()
ofe.fit(data=X, label=Y, n_jobs=n_jobs)
train_x, valid_x = transform(train_x, valid_x, ofe.new_features_list[:10], n_jobs=n_jobs)
X, test_x = transform(X, test_x, ofe.new_features_list[:10], n_jobs=n_jobs)

**Gives this error:

TypeError Traceback (most recent call last)
Cell In[5], line 179
176 Y = pd.DataFrame({"close": list(df_train['close'][1:])})
178 n_jobs=4
--> 179 ofe = openfe()
180 ofe.fit(data=X, label=Y, n_jobs=n_jobs)
181 train_x, valid_x = transform(train_x, valid_x, ofe.new_features_list[:10], n_jobs=n_jobs)

TypeError: 'module' object is not callable

Change the import and code resolved the issue for me:

from openfe import OpenFE, transform

n_jobs=4
ofe = OpenFE()
ofe.fit(data=X, label=Y, n_jobs=n_jobs)
train_x, valid_x = transform(train_x, valid_x, ofe.new_features_list[:10], n_jobs=n_jobs)
X, test_x = transform(X, test_x, ofe.new_features_list[:10], n_jobs=n_jobs)

fatal Python error: Aborted

"python tools/algorithmic_trading/train.py" error

labeling_util.py
ModuleNotFoundError: No module named 'dtaidistance'

ModuleNotFoundError: No module named 'fastdtw'

KeyError: 'PortfolioManagementEIIETrainer is not in the trainer registry'

刚开始运行Tutorial1_EIIE，在运行下面代码时出现报错：
trainer = build_trainer(cfg, default_args=dict(train_environment=train_environment,valid_environment=valid_environment,test_environment=test_environment,agent=agent,device=device))
work_dir = os.path.join(ROOT, cfg.trainer.work_dir)

if not os.path.exists(work_dir):
os.makedirs(work_dir)
cfg.dump(osp.join(work_dir, osp.basename(args.config)))

报错：KeyError: 'PortfolioManagementEIIETrainer is not in the trainer registry'
KeyError Traceback (most recent call last)
Input In [19], in <cell line: 1>()
----> 1 trainer = build_trainer(cfg, default_args=dict(train_environment=train_environment,valid_environment=valid_environment,test_environment=test_environment,agent=agent,device=device))
2 work_dir = os.path.join(ROOT, cfg.trainer.work_dir)
4 if not os.path.exists(work_dir):

File D:\0quant_nft_nlp\model_quant_qlib\TradeMaster\TradeMaster-1.0.0\trademaster\trainers\builder.py:8, in build_trainer(cfg, default_args)
6 def build_trainer(cfg, default_args=None):
7 cp_cfg = dict(cfg.trainer)
----> 8 trainer = build_from_cfg(cp_cfg, TRAINERS, default_args)
9 return trainer

File D:\0quant_nft_nlp\model_quant_qlib\TradeMaster\TradeMaster-1.0.0\trademaster\utils\utils.py:107, in build_from_cfg(cfg, registry, default_args)
105 obj_cls = registry.get(obj_type)
106 if obj_cls is None:
--> 107 raise KeyError(
108 f'{obj_type} is not in the {registry.name} registry')
109 elif inspect.isclass(obj_type):
110 obj_cls = obj_type

KeyError: 'PortfolioManagementEIIETrainer is not in the trainer registry'

请教怎么解决，谢谢。

bug cannot run tools/train_eiie.py

"pip install -r requirements.txt" error

Collecting gym<0.22 (from ray[rllib]==1.13.0->-r requirements.txt (line 20))
Using cached gym-0.21.0.tar.gz (1.5 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
error in gym setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

MIssing file 'data/algorithmic_trading/AAPL/test_labeled_3_24_-0.05_0.05.csv'

Can not find this file in repo: 'data/algorithmic_trading/AAPL/test_labeled_3_24_-0.05_0.05.csv'

No CUDA runtime is found, using CUDA_HOME='C:\ProgramData\miniconda3\envs\TradeMaster' Compiling mmcv._ext only with CPU

when I install requirements of TradeMaster, I got this error:

Building wheels for collected packages: mmcv
Building wheel for mmcv (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [956 lines of output]
No CUDA runtime is found, using CUDA_HOME='C:\ProgramData\miniconda3\envs\TradeMaster'
Compiling mmcv._ext only with CPU
running bdist_wheel
C:\ProgramData\miniconda3\envs\TradeMaster\lib\site-packages\torch\utils\cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow dist
utils backend.
warnings.warn(msg.format('we could not find ninja.'))

NameError: name 'TrainerRegistry' is not defined

刚开始运行Tutorial1_EIIE，在运行下面代码时出现报错：
trainer = build_trainer(cfg, default_args=dict(train_environment=train_environment,valid_environment=valid_environment,test_environment=test_environment,agent=agent,device=device))
work_dir = os.path.join(ROOT, cfg.trainer.work_dir)

if not os.path.exists(work_dir):
os.makedirs(work_dir)
cfg.dump(osp.join(work_dir, osp.basename(args.config)))

报错：NameError: name 'TrainerRegistry' is not defined
请教怎么解决，谢谢。

ETEO Algorithm and data

Hi,

I am trying to figure out ETEO algorithm for OE. However, could you please provide the source paper of ETEO algorithm, an arXiv link should be helpful.

Also, could you please help to explain the data used in ETEO algorithm, e.g. bids_distance_x, bids_notional_x, bids_cancel_notional_x, bids_limit_notional_x, bids_market_notional_x, asks_distance_x, asks_notional_x, asks_cancel_notional_x, asks_limit_notional_x, asks_market_notional_x. Normally, I only notice bid ask price and volume, so it's difficult for me to collect data on my own.

Thank you for your help.

How to define num_action for new trade type?

If I train the model for new trade type, for example, Japanese Stock or London Future, how to define num_action?

Feature request: 3-D volume modelling for price trend observation

建立3维模型后可以将趋势线向传统的 收盘价走势图做投影，
发现交易均价点与收盘价有乖离现象，该现象对短线判断 收盘价走势有明显提示功能。

pretrained mode loading

Hi,

some model are loading pretrained model file with fixed input dimension. such as "sarl", "imitator model"
self.net.load_state_dict(self.network_dict)
how this pretrained mode is generated, and is there any manual to pretrain our own model ?

ETEO Algorithm logical issues

Hi,

I haven't still got the whole idea of ETEO implementation, so I create this new thread. Hope you guys can discuss and give some comments.

1. Some conditional statement regarding if order[1] == self.data["midpoint"] ... * 0.01 almost never happens. Because the predicted price order[1] rarely satisfy this condition. I'm wondering why you didn't use <= or >= instead of ==
2. What is the meaning behind start_volume, because you calculate start_volume, and assign it to order then order_history, and use it to calculate tradable_volume, but then again, the condition if order[1] == self.data["midpoint"] ... * 0.01 rarely happen.
3. Also, should I modify buy_volume = min(action[0], **self.state[-1]**), because buy volume should not larger than order left.
4. Somehow, I find duplicating check action and order is little weird (with the same conditional)
5. Currently, action[0] is always positive, which mean we are going to buy more bitcoin. Do we need to train another model to sell? I don't think 1 AI model can handle 2 buy & sell tasks.

I am looking forward to your discussion.
Thank you.

tools/market_dynamics_labeling/run.py not taking in command line config arguments

The run.py script does not take in the command line arguments, but only reads the values from configs/evaluation/market_dynamics_modeling.py file.

Example:
python tools/market_dynamics_labeling/run.py --data_path data/portfolio_management/dj30/DJI.csv --method linear --fitting_parameters 2/7 2/14 4 --labeling_parameters -0.25 0.25 --regime_number 3 --length
_limit 24 --PM data/portfolio_management/dj30/test.csv

I added print statement to run.py to print the cfgs passed to build_market_dynamics_model:
model = build_market_dynamics_model(cfg)

this is cfgs values:
Config (path: /home/TradeMaster/configs/evaluation/market_dynamics_modeling.py): {'market_dynamics_model': {'data_path': 'data/high_frequency_trading/small_BTC/test.csv', 'fitting_parameters': ['2/7', '2/14', '4'], 'labeling_parameters': [-0.01, 0.01], 'regime_number': 3, 'length_limit': 300, 'OE_BTC': False, 'PM': '', 'process_datafile_path': '/home/TradeMaster/data/high_frequency_trading/small_BTC/test_labeled_3_300_-0.01_0.01.csv', 'market_dynamic_labeling_visualization_paths': ['/home/TradeMaster/data/high_frequency_trading/small_BTC/MDM_linear/3_300_-0.01_0.01_HFT_small_BTC.png'], 'type': 'Linear_Market_Dynamics_Model'}, 'task_name': 'algorithmic_trading', 'dataset_name': 'BTC'}

AlphaMix+ and others algorithms

Is it possible to embed AlphaMix+ into this framework? Or it uses another different framework (MoE).

Also, when I trained EIIE and DeepTrader, the weights are either 0-1 or uniform distributed. I don't know if there is any wrong with the implementation. A quick comparison between the implementation of DeepTrader in this repo and https://github.com/CMACH508/DeepTrader/ shows that the models are different (softmax vs. relu, batch norm, etc.)

I hope you guys can give a quick comments on these issues.

Support for multi-agent

May i know if it supports the simulation of multi-agent trading? Thanks!

Error running Tutorial3_SARL.ipynb ray.get(f.remote(sharpe['table'])) KeyError: 'table'

When running Tutorial3_SARL.ipynb, I am getting error when running trainer.test().

KeyError Traceback (most recent call last)
Cell In[6], line 12
10 ray.init(ignore_reinit_error=True)
11 register_env("portfolio_management_sarl", lambda config: env_creator("portfolio_management_sarl")(config))
---> 12 trainer.test();

File /home/TradeMaster/trademaster/trainers/portfolio_management/sarl_trainer.py:167, in PortfolioManagementSARLTrainer.test(self)
164 # ray.get(f.remote("Test Best Episode Reward Sum: {:04f}".format(episode_reward_sum))
165 break
--> 167 ray.get(f.remote(sharpe['table']))
168 rewards = self.test_environment.save_asset_memory()
169 assets = rewards["total assets"].values

KeyError: 'table'

I was able to comment out line #167 to get the code to complete.

Error installing packages in colab

Hi many thanks for this wonderful resource, I have a question if I may. I tried to install the packages in your new colab version but there seems to be many conda packages in the requirements.txt and I get this type of error:

ERROR: Could not find a version that satisfies the requirement ca-certificates (from versions: none)
ERROR: No matching distribution found for ca-certificates

How did you manage to install the all the dependencies?

One workaround I tried is to install conda in colab by running this cell

!pip install -q condacolab
import condacolab
condacolab.install()

then converting the requirements.txt into this environment.yaml file but the installation process is very slow (over 1 hour) as there are many dependencies. Ideally I would like to install with pip and was wondering if you have a solution?

Documents for the difference of RL environments

是否有以下环境的 一些介绍资料？比如porfolio 各个环境的 区别？
一、Porfolio

1. deeptrader_environment
   2)eiie_environment
   3)environment
   4)investtor_imitator_environment
   5)sarl_environment
   二、high_frequency_trading
2. environment
   三、order_excution
3. eteo_environment.py
4. pd_environment.py