The Sakata Index is a technical indicator calculated using a neural network designed for the Nikkei 225. Here is a part of the output1.log file:
date predic actual diff acc
2024-04-19 99.77 98.80 -0.97 -0.97
2024-04-22 100.34 101.57 1.22 0.26
...
2024-07-03 100.19 101.19 1.00 2.32
2024-07-05 100.32 99.04 -1.28 1.04
Mean Absolute Error: 0.68%
Epoch: 500000 BatchSize: 50 FinalLSM: 0.00166
Norm: 74.62
===
date predic actual diff acc
2024-04-22 100.33 101.57 1.24 1.24
2024-04-23 100.41 100.61 0.20 1.44
...
2024-07-05 100.31 99.04 -1.27 0.87
2024-07-08 100.13 100.24 0.11 0.98
Mean Absolute Error: 0.67%
Epoch: 500000 BatchSize: 50 FinalLSM: 0.00167
Norm: 67.51
===
- Date: This is the label for the date.
- Prediction: A value generated by a neural network model that attempts to predict the opening value of the N225 ETF.
- Actual: The actual opening value of the N225 ETF, which serves as the "teacher signal" for the neural network's predictions.
- Difference: The difference by subtracting the predicted value from the actual value.
- Accumulator: A running total of the daily differences.
The Sakata Index is a Norm: value for each batch period. The value is the normalized Accumulator for each batch period excluding the latest date. The normalization is the min and max values. Because the values are normalized excluding the latest date, they may fall below 0 or exceed 100. The signal becomes strong to buy as the Sakata Index approaches a negative value; conversely, the signal becomes strong to sell as it approaches over 100. Finally, this program's goal is calculating the Sakata index for each period.
The important aspect of the Sakata index is not bringing Prediction close to Actual, but accumulating the difference by subtracting Prediction from Actual. When this accumulated error reaches a certain threshold, it triggers a strong trading signal.
The Sakata Index is not a universal indicator and is weak in identifying trends. When the Nikkei 225 continues to rise, it stays above 80, and when it continues to fall, it stays below 20. In such cases, above 80 does not necessarily mean a sell, and below 20 does not necessarily mean a buy. Additionally, it often exhibits similar characteristics to the RSI.
Why Sakata?
Named after the Bioneural Device from Sakata Industry featured in Front Mission 2.
To use this program, follow these steps:
-
Prerequisites:
- Python 3
- Node.js 22
- C language
-
Install brain.js, tensorlow and MinGW-w64:
-
-
sudo apt-get install -y build-essential libglew-dev libglu1-mesa-dev libxi-dev pkg-config
npm install --no-audit
- On a Raspberry Pi 4, it takes approximately 12 minutes.
-
- Install Visual Studio 2022 or later.
- Install
Desktop development with C++
workload from Visual Studio Community. npm install --no-audit
-
-
-
pip install tensorflow
- On Raspberry Pi OS, type the following command to install h5py before installing TensorFlow:
sudo apt-get install libhdf5-dev
- On Raspberry Pi OS, type the following command to install h5py before installing TensorFlow:
-
pip install tensorflow
-
-
- If a C language compiler is available, the step can be skipped.
-
- This link is provided for setting up MinGW. However, it is not recommended due to an error with "The file has been downloaded incorrectly".
-
-
Install other packages:
pip install -r requirements.txt
-
Run scripts:
- Get latest raw data.
npm run getdata
- Normalize the raw data and split by batch size.
npm run cooking
- To run scripts and append the results.
npm run output1 npm run output2 npm run output3
- On Windows, replace
npm run output3
withnpm run output3-cache
.
- On Windows, replace
- To visualize the results. All results are stored in the result folder.
npm run plot-triple
- Get latest raw data.
-
Validation
- Please verify that the results in
output1-spot.log
completely match those inoutputc-spot.log
using the following command.
node ./src/nodejs/output1-spot.js > ./result/output1-spot.log gcc -O2 -o cdevice ./valid/cdevice.c -lm ./cdevice >> ./result/outputc-spot.log
- Please verify that the results in
The hdatexyt.csv
file should contain the following columns:
date
: This is the label for the date.close_x
: Closing value of US major ETF, input_x0close_y
: Closing value of USD/JPY, input_x1open_t
: Opening value of N225 ETF, teacher signal
All of the data are based on EDT. If the Japanese or U.S. market is on a holiday, the date will be skipped. close_x and close_y can be replaced with any data that correlate with the Nikkei 225.
Why the opening value of N225 ETF?
To avoid training on the initial values of the Nikkei 225.
We will train the same data using three different approaches. It would be good to compare the Norm and the opening price of the Nikkei 225 to find the output with the best performance.
- Train using an original neural network from without using deep learning libraries
npm run output1
- Train using brain.js
npm run output2
- Train using TensorFlow
npm run output3
-
output1.log
This is a log of the results obtained by using my implemented neural network to learn market data from approximately six months ago to today, segmented into batch-day period intervals and learned day by day. The value ofNorm:
for each period corresponds to the Sakata Index. The training parameters are as follows:- Input: 3 layers (including bias)
- Hidden: 4 layers (including bias)
- Output: 1 layer
- Initial weight: 0.5
- Max training iterations: 500000
- Activation function: Sigmoid
- Loss function: Least Square Method
- Learning rate: 0.5
- Biases: -1
- Training data: The same dataset is used for both training and testing the model.
Why is there a program that builds a neural network from without using deep learning libraries?
I wrote the first program for the Sakata Index in C in 2008. At that time, Python was not as widespread as it is today, neural networks were not given much attention, and there were no libraries available. The output1.js file is a rewrite of the code originally written in C into Node.js. The C code is still alive and well, and I sometimes use it for verification.
-
output2.log
This is a log of the results obtained by using brain.js to learn market data from approximately six months ago to today, segmented into batch-day period intervals and learned day by day. The value ofNorm:
for each period corresponds to the Sakata Index. The training parameters are the almost same as in output1.log. The training parameters are as follows:- Input: 3 layers
- Hidden: 4 layers
- Output: 1 layer
- Training iterations: 500000
- Activation function: Sigmoid
- Loss function: Mean Squared Error
- Learning rate: 0.5
- binaryThresh: 0.5
- errorThresh: 0.00001
-
output3.log
This is a log of the results obtained by using TensorFlow to learn market data from approximately six months ago to today, segmented into batch-day period intervals and learned day by day. The value ofNorm:
for each period corresponds to the Sakata Index. The training parameters are as follows:- Input: 2 layers
- Hidden: 16 layers
- Output: 1 layer
- Optimization: Adam
- Initial weight: 0.5
- Maximum training iterations: 1000
- Activation function: Sigmoid
- Loss function: Mean Squared Error
- Learning rate: 0.001
- Batch size: depend on setting.py
- EarlyStopping Parameters:
- min_delta: 0.0001
- patience: 300
- mode: min
Why is the loss function sigmoid and not ReLU?
ReLU is a fast and effective activation function used in many deep learning models. However, for nonlinear and complex problems such as financial market prediction, sigmoid or other activation functions may be more suitable. This is because the sigmoid function constrains the output to a range between 0 and 1, allowing for probabilistic interpretation, which can be useful in financial market prediction.
-
plot-triple.png
An image that combines the logs fromoutput1.log
,output2.log
, andoutput3.log
into a single line graph. When the three log files are trained over the same period, the following command will output a line graph.
npm run plot-triple
Note
In my original neural network (output1.js), I improved the speed of dot product calculations by replacing the dot function from mathjs with the standard reduce function. Consequently, the learning speed is now faster compared to brain.js.