cinderella-man / igthorn Goto Github PK

View Code? Open in Web Editor NEW

97.0 9.0 14.0 15.29 MB

Cryptocurrency trading platform

License: MIT License

Elixir 83.03% CSS 8.44% JavaScript 1.53% HTML 6.88% Shell 0.12%

binance elixir elixir-phoenix trading-bot strategies streaming phoenix-liveview cryptocurrency

igthorn's People

Contributors

Stargazers

Watchers

Forkers

soimil haibo32012 thorjeus simhaonline seanwbren cdesch johnmcdowall baitcenter mmomaster-network gustavoarmoa davidthinh nikolaiivanov mushu8 ttitcombe

igthorn's Issues

Remove "empty" pages from docs

Currently generated docs (using mix docs) are full of empty pages for modules like Hefty.Algos.Naive.

We should remove completely empty modules

Investigate `Phoenix:live_reload :error` info messages

iex(1)> 15:46:11.229 application=phoenix module=Phoenix.Channel.Server [info] Replied phoenix:live_reload :error
iex(1)> 15:46:21.235 application=phoenix module=Phoenix.Channel.Server [info] Replied phoenix:live_reload :error
iex(1)> 15:46:31.241 application=phoenix module=Phoenix.Channel.Server [info] Replied phoenix:live_reload :error
iex(1)> 15:46:41.245 application=phoenix module=Phoenix.Channel.Server [info] Replied phoenix:live_reload :error
iex(1)> 15:46:51.248 application=phoenix module=Phoenix.Channel.Server [info] Replied phoenix:live_reload :error
iex(1)> 15:47:01.252 application=phoenix module=Phoenix.Channel.Server [info] Replied phoenix:live_reload :error
iex(1)> 15:47:11.257 application=phoenix module=Phoenix.Channel.Server [info] Replied phoenix:live_reload :error

Orders list

Paginated list of order needs to be implemented with search on symbol

Rethink pagination

Pagination could be better:

amount of rows per page looks a bit odd (styling)
refresh is completely voiding the pagination state (both number of rows per page as well as current page)

Would assume that current state should be put in browser link

Production bug - `constraint error when attempting to insert`

Observed in live system:

iex(1)> 15:47:20.982 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(8654) - Rebuy triggered for trade 253938705 bought @ 0.24153000 as price fallen below 0.24128847
iex(1)> 15:47:20.983 application=hefty module=Hefty.Algos.Naive.Leader [info] Rebuy notification received, starting a new trader
iex(1)> 15:47:21.001 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader(9241) on symbol XRPUSDT with budget of 11.36414556234
iex(1)> 15:47:21.261 application=phoenix module=Phoenix.Channel.Server [info] Replied phoenix:live_reload :error
iex(1)> 15:47:23.118 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(9241) - Placing BUY order for XRPUSDT @ 0.24099, quantity: 47.1
iex(1)> 15:48:38.045 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(9241) received an transaction of 0.40000000 for BUY order 253948392 @ 0.24099000
iex(1)> 15:48:38.751 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(9241) received an transaction of 33.00000000 for BUY order 253948392 @ 0.24099000
iex(1)> 15:48:39.005 application=hefty module=Hefty.Algos.Naive.Trader [info] Placing SELL order for XRPUSDT @ 0.24195, quantity: 47.1
iex(1)> 15:48:39.812 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(9241) received an transaction of 13.70000000 for BUY order 253948392 @ 0.24099000
iex(1)> 15:48:40.215 module=gen_server [error] GenServer #PID<0.2938.0> terminating
** (Ecto.ConstraintError) constraint error when attempting to insert struct:

    * trades_pkey (unique_constraint)

If you would like to stop this constraint violation from raising an
exception and instead add it as an error to your changeset, please
call `unique_constraint/3` on your changeset with the constraint
`:name` as an option.

The changeset has not defined any constraint.


    (ecto) lib/ecto/repo/schema.ex:687: anonymous fn/4 in Ecto.Repo.Schema.constraints_to_errors/3
    (elixir) lib/enum.ex:1327: Enum."-map/2-lists^map/1-0-"/2
    (ecto) lib/ecto/repo/schema.ex:672: Ecto.Repo.Schema.constraints_to_errors/3
    (ecto) lib/ecto/repo/schema.ex:274: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
    (hefty) lib/hefty/trades.ex:70: Hefty.Trades.create_trade/1
    (hefty) lib/hefty/algos/naive/trader.ex:269: anonymous fn/5 in Hefty.Algos.Naive.Trader.handle_info/2
    (ecto_sql) lib/ecto/adapters/sql.ex:874: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1415: DBConnection.run_transaction/4
    (hefty) lib/hefty/algos/naive/trader.ex:268: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Broadcast{event: "trade_event", payload: %Hefty.Repo.Binance.TradeEvent{__meta__: #Ecto.Schema.Metadata<:loaded, "trade_events">, buyer_market_maker: true, buyer_order_id: 25394839$
iex(1)> 15:48:40.252 application=hefty module=Hefty.Algos.Naive.Leader [info] Leader restarts process as it died
iex(1)> 15:48:40.252 application=hefty module=Hefty.Algos.Naive.Leader [info] Trader found in the list of traders. Removing

Production related issue of order primary key

Cascading effect on not storing trades that happened:

iex(1)> 03:13:11.739 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(17919) received an transaction of 73.50000000 for BUY order 306907004 @ 0.25591000
iex(1)> 03:13:12.004 module=gen_server [error] GenServer #PID<0.883.0> terminating
** (Ecto.ConstraintError) constraint error when attempting to insert struct:

    * trades_pkey (unique_constraint)

If you would like to stop this constraint violation from raising an
exception and instead add it as an error to your changeset, please
call `unique_constraint/3` on your changeset with the constraint
`:name` as an option.

The changeset has not defined any constraint.

    (ecto) lib/ecto/repo/schema.ex:687: anonymous fn/4 in Ecto.Repo.Schema.constraints_to_errors/3
    (elixir) lib/enum.ex:1327: Enum."-map/2-lists^map/1-0-"/2
    (ecto) lib/ecto/repo/schema.ex:672: Ecto.Repo.Schema.constraints_to_errors/3
    (ecto) lib/ecto/repo/schema.ex:274: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
    (hefty) lib/hefty/trades.ex:70: Hefty.Trades.create_trade/1
    (hefty) lib/hefty/algos/naive/trader.ex:269: anonymous fn/5 in Hefty.Algos.Naive.Trader.handle_info/2
    (ecto_sql) lib/ecto/adapters/sql.ex:874: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1415: DBConnection.run_transaction/4
    (hefty) lib/hefty/algos/naive/trader.ex:268: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Broadcast{event: "trade_event", payload: %Hefty.Repo.Binance.TradeEvent{__meta__: #Ecto.Schema.Metadata<:loaded, "trade_events">, buyer_market_maker: true, buyer_order_id: 306907004
, event_time: 1574219590552, event_type: "trade", id: "aad67857-a5ee-4ccd-9832-307d26a4e3d5", inserted_at: ~N[2019-11-20 03:13:10], price: "0.25591000", quantity: "73.50000000", seller_order_id: 306919454, symbo
l: "XRPUSDT", trade_id: 40912131, trade_time: 1574219590491, updated_at: ~N[2019-11-20 03:13:10]}, topic: "stream-XRPUSDT"}
iex(1)> 03:13:12.005 application=hefty module=Hefty.Algos.Naive.Leader [info] Leader restarts process as it died
03:13:12.005 application=hefty module=Hefty.Algos.Naive.Leader [info] Trader found in the list of traders. Removing
03:13:12.006 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader on symbol XRPUSDT with budget of 30.0670539133375

As we can see it failed to write trade to a db and this was in the end completely ignored by the system, it started new trader and kept trading as money would never be spent..

Cased account to have insufficient founds...:

iex(1)> 09:10:42.935 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader(41042) on symbol XRPUSDT with budget of 30.0931284621375
iex(1)> 09:10:48.421 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(41042) - Placing BUY order for XRPUSDT @ 0.24914, quantity: 120.7
iex(1)> 09:10:48.969 module=gen_server [error] GenServer #PID<0.1420.0> terminating
** (MatchError) no match of right hand side value: {:error, %Binance.InsufficientBalanceError{reason: %{code: -2010, msg: "Account has insufficient balance for requested action."}}}
    (hefty) lib/hefty/algos/naive/trader.ex:643: Hefty.Algos.Naive.Trader.place_buy_order/2
    (hefty) lib/hefty/algos/naive/trader.ex:204: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

This started a cycle of doom that resulted in OS' OOM crash erlang VM:

iex(1)> 09:11:15.033 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader on symbol  with budget of 
iex(1)> 09:11:15.158 module=gen_server [error] GenServer #PID<0.1439.0> terminating
** (FunctionClauseError) no function clause matching in Hefty.Algos.Naive.Trader.place_buy_order/2
    (hefty) lib/hefty/algos/naive/trader.ex:621: Hefty.Algos.Naive.Trader.place_buy_order("0.24928000", %Hefty.Algos.Naive.Trader.State{budget: nil, buy_down_interval: nil, buy_order: nil, id: nil, pair: nil, profit_interval: nil, rebuy_interval: nil, rebuy_notified: false, retarget_interval: nil, sell_order: nil, stop_loss_interval: nil, stop_loss_triggered: false, strategy: :blank, symbol: nil, trade: nil})
    (hefty) lib/hefty/algos/naive/trader.ex:204: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Broadcast{event: "trade_event", payload: %Hefty.Repo.Binance.TradeEvent{__meta__: #Ecto.Schema.Metadata<:loaded, "trade_events">, buyer_market_maker: true, buyer_order_id: 307168352, event_time: 1574241074997, event_type: "trade", id: "c165553a-da52-4250-b2cc-8b8310b019ba", inserted_at: ~N[2019-11-20 09:11:15], price: "0.24928000", quantity: "1500.00000000", seller_order_id: 307168360, symbol: "XRPUSDT", trade_id: 40929113, trade_time: 1574241074991, updated_at: ~N[2019-11-20 09:11:15]}, topic: "stream-XRPUSDT"}
iex(1)> 09:11:15.159 application=hefty module=Hefty.Algos.Naive.Leader [info] Leader restarts process as it died
iex(1)> 09:11:15.159 application=hefty module=Hefty.Algos.Naive.Leader [info] Trader found in the list of traders. Removing
iex(1)> 09:11:15.159 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader on symbol  with budget of 
iex(1)> 09:11:15.270 module=gen_server [error] GenServer #PID<0.1440.0> terminating
** (FunctionClauseError) no function clause matching in Hefty.Algos.Naive.Trader.place_buy_order/2
    (hefty) lib/hefty/algos/naive/trader.ex:621: Hefty.Algos.Naive.Trader.place_buy_order("0.24928000", %Hefty.Algos.Naive.Trader.State{budget: nil, buy_down_interval: nil, buy_order: nil, id: nil, pair: nil, profit_interval: nil, rebuy_interval: nil, rebuy_notified: false, retarget_interval: nil, sell_order: nil, stop_loss_interval: nil, stop_loss_triggered: false, strategy: :blank, symbol: nil, trade: nil})
    (hefty) lib/hefty/algos/naive/trader.ex:204: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Naive Trader Settings' update

After naive trader settings update leader -> traders should be informed about the change and update their budgets accordingly.

Searching by symbol should be case insensitive

When searching by symbol in "Streaming Settings" page i should execute search without case sensitivity.

Currently searching for usdt returns 0 results when search by USDT returns many - both should return the same results

The ccxt standard will help you get many exchanges integrated at once. There is an elixir lib someone started to build but stopped on the way. I was thinking you might not now about ccxt and I wanted to share this with you.

https://github.com/ccxt/ccxt
https://hex.pm/packages/ccxtex

Rewrite tests as they fail in CI

Currently all tests for naive trader are setting up whole environment(BinanceMock, DummyStreamer etc) and try to publish events that will cause some specific behaviour of Naive trading strategy (buy, sell, stop loss, rebuy, retarget etc.).

All those tests are working locally but as pointed above they require processes with state existing which means that they can't be run parallel.

Other sad fact is that they run completely fine locally but intermittently fail in CI.

Opportunity here would be to refactor tests (and possibly code) so they confirm expected behaviours but they don't rely on global state (processes etc)

Error when running seed.exs

I ran into this error when getting setup with the readme process:

docker-compose up -d
mix deps.get
cd apps/ui/assets && npm install && cd ../../..
cd apps/hefty && mix ecto.reset && cd ../..

when running cd apps/hefty && mix ecto.reset && cd ../.. i get the following error

20:06:53.661 application=ecto_sql module=Ecto.Migration.Runner [info] == Migrated 20190820071000 in 0.0s
20:06:53.677 application=ecto_sql module=Ecto.Migration.Runner [info] == Running 20190911200414 Hefty.Repo.Migrations.CreateSettingsTable.change/0 forward
20:06:53.677 application=ecto_sql module=Ecto.Migration.Runner [info] create table settings
20:06:53.680 application=ecto_sql module=Ecto.Migration.Runner [info] == Migrated 20190911200414 in 0.0s
20:06:59.217 [info] Fetching exchange info to retrieve assets and symbols
** (MatchError) no match of right hand side value: {:error, {:http_error, %HTTPoison.Error{id: nil, reason: {:option, :server_only, :honor_cipher_order}}}}
    priv/repo/seeds.exs:111: (file)
    (elixir) lib/code.ex:813: Code.require_file/2
    (mix) lib/mix/tasks/run.ex:145: Mix.Tasks.Run.run/5
    (mix) lib/mix/tasks/run.ex:85: Mix.Tasks.Run.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/task.ex:365: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:292: Mix.Task.run/2
    (mix) lib/mix/task.ex:365: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:292: Mix.Task.run/2
    (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2

I looks like the migrations run fine but there is an issue with the seed.exs

is there some binance configuration that needs to be done?

Lack of dates in logs

Currently live logs contain exact time with milliseconds but they lack of date part (Y-m-d) - example:

iex(1)> 15:43:32.688 application=plug request_id=FcgEIZ4p7fcuofr4AAID module=Plug.Logger [info] Sent 200 in 115ms

iex(1)> 15:43:32.688 is missing the date.

Error when going to root_path after backtesting: count_gaining_losing

I get an error from the count_gaining_losing function when hitting the root_path (localhost:4000)

error: no match of right hand side value: [%{false: 9}, %{true: 15}]

image:

Production bug - stop loss related - race condition

After initial investigation I believe it's related to stop loss being triggered on order that was filled.

Stack trace:

iex(9)> 19:49:20.503 module=gen_server [error] GenServer #PID<0.2731.0> terminating
** (FunctionClauseError) no function clause matching in Hefty.Algos.Naive.Trader.store_order/2
    (hefty) lib/hefty/algos/naive/trader.ex:721: Hefty.Algos.Naive.Trader.store_order(%{
			"clientOrderId" => "pdqIWjicGMqKKK6V7w0l1A",
			"cummulativeQuoteQty" => "28.35532800",
			"executedQty" => "111.60000000",
			"fills" => [
				%{
					"commission" => "0.00117660",
					"commissionAsset" => "BNB",
					"price" => "0.25408000",
					"qty" => "111.60000000",
					"tradeId" => 36226832
				}
			],
			"orderId" => 251895741,
			"orderListId" => -1,
			"origQty" => "111.60000000",
			"price" => "0.00000000",
			"side" => "SELL",
			"status" => "FILLED",
			"symbol" => "XRPUSDT",
			"timeInForce" => "GTC",
			"transactTime" => 1569350960304,
			"type" => "MARKET"
		},
		246037688
	)
    (hefty) lib/hefty/algos/naive/trader.ex:616: Hefty.Algos.Naive.Trader.handle_stop_loss/4
    (hefty) lib/hefty/algos/naive/trader.ex:470: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Broadcast{event: "trade_event", payload: %Hefty.Repo.Binance.TradeEvent{__meta__: #Ecto.Schema.Metadata<:loaded, "trade_events">, buyer_market_maker: true, buyer_order_id: 251895588, event_time: 1569350959202, event_type: "trade", id: "f4193777-a972-4888-8b9e-43683f8f1ab7", inserted_at: ~N[2019-09-24 18:49:19], price: "0.25413000", quantity: "4985.80000000", seller_order_id: 251839179, symbol: "XRPUSDT", trade_id: 36226791, trade_time: 1569350959190, updated_at: ~N[2019-09-24 18:49:19]}, topic: "stream-XRPUSDT"}

Also, it's worth considering that possibly binance.ex should be extended by new fields like: fills, orderListId or cummulativeQuoteQty

cinderella-man / igthorn Goto Github PK

igthorn's People

Contributors

Stargazers

Watchers

Forkers

igthorn's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs