exchange-core / exchange-core Goto Github PK

Hello,

First thanks for your contribution!
Actually I got the code from git, and tried to build, but failed.

Could you check the build status in clean workspace?

Thanks. :)

APPLICATION FAILED TO START

Description:

Field resultsConsumer in org.openpredict.exchange.core.ExchangeCore required a bean of type 'java.util.function.Consumer' that could not be found.

Action:

Consider defining a bean of type 'java.util.function.Consumer' in your configuration.

Hello, I noticed that in "ITCoreExample.java", the balance of the account can be increased. How can I get the current balance of the user after the increase is completed? Does the balance matter in the exchange-core?
I try to pass
Future<SingleUserReportResult> report1 = api.processReport(new SingleUserReportQuery(USER1), 0); report1.get().getAccounts();
Get, but return {"empty":false}

Please share inital syncup file state_0_MATCHING_ENGINE_ROUTER_0. Unable to start process without this file.

MatchingEngineRouter only support one symbol now?

Hi,

Does it have a system to recover users balances, orderbook etc. in case the matching engine crash because of server shutdown and another factors ?

Thank you !

I have a use case where user/balance/execute order systems are already built and I'm looking to only hook into a new order matching engine.

On this regard I have two questions:

1. Can I abstract user and other services from this code to only work at the buy/sell order level. Output should be transactions to execute.
2. Does this engine support 1:M matching? If there was 1 user selling 1BTC for USD and 3 users buying 0.3 BTC from USD.

Thanks and any help is greatly appreciated :)
Rohin Gopalakrishnan

It is possible to make risk engine as optional feature? Does it make sense from your perspective?
I'm trying to use your engine in distributed ledger environment.(Work like a charm, by the way) . The issue is that balances synchronization is very costly task. The idea is to use "smart contract" as a risk engine on upper layer, because balance on ledger is actual single source of truth.
What do you think? Please advice.

Now we have a question: how to ensure the accuracy of the data(account / order / balance ) when saving the snapshot?
E.g. transaction

I already have this user, balance, order. I just want to use match.

1. Use custom balance management
2. Use a custom user architecture
3. Use custom currency symbol management
   thank you.

Hi, I have trouble when I downloading maven plugins.
It seems that I could not download the file named maven-scm-provider-gitexe-1.9.5.pom.
I tried to change the access area of my network, but I still couldn't download the file.
Waiting for your reply.
Thanks a lot.

I have imported source code into eclipse but it showing error at code level like " The method builder() is undefined for the type ApiPlaceOrder","The constructor StateHashReportResult(new TreeMap<>()) is undefined". So how do compile and rune the code.

Thanks

Hi! I find your project very interesting.
Is there any plan to support more order types such as stop market or stop limit orders... or maybe iceberg ... conditional orders ... etc ...

Is it possible with your structure while keeping performance ?

Thanks

Can you please explain difference between Fast and Direct in terms of Use Cases?

Maksim, can you please explain why did you removed activeOrderId, activeOrderUid, activeOrderAction from MatcherTradeEvent?

Because from consuming perspective it's essential information.
As a core consumer I do want to know these details.

What is the difference of the codes about lamx/disruptor between in the src directory and https://github.com/LMAX-Exchange/disruptor? Why not using lamx/disruptor via maven or gradle:

or

implementation 'com.lmax:disruptor:3.4.2'

@mzheravin

I wonder how you'd do it and how it would impact performance? Won't this create a bottleneck (at least per order book)?

Hi @mzheravin
I am running test cases for checking new concepts from new changes, but I always get error as in attachment image?

Another question,
In ExchangeCore class when place an order I hold balance by orderAmount (in below image), after that matcher trade I un-hold by handling in "handlerRiskRelease" after callback resultConsumer. It is always delay (it is waiting for new summit order), so it cannot release un-hold orderAmount this here, and So I handled un-hold orderAmount in resultConsumer.

HI @mzheravin ,
Is this project supports Limit Orders and MarketOrders?

String exchangeId = "MY_EXCHANGE" ;
long snapshotBaseSeq = 0 ;
ExchangeConfiguration conf = ExchangeConfiguration.defaultBuilder()
.serializationCfg(SerializationConfiguration.DISK_JOURNALING)
.initStateCfg(InitialStateConfiguration.lastKnownStateFromJournal(exchangeId, 1, snapshotBaseSeq))
.build() ;

And then start the program. And when I add users and orders, It go to error below:
java.lang.IllegalStateException: File already exists: .\dumps\MY_EXCHANGE_journal_1_0001.ecj
at exchange.core2.core.processors.journaling.DiskSerializationProcessor.startNewFile(DiskSerializationProcessor.java:694) ~[classes/:na]
at exchange.core2.core.processors.journaling.DiskSerializationProcessor.writeToJournal(DiskSerializationProcessor.java:270) ~[classes/:na]
at com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:168) [disruptor-3.4.2.jar:na]
at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) [disruptor-3.4.2.jar:na]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_231]

I use this in my application and I have two servers, does this engine support Distributed deployment，if not ,how can I ensure the engine works when one machine down

Hi@mzheravin,
As I can see in the source code, the journaling module is served as an event handler.
All event handlers are executed in order, do you even consider IO operations in the journaling module would
slow down the performance of the whole system.

Is it better to pull the journaling module out of the event handler group and to do the persisting operations before publishing events to ringbuffer.

I tried the tests ITExchangeCoreIntegration.basicFullCycleTestExchange with a same uid(UID_1).
I want to matching my orders by myself, but get a lot of asserts.

In exchange-core project, how do I get trade details and trade price( which price offer get matched).

Thanks

Hello,
I want to use your this exchange-core.
But I dont know how to build this source.
Can you help me to build this?
Please give me guide.

I downloaded the zip file.. and I imported to downloaded project into STS 4 and I am getting too many errors in java files.

Errors Like :
The blank final field uid may not have been initialized

and there are undefined methods are being called.

I have cleaned the project and I refreshed the project. still, I am getting many errors.

Could you please tell me why I am getting these errors..
And I could you please what is java version. And in which IDE u wrote the code.

I think you are missing for holding balance:

1. for action ASK, you will pending size of quote currency, but for action BID, you must be pending size * price of base currency.
2. for market order, when action ASK like as above, but for action BID, how to choose the price for holding base currency = size * ?price

Thanks,
Huy Bui.

Is there support for pegged order types ? or support for limit orders that dynamically move with the best market quotes ?

Hi

It's awesome that your project can process 5M orders per seconds !

But in case we can adopte it like this:

• Receive new orders from apache KAFKA (Kafka can handle up to 1M messages per second)
• Send WebSockets to users in order to change browser data like: Orderbook, Latest price, Trade history etc ... (Performances depend on the number of subscribed users and topics, In my case, I am thinking to use RabbitMQ that can handle 1M requests per seconds aswell, and for each order, we can send more than 5 messages: For Seller and Buyer to show updated balance, trade history, orderbook etc...)
• Update users balances inside NoSQL in memory database like REDIS, and balance is updated each time the order of specific user is updated.

WIth all this, we can say that the number of processed orders will be decreased by more than 5x times.

Can it be customized with this features ? DO you have better approach for better performances ?

Thanks in advance :)

I have successfully installed it and build it how can I run this. Any documentations?

Hi mzheravin, I am a developer of Digital Money Corporation. I saw your open source exchange core project on github. I am very interested. I have a few questions I want to ask you.
Is the order suitable for using TreeMap?
Due to the particularly large order, it may cause TreeMap to maintain its own balance, and a large number of rotations will cause performance degradation.
2.UserProfile I will put it in the ChronicleMap, but I don't know if it works,
3. Is it appropriate to use the Chronicle Queue to replace the missing person?
I hope that you can give some advice.
Thank you.

Hi Maksim, I hava a question about the core. We are using your core as a dependency. We are trying to run the project taking our first steps to know how it works. Our first issue is related to performance. When the project start our CPU consumption goes too high (more than 500% up). Is this normal? We are running the project in development mode, using a MACBook Pro Core I5 2.9Ghz.

I´m using default configuration as your readme file says
ExchangeConfiguration conf = ExchangeConfiguration.defaultBuilder().build();
If not, could be something related to the configuration?
Or could you suggest me where we can find the possible issue?
Thanks in advance,
JC

Hi,

I noticied that the matching engine and the way to add new users, balances etc. is made from inside the same application like the example you give to us on the readme page.

Why not adding new orders, users, etc. using a requests to the API it self from outside the application ?

This will make it easy to manage the exchange.

I already found an example of REST program related to exchange-core but it is for an old version:
https://github.com/exchange-core/exchange-gateway-rest

Thank you

HI Team,
I am setting up the exchange-core on my local machine. During build, am getting below error.

"The method get(int, Supplier) is ambiguous for the type ObjectsPool." position = objectsPool.get(ObjectsPool.SYMBOL_POSITION_RECORD, SymbolPositionRecord::new);
Class - RiskEngine.java, line 359.

Please help me in resolving this issue.

Regards
Abhishek

RiskEngine.java line 530, probably bug: freeMargin = position.estimateProfit(spec, lastPriceCache.get(spec.symbolId));

freeMargin += position.estimateProfit(spec, lastPriceCache.get(spec.symbolId)); ?

Could you provide a simple example on the usage of the library.
Just create a 2-3 users
add some bids
get the order book list
get the matches of the bids

I just build it and try to use it on an app, it is not so clear how to do those steps.

Thank you.

In exchange-core project, where orders are store? If I keep the order inside a database then what should I do?

Thanks

Hello, can in your code more explanation, so easy to understand。I can write instructions in Chinese

I'm trying to run some of the tests and I get the following error: "java: cannot access sun.misc.Unsafe"

Also, running "mvn -Dtest=PerfThroughput#throughputTest test" gives me the following error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M3:test (default-test) on project exchange-core: No tests were executed!

Thank you.

For the moment, we would like to validate the full cycle exchange considering our user cases. So, I am moving forward with other tests.

Im trying to implement a few user cases to start to test our user cases and learn how the engine works internally:

1. Exchange configuration
   ExchangeCore exchangeCore = ExchangeCore.builder()
   .resultsConsumer(eventsProcessor)
   .serializationProcessorFactory(serializationProcessorFactory)
   .exchangeConfiguration(conf)
   .build();

2. Adding users to exchange.
   We are creating users in our own backoffice. We will push the UID required mapping it with our own ID. api.submitCommandAsync (ApiAddUser.builder ()). Is working well.
   We added two users UID 301 and UID 302 following your examples.

3. Balance. We are managing deposit and withdrawal into the our own backoffice. We need to maintain syncronized the balance in real-time considering trades and deposit-withdrawals. We are mapping it to the api function api.submitCommandAsync (ApiAdjustUserBalance.builder (). Is working well.

4. Currency and Symbol definition:
   I would like to define once currency pair to start. So, we are defining BTC-USD using CoreSymbolSpecification as follow:

public static final int SYMBOL_EXCHANGE_FEE = 9340;
public static final int COIN_XBT = 3762; // satoshi, 1E-8
public static final int COIN_USD = 840;

public static final CoreSymbolSpecification SYMBOLSPECFEE_XBT_USD = CoreSymbolSpecification.builder ()
.symbolId (SYMBOL_EXCHANGE_FEE)
.type (SymbolType.CURRENCY_EXCHANGE_PAIR)
.baseCurrency (COIN_XBT) // base = satoshi
.baseScaleK (1_000_000) // 1 lot = 1M satoshi (0.01 BTC)
.quoteCurrency (COIN_USD) // quote = usd
.quoteScaleK (1) // 1 step = 1 usd
.takerFee (0) // taker fee 0 usd per 1 lot
.makerFee (0) // maker fee 0 usd per 1 lot
.build ();

Is this configuration right? I don´t understood so good how baseScale should be configured.

I need to add different FIAT currencies. For ARS (Argentinian pesos) and USD (American Dolares) to be exchanged for BTC and USDT (Tether).

Could I configure the takerFee and makerFee as porcentual? I need to charge 0,25% for each transaction as commission.

5. Place an order Limit GTC.
   We are having problems placing orders.

Our JSON definition:

"type":"OE",
"data": {
"uid":301,
"orderId": 3432342343534,
"price": 9344,
"size": 1,
"symbol": "9340",
"type": "GTC",
"action": "BID"

We tried two ways to do it.

a) With RiskEngine enabled (default config) but is not working.
matching_1 | 2020-05-19 20: 04: 06.741 WARN 1 --- [Thread-R1_0] e.core2.core.processors.RiskEngine: 3432342343534 risk result = RISK_NSF uid = 301: Can not place OrderCommand (command = PLACE_ORDER, orderId = 3432342343534, symbol = 9340, price = 9344, size = 1, reserveBidPrice = 9344, action = BID, orderType = GTC, uid = 301, timestamp = 0, userCookie = 0, eventsGroup = 4, serviceFlags = 1, resultCode = NEW, matcherEvent = null, marketData = null)
matching_1 | 2020-05-19 20: 04: 06.747 WARN 1 --- [Thread-R1_0] e.core2.core.processors.RiskEngine: 3432342343534 accounts: {3762 = 1, 840 = 20}
matching_1 | Command result: IEventsHandler.ApiCommandResult (command = [ADD o3432342343534 s9340 u301 B: GTC: 9344: 1], resultCode = RISK_NSF, seq = 6)

b) With RiskEngine disabled. We are having compilation problem.
We are using your latest version (0.5.0) and we added the configuration with corresponding dependencies.

OrdersProcessingConfiguration.NO_RISK_PROCESSING. I have the follow message as compilation result:

cannot find symbol
[ERROR] symbol: variable NO_RISK_PROCESSING
[ERROR] location: class exchange.core2.core.common.config.OrdersProcessingConfiguration

At the moment we can’t move forward because of these conflicts, I would really appreciate if you can help me to figure out how to fix it!. I don’t sure if must post it as issue on Git because could be only problems in my side.

Thank a lot for your support!

Hello, I'm also a java engineer. I'm really happy to find a matchmaking system. Thank you very much for your contribution! Can you explain the operation principle of the system, such as how to ensure the data security when the system crashes, can you explain it briefly? Thank you very much

Hi Maksim, I hava a question about the core. We are using your core as a dependency. But when starting the project throw a exception " Can not access Disruptor internals" .

Hello, I see that the data type of "price" and "size" of the order is "long". Is there a problem if the value exceeds the maximum value of "long"? For example, the accuracy of "ETH" is 18 (WEI). If 10 "ETH" is "10000000000000000000", it will be greater than the maximum value of "long". Is there a solution, or am I misunderstanding?

Looking forward to your reply. Thank you

Hi

I'm trying to use this project as a maven module in another project, however I'm getting the below error which seems to be related to the Lombok Builder annotation.

I'm compiling exchange-core using `mvn clean install' .

java.lang.Error: Unresolved compilation problems: 
	The type org.openpredict.exchange.core.ExchangeCore$ExchangeCoreBuilder cannot be resolved. It is indirectly referenced from required .class files
	The method builder() from the type ExchangeCore refers to the missing type ExchangeCore$ExchangeCoreBuilder

Is there are more correct maven command I should use?

I'd like to run two of these exchange cores, one as the master and one as backup (shadowing the master) in case the first one fails. If each of the matchers are given the same input of orders, would the expected output be the same?

In multi-threaded environment, getting java.util.ConcurrentException in OrderBucketNative class for below line.
final Map.Entry<Long, Order> next = iterator.next();

In this class LinkedhashMap is being used to collect and remove all matching orders. So can I use any Concurrent Collection instead to avoid ConcurrentException.

Thanks

Exchange-core includes:

• orders matching engine
• risk control and accounting module
• disk jouraling and snapshots module
• trading, admin and reports API

Can I fix it? : )

such as master-slave mode?