Comments (12)
That's true. I'll take a look at it when I get back from vacation in a week...
from xchange.
Thanks Tim.
Enjoy your vacation!
from xchange.
Hi Aido, I'm taking a look at this now. Do you know of a good way to test for a disconnect from MtGox Streaming? Also, how often does it occur by itself on average?
from xchange.
Hi Tim,
Thanks for looking at this.
Connections to the MtGox StreamingMarketDataService consistently drop every hour or so.
I am not at my computer right now but when the connection drops I get a "java.lang.IllegalStateException: Timer already cancelled" exception. I thought that if I catch this exception I could then somehow utilise the isConnected method in the socketIO class and then reconnect (http://xeiam.com/xchange/javadoc/com/xeiam/xchange/streaming/socketio/SocketIO.html#isConnected().
).
I will confirm and update this comment when I get to my development machine shortly.
from xchange.
Hi Aido,
I just built a new SNAPSHOT release (1.2.1-SNAPSHOT). What I did was add a method isConnected() to StreamingExchangeService which you can access as shown in *.streaming.TickerDemo.java.
Thanks for the info about the exception. I'll let the TickerDemo run until the connection drops and see the stacktrace for myself. If you see it, and you could post the stacktrace here, that would help as well.
In the next coming days I'd like to beef up the socketIO streaming code so that coders like you won't have to worry about catching exceptions and reconnecting. I'm going to try to come up with an elegant solution that perhaps even automatically reconnects.
Cheers, Tim
from xchange.
Hi Tim,
Excellent, thanks a lot.
I have not been able to reproduce the "java.lang.IllegalStateException: Timer already cancelled" exception. It may have been coincidental.
However, I have implemented the isConnected() method. I'm not sure if it is working properly.
When I manually bring down my internet connection get the following repeated several times in the error log:
2012-12-05 00:34:10,453 : ERROR com.xeiam.xchange.service.RunnableSocketIOEventProducer [backgroundTimer] - onError: Timeout Error. No heartbeat from server within life time of the socket. closing.
I also get the following exception:
Exception in thread "backgroundTimer" Exception in thread "backgroundTimer" java.lang.NullPointerException
at com.xeiam.xchange.service.RunnableSocketIOEventProducer.onError (RunnableSocketIOEventProducer.java:190)
at com.xeiam.xchange.streaming.socketio.IOConnection.error(IOConnection.java:437)
at com.xeiam.xchange.streaming.socketio.IOConnection$HearbeatTimeoutTask.run(IOConnection.java:159)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
The code I am using to keep the connection alive is first wait for initial connection then if connection fails try again every minute. From looking at the error messages in the error log, I don't think it is getting as far as the retry code:
exchange = com.xeiam.xchange.mtgox.v1.MtGoxExchange.newInstance();
marketData = exchange.getStreamingMarketDataService();
tickerQueue = marketData.requestTicker(Currencies.BTC, currency.getCurrencyCode());
while(!marketData.isConnected()){
try {
log.info("Waiting to connect to "+currency.getCode()+" ticker");
Thread.currentThread().sleep(Constants.ONESECOND * 10);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
.
.
.
.
.
while(!marketData.isConnected()){
try {
marketData.cancelTicker();
exchange = com.xeiam.xchange.mtgox.v1.MtGoxExchange.newInstance();
marketData = exchange.getStreamingMarketDataService();
tickerQueue = marketData.requestTicker(Currencies.BTC, currency.getCurrencyCode());
log.error("ERROR: Lost "+currency.getCode()+" ticker connection. Will try to connect again in one minute");
Thread.currentThread().sleep(Constants.ONEMINUTE);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Rather than looking to work around these errors I can wait for your 'beefier' code in the coming days.
Thanks in advance,
Aido
from xchange.
I'll take a look at this one tonight.
from xchange.
Hi Gary,
I was able to reproduce the problem by simply letting it run. It took over 3 hours, but here is the log:
07:34:15.033 [default] [main] INFO
c.x.x.m.v.s.m.s.s.MtGoxStreamingMarketDataService - requesting Ticker...
07:34:18.788 [default] [pool-4-thread-1] INFO
c.x.x.m.v.s.m.s.s.MtGoxStreamingMarketDataService - {"message":"connected"}
07:34:18.788 [default] [pool-4-thread-1] INFO
c.x.x.m.v.s.m.s.s.MtGoxStreamingMarketDataService - socketIO.isConnected(): true
07:34:19.463 [default] [pool-4-thread-1] INFO
c.x.x.m.v.s.m.s.s.MtGoxStreamingMarketDataService - {"op":"subscribe","channel":"d5f06780-30a8-4a48-a2f8-7ed181b4a13f"}
07:34:19.463 [default] [pool-4-thread-1] INFO
c.x.x.m.v.s.m.s.s.MtGoxStreamingMarketDataService - socketIO.isConnected(): true
10:51:39.012 [default] [backgroundTimer] ERROR
c.x.x.s.RunnableSocketIOEventProducer - onError: Timeout Error. No heartbeat from server within life time of the socket. closing.
10:51:39.053 [default] [pool-4-thread-1] INFO
c.x.x.m.v.s.m.s.s.MtGoxStreamingMarketDataService - {"message":"SocketIOException"}
10:51:39.053 [default] [pool-4-thread-1] INFO
c.x.x.m.v.s.m.s.s.MtGoxStreamingMarketDataService - socketIO.isConnected(): false
There are multiple things to consider, and the current implementation of the streaming socket IO code is pretty rough still. Starting from the client, if you look at *streaming.TickerDemo.java, you'll see that by calling requestTicker, you get a BlockingQueue. This is nice, but there is really no way to get any other information back such as errors that have occurred.
In MtGoxStreamingMarketDataService.java, requestTicker(), we get all the ExchangeEvents, which include the Ticker data and also any error data etc. In RunnableSocketIOEventProducer.java, I recently added code to pass a JSON snippet into the queue in all the callback methods.
All in all, the whole design needs some thought to bring it up to a production quality level.
from xchange.
Performed initial commit against the XChange-6 branch to cover this work. Can you take a look at it, Tim, and let me know if it works for you?
from xchange.
Hi Gary,
Thanks for this! I merged it into develop. I still need to create some mechanism to detect the lost connection through the messages queue and re-establish connection. Any ideas?
from xchange.
The SocketIO underlying code should automatically attempt to re-establish a connection in the event of a failure, if that fails then the DISCONNECT event type is added to the exchange event queue. Consuming applications should then be able use that to bin their market data streaming service and start again through the ticker/connect approach.
from xchange.
Grat, thanks. I'll try implementing that and hope I can test it with the disconnect events that started this feature request.
from xchange.
Related Issues (20)
- Incompatibility with Jackson 2.15.x (as used by Spring Boot 3.1.x)
- is okx thread safe? HOT 2
- large backlog of pull requests HOT 34
- BINANCE FUTURE exchangeMetaData, need some advise
- Supporting Coinbase Advanced Trading
- java.lang.NoClassDefFoundError: jakarta/ws/rs/Path HOT 4
- Binance futures URL problem HOT 5
- Add bitcoin as a topic to this repository HOT 1
- [Binance-stream] Confusion about bookTicker usage HOT 1
- Converting to BinanceKlines gets this error: java.lang.String and java.lang.Boolean are in module java.base of loader 'bootstrap' HOT 7
- BitfinexAdapters.adaptTicker(org.knowm.xchange.bitfinex.v2.dto.marketdata.BitfinexTicker bitfinexTicker) NPE HOT 5
- Invalid Symbol Error with ByBit v5 API Integration HOT 5
- [Binance] UserReference / ClientID not populated in Trades or Orders properly
- Bybit problem with Instrument conversion for options. java.text.ParseException: Unparseable date: "31MAY24" HOT 1
- Bankera exchange exists? HOT 1
- TradeOgreAdapters.adaptCurrencyPair() build a wrong CurrencyPair HOT 2
- org.knowm.xchange.exceptions.NotYetImplementedForExchangeException: getTrades of OkexStreamingExchange HOT 2
- [Binance] Kline class cast exception HOT 1
- [Bitstamp] WithdrawalRequest object does not support txid field HOT 4
- [Bitstamp] Implement client call for the V2 withdrawal-fees endpoint HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xchange.