Read/write timeout about tokio-modbus

[uklotzde]

A complete Modbus RTU example with timeout handling and error recovery can be found here:
https://github.com/slowtec/truebner-smt100/blob/master/examples/modbus-rtu.rs

[flosse]

I think you can can set the timeout within the SerialPortSettings :)

[aleksey-r]

Does not work.
Regardless of the timeout set, the program hangs until it receives a response from the device.

Cargo.toml

[dependencies]
futures = "*"  
tokio = "*"  
tokio-core = "*"
tokio-serial = "*"
tokio-service = "*"

[dependencies.tokio-modbus]
version = "*"
default-features = false
features = ["rtu"]
git = "https://github.com/slowtec/tokio-modbus"
branch = "master"

main.rs

extern crate futures;
extern crate tokio_core;
extern crate tokio_modbus;
extern crate tokio_serial;
extern crate tokio_service;
use tokio_modbus::*;
use futures::future::Future;
use tokio_core::reactor::Core;
use tokio_serial::{Serial, SerialPortSettings};
use std::time::Duration;

pub fn main() {
    let mut core = Core::new().unwrap();
    let handle = core.handle();
    let tty_path = "COM1";
    let server_addr = 0x01;

    let mut settings = SerialPortSettings::default();
    settings.baud_rate = 19200;
    settings.timeout = Duration::from_millis(500);
	
    let port = Serial::from_path_with_handle(tty_path, &settings, &handle.new_tokio_handle())
        .expect(&format!("Unable to open serial device '{}'", tty_path));

    let task = Client::connect_rtu(port, server_addr, &handle).and_then(|client| {
        println!("Reading a sensor value");
        client
            .read_holding_registers(0x082B, 2)
            .and_then(move |res| {
                println!("Sensor value is: {:?}", res);
                Ok(())
            })
    });

    core.run(task).unwrap();
}

[uklotzde]

I can confirm this on Linux after adding a timeout to the rtu-client example.

[uklotzde]

The timeout works as expected in serialport-rs. But in mio-serial read timeouts have been disabled:

berkowski/mio-serial#10

The SerialPort implementation in mio-serial simply returns a duration of 0 for timeout(), independent of the timeout in SerialPortSettings.

[uklotzde]

This is still an open issue. Using a timeout at the service level (or above) doesn't work as expected for serial ports. As mdonoughe pointed out tokio-proto's pipeline mode gets stuck and the client context becomes unusable. The issue is easy to reproduce by sending requests to disconnected slave devices that never respond.

@mdonoughe Do you have a working example on how to properly handle timeouts on serial ports?

[mdonoughe]

I have something but it's definitely not proper.

I created a wrapper around tokio-serial which uses tokio-timer to emulate read timeouts in userspace.

Unfortunately, tokio-proto gets stuck after the first error occurs, so I ended up creating a second wrapper which issues revocable proxies to the actual transport. On the other side of tokio-modbus if I get an error I just rtu::connect_slave with a new proxy to get a clean tokio-proto state.

The code for the two wrappers isn't that bad, besides possibly being over-engineered, but the program containing them is a hack to control desks based on LTC302 motor controllers, using commands guessed based on a logic analyzer dump of what happens when pressing buttons on the control panel, and it barely works so I haven't published it yet.

tokio-proto is archived and deprecated in favor of tokio-rs/tokio-tower (tokio-rs/tokio#118) but tower is described as "WIP" and does not even have a readme file yet so it's probably not time to switch.

[uklotzde]

This is the brute-force method I also had in mind. But I hesitated to implement it, because it requires a substantial re-design and I hoped there could be a smarter solution.

[uklotzde]

Getting rid of the legacy tokio stuff is our goal.

[uklotzde]

The following repository contains an example that demonstrates how to reliably handle timeouts by reconnecting the serial port:

https://github.com/slowtec/truebner-smt100

[2019-04-06T10:10:54Z INFO  modbus_rtu] Measurements { temp: Some(Measurement { ts: 2019-04-06T10:10:54.502013602Z, val: Temperature(20.32) }), vwc: Some(Measurement { ts: 2019-04-06T10:10:54.599540976Z, val: VolumetricWaterContent(0.0) }), perm: Some(Measurement { ts: 2019-04-06T10:10:54.695348466Z, val: RelativePermittivity(1.0) }) }
[2019-04-06T10:10:55Z INFO  modbus_rtu] Measurements { temp: Some(Measurement { ts: 2019-04-06T10:10:55.494739896Z, val: Temperature(20.33) }), vwc: Some(Measurement { ts: 2019-04-06T10:10:55.590601744Z, val: VolumetricWaterContent(0.0) }), perm: Some(Measurement { ts: 2019-04-06T10:10:55.686530311Z, val: RelativePermittivity(1.0) }) }
[2019-04-06T10:10:56Z INFO  modbus_rtu] Measurements { temp: Some(Measurement { ts: 2019-04-06T10:10:56.501411582Z, val: Temperature(20.32) }), vwc: Some(Measurement { ts: 2019-04-06T10:10:56.597277162Z, val: VolumetricWaterContent(0.0) }), perm: Some(Measurement { ts: 2019-04-06T10:10:56.692580367Z, val: RelativePermittivity(1.0) }) }
[2019-04-06T10:10:58Z WARN  modbus_rtu] Reconnecting serial port /dev/ttyUSB0 after error: reading permittivity timed out
[2019-04-06T10:10:58Z INFO  truebner_smt100::modbus::rtu] Connecting to serial port /dev/ttyUSB0
[2019-04-06T10:10:58Z INFO  modbus_rtu] Measurements { temp: Some(Measurement { ts: 2019-04-06T10:10:58.500550131Z, val: Temperature(20.31) }), vwc: Some(Measurement { ts: 2019-04-06T10:10:58.612426646Z, val: VolumetricWaterContent(0.0) }), perm: Some(Measurement { ts: 2019-04-06T10:10:58.724233908Z, val: RelativePermittivity(1.0) }) }
[2019-04-06T10:10:59Z INFO  modbus_rtu] Measurements { temp: Some(Measurement { ts: 2019-04-06T10:10:59.507167892Z, val: Temperature(20.34) }), vwc: Some(Measurement { ts: 2019-04-06T10:10:59.619242006Z, val: VolumetricWaterContent(0.0) }), perm: Some(Measurement { ts: 2019-04-06T10:10:59.731071098Z, val: RelativePermittivity(1.0) }) }
[2019-04-06T10:11:00Z INFO  modbus_rtu] Measurements { temp: Some(Measurement { ts: 2019-04-06T10:11:00.498118521Z, val: Temperature(20.34) }), vwc: Some(Measurement { ts: 2019-04-06T10:11:00.609392695Z, val: VolumetricWaterContent(0.0) }), perm: Some(Measurement { ts: 2019-04-06T10:11:00.706263056Z, val: RelativePermittivity(1.0) }) }
[2019-04-06T10:11:01Z INFO  modbus_rtu] Measurements { temp: Some(Measurement { ts: 2019-04-06T10:11:01.505252206Z, val: Temperature(20.32) }), vwc: Some(Measurement { ts: 2019-04-06T10:11:01.616445783Z, val: VolumetricWaterContent(0.0) }), perm: Some(Measurement { ts: 2019-04-06T10:11:01.728911489Z, val: RelativePermittivity(1.0) }) }

[uklotzde]

The proposed solution causes new issues and works only if the serial port is not used exclusively. When the port is shared (as in the example) input data may get consumed by multiple client services resulting in incomplete and dropped frames. Not always, but reproducible.

I'm still looking for a solution to get rid of stale client bindings from tokio-proto that seem to occupy the serial port forever, even when dropping the context. Stopping and restarting the core runtime would be the very last resort. Any ideas welcome.

[uklotzde]

A workaround for reliably dropping stale client connections will hopefully be available soon: #24

Successfully tested with exclusive serial port connections. The port can be reconnected and used for connecting a new client context after dropping this poison pill.

[flosse]

@aleksey-r I think we can close that issue now. Feel free to reopen it if there are questions left.