Modbus/Serial Driver Changes on 3.6 under Linux?

phildunlap

Hi Turbo,

Hmm. The packaged version of Modbus4J did go from 3.0.4 to 3.0.5 so there were a few changes. But, that it works sometimes and after restarts and whatnot is confusing detail.

When does it not work? After some amount of time, some percentage of restarts, etc
Does this apply even in cases like (6) where the restart seems to make the responses appear?

If it is a serial port issue, I think it could be interesting to see from the command line what the setting of the serial port is when in each state to see if the port is being configured differently: stty -F /dev/ttyUSB0 -a

I found this issue while investigating this question, and it would be a difference between 3.5 and 3.6, so perhaps it is the issue at hand: https://github.com/infiniteautomation/ma-core-public/issues/1497

If so, thanks for bringing it to our attention! Perhaps you could use stty command to manually tinker with the flow control settings when you are observing the issue.

Turbo

Heya:

I appreciate you looking into this. I also noticed a new update to Mango's Modbus library. Does this address this issue?

Do you want me to still look into stty output for my configurations?

Cheers,
-Greg

Turbo

Ok:

Since I'm logged in anyways, here we go:

stty -F /dev/ttyUSB1 -a
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z;
rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 0; time = 0;
-parenb -parodd -cmspar cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff -iuclc -ixany -imaxbel -iutf8
-opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
-isig -icanon -iexten -echo -echoe -echok -echonl -noflsh -xcase -tostop -echoprt -echoctl -echoke -flusho -extproc

After disable inside Mango:
stty -F /dev/ttyUSB1 -a
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z;
rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 0; time = 0;
-parenb -parodd -cmspar cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff -iuclc -ixany -imaxbel -iutf8
-opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
-isig -icanon -iexten -echo -echoe -echok -echonl -noflsh -xcase -tostop -echoprt -echoctl -echoke -flusho -extproc

After re-enable inside Mango:
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z;
rprnt = ^R; werase = ^W; lnext = ^V; discard = ^O; min = 0; time = 0;
-parenb -parodd -cmspar cs8 hupcl -cstopb cread clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff -iuclc -ixany -imaxbel -iutf8
-opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
-isig -icanon -iexten -echo -echoe -echok -echonl -noflsh -xcase -tostop -echoprt -echoctl -echoke -flusho -extproc

cwangv

@turbo
Greg
Did you see a lot of events being generated for this modbus device? Such as lot of 'RTN' events popping up?

Turbo

Nope.

Right now, the device is working fine. If I turn it OFF and ON again within Mango, it just shows the data as uncertain.

I did disable the events for this, though, as I generally don't need my database flooded with lost poll type numbers.

That all being said, it's working Fine Now, so long as I don't use the Enable/Disable Data Source buttons.

phildunlap

All those stty outputs are the same. Were those correlated with the data source not working?

Turbo

Yeah.

I was just about to go look into it again, and one of my computers in the field fell over again.

Ugh:

Anyways: Was the big you found earlier up in this post fixed?

phildunlap

I described it in the git issue, but it was that flow control was being improperly set. That doesn't seem to me that it would be the issue if toggling the data source resolves it, as it would be passing the incorrect flow control every time if that were the issue. It was fixed, but there hasn't been a core release in that time, so it's not released.

If it's not the serial port configuration, it could be the adapter or the noise on the wire. Or, it could be the modbus data source is somehow stuck, but I wouldn't expect that if you are able to disable it. Certainly it would have to finish the poll it was attempting.

Perhaps you can take a thread dump when you observe this occurring?

You could perhaps apply a band-aid of an event handler to restart the data source, if it's giving grief.

Turbo

Toggling the data source does not resolve it.

Restarting the Mango instance does.

When Mango starts, it works, fine, for apparently long periods of time. If you toggle it off and on again, it does not come back online. Only the ? shows up on data, and the Modbus tools fail to work.

If you restart all of Mango (or restart the computer entirely) it comes up on the first startup.

Turbo

Okay.

Just was verifying this: What log files can I send you to verify the behavior I'm seeing and help you out?

(1) On startup, the Data Sources come up Just Fine, and we're all solid.
The system has been running here since Tuesday of last week with data coming in fine, an no problems reported.
(2) If I toggle any single Data source, it causes all the data sources on the same port to fail. The polls are still going out, but the returns are not getting seen by Mango. I have Modbus/Serial data sources defined one data source per inverter, meaning in this case there are (5) Data sources all using /dev/ttyUSB1. They are all the same serial settings, with the only difference being the modbus address used.
(3) The only way to get that device back up again is to restart mango, at which point everything is Fine.

It looks as if disabling a Single Data Source disables the entire port-- In looking at this now, I'm seeing the same behavior I previously described on BOTH ports (USB1 and USB0).

Previously, in Mango, if you disabled, say, Data Source A on /dev/ttyUSB0, it just turned off Data Source A (stopped polling). With 3.6, it appears that disabling Data Source A disables all data sources using that particular port. Is this the way this behavior is supposed to be?

phildunlap

I sent you an email. I was not able to replicate an issue with two data sources sharing a modbus serial connection and disabling them. No it is not the expected behavior.